The following content is

provided under a Creative Commons license. Your support will help MIT

OpenCourseWare continue to offer high-quality educational

resources for free. To make a donation or view

additional materials from hundreds of MIT courses, visit

MIT OpenCourseWare at ocw.mit.edu. PROFESSOR: Hello. Welcome. Today we’re going to talk about

one last new topic which has to do with search. So as you remember, we’re

working on our last topic. The last topic was probability

and planning. Last lecture we talked

about probability. Mostly we focused on Bayes’

theorem, Bayes’ rule. That was a way of updating our

belief about some situation based on new information. And this week in Design Lab 12,

you’ll get a chance to use that in a robot application. The idea to Design Lab 12 is

going to be that a robot is pedaling along a corridor

with some obstacles off to its left. And the idea will be, you don’t

know where the robot is, but the robot will be able to

estimate where it is by the signals that it receives from

its left-facing sonars. So this is a very realistic

type of state-estimation problem. The idea is going to be that at

any given time, t, you have access to your previous belief

at time t minus 1. And you have access to a new

observation, which is the sonar to your left. And based on those two bits of

information, you will update your belief. Which means that when you start

out you’ll have no idea where you are, but that

situation should improve with time. Ok. So that’s what we’re going

to do in Design Lab 12. Today we’re going to blast

ahead and think about the other important topic,

which is search. So we’re going to think

about planning. We’re going to be

planning ahead. We’re not going to just

react to the situation that’s given to us. We’re going to try to figure out

what’s the right thing to do in the future. And to do that, we’re going to

think about all the things we could possibly do, search

through that space, and figure out the one that’s, quote,

“best.” And we’ll have to define “best” somehow. Just to get going, I want to

show you a very simple kind of a search problem. This is called the

eight puzzle. The idea is to make a plan to

go from this configuration, which we’ll call the state, to

this configuration, which we’ll call the goal state. And on each move, you get to

move one of the tiles into the free space. So I could move the 8 to the

right or the 6 down. Those are the only two things I

could do in the start state. So I have to make up my

mind which of those I would like to do. And I would like to believe

that ultimately, after a series of moves, I’m going to be

able to control this state into that state. And you can imagine guessing. And we’ll estimate in a moment

how big is the guess space. I mean, if the guess space only

had like four elements in it, guessing’s a

fine strategy. If the guess space has a lot

more elements than that, guessing’s probably

not a good idea. So I previously ran our search

algorithms, the ones that we’ll develop during lecture,

on this problem. And here’s the solution

that our search algorithm came up with. It’s not exactly what you might

do the first time you touched it. OK, I made it. If you were counting,

I made 22 moves. The question is, how difficult

was that problem and how good was that solution? Was that a good solution

or a bad solution? Is there a better solution? How much work did I have

to do in order to calculate that solution? To get a handle on that,

let’s start by asking a simple question. How many configurations

are there? I got there in 22. What was the space of things

I had to look through? How many different board

configurations exist? So think about that

for 20 seconds. Talk to your neighbor. And figure out whether it

is 8-squared, 9-squared, 8-factorial, 9-factorial,

or none of those. So how many bar configurations

do you see? Raise your hand, show me a

number of fingers, so I can figure out roughly how people

were– that’s excellent. Very good participation, and

nearly 100% correct. The answer is number (4). You can think about this. What if you took all the tiles

out and threw them on the floor, and then put them

in one at a time? Well, you would have 9

possibilities for where you wanted to put the first one. Then you would have 8

possibilities for where you wanted to put the second one. Then you’d have 7 possibilities

for where you put the third one, et

cetera, et cetera. Even though the space doesn’t

have a number on it, it still sort of counts. And so you end up with

9-factorial. And the point is, 9-factorial

is a big number. 9-factorial is 362880. So if you thought about simply

guessing, that’s probably not going to work all that

well, right? Even if you guessed,

you have on each– there’s a third of a million

different configurations that you have to look at. And that’s if you didn’t

lose track of things. If you lost track of– Oh, my. It’s coming up– almost, anyway. It looks like it’s chopped

off at the top. So ignore that for now. Look over here. So even if you didn’t confuse

yourself, there’s a space of a third of a million things

to look at. And if you confused yourself,

there’s even more. So it’s not a huge problem by

computer science standards. But it’s certainly not

a trivial problem. It’s not something that you can

just guess and get right. So what we want to do today is

figure out an algorithm for conducting a search like that. We’d like to figure out the

algorithm, analyze how well it works, optimize it, and try to

find out a way to find the best solution, where “best” for

this particular problem would mean minimum

path length. So figure out the best solution

by considering as few cases as possible. Obviously, if you enumerate

all the cases, that should work. The problem is, it will be

interesting to solve problems where that enumeration

is quite large. Even here, the enumeration

is quite large. So let’s think about

the algorithm. And I’ll think about the

algorithm by way of an even simpler, more finite problem. What if I thought about a grid

of possible locations where I could be. Maybe this is the intersections of streets in Manhattan. I want to go from point A

to point I. What’s the minimum-distance path? I hope you probably can

all figure that out. What I want to do is write

an algorithm that can figure that out. And then if we write the

algorithm well, we’ll be able to use it for the tile problem,

which is not quite so easy to do. The way we’re going to think

about doing that is to organize all of our possible

paths through that maze, in a tree. So if I started at A, I have

a decision to make. I could either go to B or D. Then if I went to, say, B, I

could either then go to A or C or E. I’ve organized them

alphabetically for no particularly good reason. Just– I needed some order. Then if I went from A to B to A,

say, then I could either go from A to B or D. That’s

illustrated here. So the idea then–

ah, it works. So now it looks like they’re

all three working. So the idea is, think about

the original problem. The original problem is

find the shortest path through some grid. I want to go from A to I. And

I’ll think about all the possible paths on a tree. Then the problem is that for

the kinds of problems we’re going to look at, that tree

could be infinite in length. Oh, that’s a bummer. That means that the strategy of

building the tree and then searching it is probably

not a good strategy. So what we’ll do instead is,

we’ll try to write the algorithm in such a way that

we construct the tree and search for the best solution

all in one pass. Then hopefully, if we find a

solution in some finite number of steps, we’ll only have

built part of the tree. But we’ll have built the part

that has the answer. The idea, then, is going to be,

think about what is the path we want to take, by

thinking about the tree of all possible paths. But what we want to do is write

code that will construct the tree on the fly, while it’s

considering how good were all the different nodes. Ok. So how are we going

to do that? We’ll be working in Python,

not surprisingly. We’ll represent all the

possible locations. We’ll call those states. So the problem will have states

A, B, C, D, and we’ll just represent those

by strings. That makes it flexible. That makes it arbitrary. Then we’ll think about

transitions, not by enumerating them. Remember, we don’t want to

enumerate them, because there could be infinitely

many of them. So how’s the other way

we could do it? Well, we’ll embody that

information in a program. We’ll write a procedure called

“successor” that will, given the current state and action,

figure out the next state. So that’s a way that we can

incrementally build the tree. So imagine here, if I started

in A and I executed action 0 or 1, I would end up in

B or D, respectively. So I tell you the current state

and the current action, and the successor program

then will return to you the new state. That’s all we need to construct

the tree on the fly. Then, to specify the particular

problem with interest, I have to tell

you where you start. So I have to define

initial state. And I have to tell

you where to end. I could just tell you

the final state. But in some of the problems of

the type that we will want to do, there could be multiple

acceptable answers. So I don’t want to just give

you the final state. I’ll give you a test. I’ll give you another procedure,

called “goalTest.” And that goal test, when passed

an input which is a state, will tell you whether or

not you reached the goal. That way, for example, all the

even-numbered squares could satisfy the goal, if that were

the problem of interest. Or all the states on the right

could satisfy the goal. It’s just a little bit

more flexible. The idea, then, is that in order

to represent that tree, we’ll do it by specifying a

procedure called successor and specifying the start state

and the goal test. So here’s how I might set that

up for the simple Manhattan problem that I showed–

that I started with. So I want ultimately to have

something called successor that eats a state

and an action. I’ve built the structure of

Manhattan into a dictionary. The dictionary lists

for every state– A, B, C, D, E, F, G, H, I– for every state, it associates

that state with a list of possible next states. So if I’m in A, I could next be

in B or D. I could next be in B or D. I’ve organized these arbitrarily

in alphabetical order so I can remember

what’s going on. So the next states are all

in alphabetical order. The number of next states

depends on the state. I’m not going to worry

about that too much. I’m just going to specify the

action as an integer– 0,1,2,3 — however many I need. So the possible actions are

taken from that list. The possible action might be

do action 0, do action 1, do action 2. So if I did action 2 starting

on state E, I would go to– so action started at 0 — so 0, 1, 2 — I would go to state F. OK? Is that all clear? The initial state is A, and the

goal state is, return S equal to I. So if S is equal

to I, it returns True. If S is not equal to I,

it returns False. I’m not quite done. That’s enough to completely

specify the tree, but now I have to build the

tree in Python. Not surprisingly, from our

object-oriented past, we will use an object-oriented

representation for that tree. So we’ll specify every node in

the tree as an instance of the class SearchNode. SearchNode is trivial. SearchNode simply knows,

what was the action that got me here? Who’s my parent? And what’s my current state? So when you make a new node,

you have to tell the constructor those

three things. What was the action

that got me here? What’s my current state? And who is my parent? Knowing the node, you’re

supposed to know the entire path that got you here. So we’ll also add a method

which reports the path. So if I happen to be in node

E, my path ought to be I started in A, I took action 0

and got to B, and then I took action 2 and got to E.

This subroutine is intended to do that. If my parent is “none,” which

will happen for the initial state, simply report that the

path to me is none, A. However, if I’m anybody other

than the initial node, if I’m other than the initial node,

then figure out the description of the path to my

parent, and add the action that got me here,

and my state. So that’s what this is. OK, so what are we doing? We specify a problem by telling

you the successor function, the start state,

and the goal test. Then we provide a class by which

you can build nodes to construct, on the fly,

the search tree. Now we’re ready to write

the algorithm. Here’s the pseudocode

for the algorithm. What do we do? We initialize– so we’re going to be

doing a s– oh, this is very confusing. I’m trying to solve

a search problem. To solve the search problem,

I’m going to search through the tree. So I’m going to think about the

state of my search through the tree by way of something

we’ll call the agenda. Very jargon-y word. I completely apologize for it. I didn’t invent it. It’s what everybody calls it. Sorry. The agenda is the set

of nodes that I’m currently thinking about. So I’ll initialize that to

contain the starting node. Then I’ll just systematically

keep repeating the same thing over and over again. Take one of the nodes out of

the agenda, think about it, replace that node

by its children. While I’m doing that, two things

are supposed to happen. I’m supposed to construct

the search tree. But I’m also going to be looking

over my shoulder to see if I just constructed a

child who is the answer. Because if I just constructed

the answer, I’m done. Ok. So initialize the agenda

to contain just the starting node. Then repeat the following

steps. Remove one node from

the agenda. Add that node’s children

to the agenda. And keep going until one

of two things happens. Either you found it– goal test returned True– or the agenda got empty,

in which case there must not be a solution. If I’ve removed all of my

options and still haven’t found anything, then there’s

no solution. Ok. So what’s the program

look like? It’s actually remarkably simple,

especially when you think about just how hard

the problem is. Imagine if you wanted to do that

tiles problem with a very simple-minded “if this then

this, if this then this.” We’re talking about a third

of a million ifs, right? That’s probably not the

right way to do it. This program is going to end

up being about this long. It’ll fit on this page. And it’s going to be able to

handle that case, or even harder cases. Define the search procedure. The search procedure is

something that’s going to take the initial state, the goal

test, the possible actions, and the successor sub routine,

the successor procedure. That’s everything you need

to specify the problem. And it’s going to return

to me the optimal path. First, step (1). Initialize the agenda to

contain the start node. I want to put the

start node in. Well, there’s a chance– I want this procedure to

be general purpose– there’s a chance that that

start node is the answer. So take care of that first. If you’re already there,

return the answer. The path to the answer is me. I’m trying to create

the agenda. I’m trying to put the first

node into the agenda. There’s a chance that first

node is the answer. If that first node’s the answer,

return the path to me. Which is, take no action. You’re here. But that’s not likely to be

the case for the kinds of questions we ask, in which case

we will create a list that contains one node, which

is the node that represents the start node. Then, repeat ‘remove

a node’ — which we’ll call the parent– from the agenda. And substitute — replace that node that we pulled

out of the agenda — replace that with

the children. While not empty of agenda– empty is some kind of a

pseudo-routine that I’m going to fill in, in a minute– while the agenda is not empty,

get an element out of the agenda, which we’ll

call the parent. Then I want to think about

all the children. Well, there’s a list of

possible actions. So for a in actions, do

the following things. Each parent can have multiple

children, one for every possible action. So for a in action, figure out

what would be the new state. The new state is just the

successor of the parent state. Remember, the parent

is a node, right? The parent is a node. We’re constructing nodes

in the search tree. But nodes know their state. So figure out the new state,

which is the successor of my parent, the guy that I pulled

out of the agenda. Make a new node, which

corresponds to this child. Then ask the question,

did the new state satisfy the goal test? If it did, the answer is the

path to the new node. Ok. So create a new state, which is

the successor of the parent under the given action

A. Create a new node. See if it’s the end. If it is, just return,

I’m done. Return from search. Otherwise, add it– again, one

of these pseudo-procedures. We’ll fill that in,

in a minute– add the new node into

the agenda. There’s several things

that could happen when I run this loop. If the node has no children, I

will take out the parent and not put anything back in. If the node has multiple

children, I could take out one node and put in more nodes than

I took out, so the agenda could get longer. So the agenda could either

increase in length or decrease in length as a result of

spinning around this loop. Also, we could either identify

a goal or fail to identify a goal. So as it’s increasing and

decreasing, we either will or won’t find an answer. Ok. Now the trick– the only thing that makes

this complicated– is that order matters. So those pseudo-operations,

whatever they were– get element and add– exactly how I get element and

exactly how I add it to the agenda affects the way

I conduct the search. Let’s think of something

very simple. Let’s always remove the first

node from the agenda and replace it by its children. So pull out the first node. And put back into the beginning

of the agenda the children of the first node. How? I would start out in step (0). I would put the start node

into the agenda. So now there’s one element in

the agenda, the start node. Then, on the first pass

through the loop, I would pull out that. That’s the first node

in the agenda. There’s only one node

in the agenda. That is the first one. Pull out the first one and

replace it by its children. Its children are AB and AD. I’m representing the nodes in

this notation by the path, because the same state

can appear multiple times in the tree. Notice that I could

walk ABAB — which would correspond to

the same state being repeated in the tree. So I can’t, when I’m writing

it down here, represent the node by a state. But I can represent

a node by a path. So on the first pass through the

loop, pull out the first item in the agenda, which

is A, and push back that A’s children. Well A’s children

are AB and AD. Ok. So now on the second pass, the

rule is pull out the first guy and replace it by

the children. Now the first guy is AB. So I’m here. So pull that guy out and replace

him by his children. His children are ABA,

ABC, and ABE. AD is left over. The number of elements in

the agenda got bigger. Next step, pull out the first

item in the agenda, replace it by its children. The first item in the

agenda is ABA. The children of ABA

are ABAB and ABAD. Ok. Notice the structure

of what’s going on. Ignore the stuff on the bottom,

and just watch the picture on the top. So I start by putting A in the

agenda, then its children, then its children, then

its children. So when I implemented

the algorithm– take out the first and replace

it by its children– I’m searching along

the depth first. I’m going deeper and deeper into

the tree without fully exploring all the horizontal

spaces. So I’m tracing a line

down that way. If you imagine this tree– I’ve only represented the first

three layers of nodes here– this tree goes

on forever. It’s an infinite tree, because

you can walk around in that Manhattan grid forever. There’s no limit to how long

you can walk around. So although I’m only listing the

first three, the tree, in principle, goes on forever. And this algorithm will have

the feature that it walks along the left edge. We call that depth-first search

because we’re exploring depth first, as opposed

to breadth. That results because

of our rule. The rule was, replace the first

node by its children. Let’s think about a

different rule. Let’s replace the last

node by its children. We start by initializing the

agenda to the node that represents the start state. So that’s the path A.

Then pull out the last node in the agenda– that’s A– and replace it by

its children. Its children are still AB and

AD, just like before. Now the answer differs from

the previous answer. Because when I pull out the last

node, I’m pulling out AD now, instead. And now I replace AD by

its children, which are ADA, ADE, ADG. Repeat, and what I’ve got is

a different, but still depth-first search. So I’ve looked at two different

orderings– pull out the first node from the

agenda and replace it by children, pull out

the last node and replace it by its children. Both of those algorithms give an

exploration of the decision tree searching out

depth first. So it’s going to try to

exhaustively go through the entire depth before it tries

to explore the width. As an alternative, think

about a slightly more complicated rule. Remove the first element from

the agenda and add its children to the end

of the agenda. So initialize it with the start

state, A. Pull out the first element from the

agenda– that’s A– and replace it by its children,

which is AB, AD. Now pull out the first guy– the first guy is AB– and put its children

at the end. It’s children are

ABA, ABC, ABE– ABA, ABC, ABE– and they are now

put at the end. So that on the next step, I’ll

pick up AD– the guy at the beginning– and put AD’s children

at the end. et cetera, et cetera, et

cetera, et cetera, et cetera, et cetera. The idea being– and now, pay no

attention to the bottom for a moment and just think

about the pattern that you see at the top. In this order, where we remove

the first node and put its children at the end of the

agenda, has the effect of exploring breadth first. So we call that a

breadth-first. so the idea is, we got this

generic set of tools that let us construct search trees. But the order by which we

manipulate the agenda plays a critical role in how the

search is conducted. And the two that epitomize the

two extreme cases are, what would happen if I replace the

last node by its children? Or what would happen if I remove

the first node and put its children’s at the end? Those two structures

have names because they happen so often. We’ll call the first

one a stack and the second one a queue. The stack-based is going

to give us depth first. The queue-based is going to

give us breadth first. So, stack. How do you think

about a stack? You think about a stack by

saying, OK, I’ve got a stack. A stack is like a

stack of dishes. So here’s my table, and I’m

going to rack my dishes up. I’m going to put them

on a stack. So I make a stack. OK, I made the stack– push a 1, push a 9, push a 3. Push a 1, push a 9, push a 3. That’s how I do a stack. Then pop. When I pop, the 3 comes out. Then pop, then the

9 comes out. Then push a minus 2. Then pop. Now the minus 2 comes out. OK? It’s stack-based. So the last in becomes

the first out. That was the rule that we

wanted to have for the depth-first search. It’s very easy to

implement this. We can implement it as a list. All we need to do is be careful

about how we implement the push and pop operators. So if we set up the push

operator to simply append to the end, and then pop ordinarily

pops from the end, we’ll get the behavior

of a stack. That gives me, then, the rules

that I would want to use for those procedures, the

get element an add. I will use these stack-based

operators. The other alternative

is a queue. A queue is different. A queue is like when you’re

waiting in line at the Stop & Shop. The queue is, I’ve got this

queue here and I’ve got the server over here. The first person who comes

into the queue– so say I push one. So now 1 goes into the queue. Then another person walks

up while he’s– the second person lines up

behind the first person. Then I push a 3. But the way the queue works is

that when I pop the next person off the queue, I take

the head of the line. So the 1 comes out. If I pop again, the

9 comes out. If I then push a minus 2 and

pop, then the next person in the queue comes out. And it’s like that. It’s queue based versus

stack based. And the queue based is the one

that we want to do for a breadth-first organization. And the implementation of a

queue is very trivially different from the

implementation for a stack. The only difference is that I’ll

manipulate the list by popping off from the

head of the queue. So pop takes an optional

argument, which when 0, tells you the– the argument tells you

which element to pop. So when you specify the zero-th

one, it takes it from the head of the queue. That makes it very easy now to

replace the pseudo-procedures with real procedures. If I wanted to implement a

depth-first search, I would replace the “create a list that

contains the agenda” with “create a stack that will

contain the agenda.” So create a new stack, the agenda

is a stack. And then rather than simply

sticking the node– the start node– into a list, I will push

it into the stack. So agenda is a stack. Agenda.push, the initial node. And then every time I want to

get a new element out, I’ll agenda.pop it. And every time I want to put

something into it, I’ll agenda.push it. Other than that, it looks just

the same as the pseudocode that I showed earlier. So there is an implementation,

then, for a depth-first search. If I wanted instead to do

breadth, it’s trivial. Change the word “stack” to the

word “queue.” Now create an agenda that is a queue, but

queues have the same operations– push and pop and empty– that stacks have. So nothing else in the

program changed. All I needed to do is change

the structure of the thing that’s holding the agenda. Everything else just follows. Ok. So that’s everything

we need, right? Now what I want to do is think

through examples and think about the advantages and

disadvantages of different kinds of searches. And I want to go on to the

second step that I raised in the first slide. I want to think about, how

do I optimize the search? As I said, even that simple

little tile problem, even the eight puzzle– eight sounds easy, right? Even the eight puzzle

had a third of a million different states. I don’t necessarily want to

look through all of them. I want to think now about

these different search strategies, and how optimal

are they relative to each other, and are there ways

to improve that? Now some of you may have noticed

that all of these paths don’t seem equally good. So take a minute. Think about it. Remember the problem. The problem was this walk around

Manhattan problem. I wanted to go from A to I.

This was the tree of all possible paths from A to I.

What I’d like you to do is think about whether all of those

paths are important. Could we get rid of

some of them? So the question is, can I

throw away some of the terminal nodes? Notice that I’m using the word

“terminal” kind of funny here. The tree keeps going. The tree is actually

infinite in length. So by “terminal,” I just

mean this row three. So could I throw away some of

the nodes in row three? And in particular, how many of

them could I throw away? 0, 2, 4, 6, or 8? or Raise your hand with

the funny coding. And the answer is– come on, come on. Raise your hands, raise

your hands. Blame it on your neighbor. That’s the whole point. OK, it’s 2/3 (5) and 1/3 (4) How’d you get (5) and (4)? Yes? AUDIENCE: When you have that

[INAUDIBLE] nodes, you may know that there’s

[? a procedure ?] [INAUDIBLE] before. PROFESSOR: Good. If you’re walking around

Manhattan, and you’re trying to go from A to I, and if you

spun around in this loop and came back to A, that would

probably be a bad path, right? So revisiting a place you’ve

been before is probably a bad idea, if what your goal was, was

to get from A to I in the shortest possible distance. So that’s exactly right. So I would like to identify

instances where I go back to where I started. So for example, that

A. That A is bad. That means I went back

to the start place. I’m just starting over. So if I think about those, I

can identify by red all the places where I’m repeating. So ABA, don’t really care

what happens after that. ABCB, well, that’s B again. So that’s just brain

dead, right? So I can actually remove a fair

amount of the tree by simply getting rid

of silliness. Don’t start the path over again,

where “over” means if you come to a place you’ve

been before, stop looking there. That’s not the right answer. And so you can see there

that I actually deleted half of the tree. The number of nodes on the

third line was 16. And 8 of them had the property

that they repeated. Yes? AUDIENCE: [INAUDIBLE] [? after D. ?] Are you [INAUDIBLE]? PROFESSOR: This B and D. So

there’s no reason to consider this D, even though the

D didn’t repeat. AUDIENCE: That would

be [? AD? ?] PROFESSOR: ABC, AUDIENCE: [? That would be E? ?] PROFESSOR: ABED. AUDIENCE: [INAUDIBLE] PROFESSOR: So I didn’t, in this

path, ever hit D before. AUDIENCE: When it did lead to

the path there, you’ll get the same one, AD [INAUDIBLE] PROFESSOR: I guess I

don’t understand. AUDIENCE: What I’m saying is

that without getting the path AD, I still get (2)? PROFESSOR: Yes. So this D seems clearly inferior

to that D. Yes, that’s absolutely true. So this is a very simple rule

for removing things. You’re thinking of a

more advanced rule. So if you saw– if there’s a shorter path to a

particular place, don’t look at the longer path. You’re absolutely right. So in fact, there might

be more severe pruning that you could do. There might have been

an answer that was bigger than 8 — right? And so you’re absolutely

right. Ok. Let me ignore that for the

moment and come back to it in about four slides. You’re absolutely right. So what we want to do now is

take that idea of throwing away silly paths and formalize

it so that we can put it into the algorithm. And we’ll think about that

as pruning rules. So the first pruning rule

is the easy one. Don’t consider any path that

visits the same state twice. That doesn’t pick up your case,

but it does pick up 8 cases here. So that’s easy to implement. All we need to do is– down here where we’re thinking

about whether this is a good state to add or not– we just ask, is it

in the path? So if the state that I’m about

to put in the path is already in the path, don’t

put it there. If you don’t shove

it back into the agenda, it’ll get forgotten. So before you shove it into the

agenda, ask yourself the question, is it already

in the path? And so I do that here. Keep in mind, I popped out an

element called the parent. I’m looking at the children. The children’s state is called

“new state.” So I ask, is new state in the parent’s path? So parent.inpath of new state. So that means I have

to write inpath. Inpath is easy. It’s especially easy if

we use recursion. So inpath says, if my state is

state, then return True. I’m in the path. If that’s not true, and I don’t

have a parent, then that means I’m the start state. That means it wasn’t

in the path. And if neither of those

is true, ask the same question of my parent. So that makes it recursive. So consider two cases

that could either make it true or false. It would be True if I’m

currently sitting on a node that happens to be

the same state. It would be False if I recursed

the whole way back to the start state and hadn’t

found it yet. So there are two termination

states– I landed on a state in the path

that was the same as new state, or I ran the whole way

back to the start state and didn’t find it. Those two terminate by

doing returns — return True or return False. The other option is that

I don’t know the answer, ask my parent. So just recurse on inpath,

and ask my parent to do the same thing. So that’s the way I

can figure out– I can implement pruning

rule (1). Now pruning rule (2) — if multiple actions lead to

the same state, only think about one of them. That actually doesn’t happen on

the Manhattan grid problem. Because you can imagine search

cases where there are three different things that

you could do. In fact, you saw some of those

when you were coding the robot last week. There were multiple ways you

could end up at the state at the end of the hall. You could get there by being

there and moving left, which you hit the wall. Or you could get there by being

here and moving left. Both of them left you

in the same place. So if you’re planning a search,

you don’t need to distinguish among those, because

they take the same amount of steps. So since they take the same

amount of steps, we don’t need to search further. So we can collapse them. That’s called pruning

rule (2). That’s also easy to implement. What we do is, we keep track of,

for every parent, what are all of its children. If the parent already has a

child at that place, throw away the excess children. That doesn’t sound good. So keep track of how many

child states I have. Make a list. And if the new state didn’t

satisfy the goal, ask if it’s already in the list

of children. If it’s already there, pass. Don’t do anything. Otherwise, do pruning

rule (1). And then, before you push it

into the agenda, also push it into the list of new children. That’s a way of making sure that

if there’s multiple ways to get the same state, you

only keep track of one. So that’s an additional

pruning rule. So now let’s think about how

we would implement these. Let’s think about the solution

to a problem where we want to apply a depth-first search on

this Manhattan problem, to get from A to I. So let’s

think about– let’s go up– so I want to think about, how do

I apply depth-first search to that problem? So think about the agenda. So the agenda, I initialized

it with the node that corresponds to the start state,

so that’s A. I’m doing depth first. What’s the rule for

depth first? Pop the last guy, replace

it by his children. OK, so pop the last guy. What’s the last guy? The last guy is A. Replace

it by his children. What’s his children of A? Well, there’s two of

them, AB and AD. Ok. So I’m done with the loop

for the first level. So pop the last guy,

that’s AD. Replace it by his children. What are the children of AD? Well, what could D do? D could go to A or E or G.

A’s brain dead, so I don’t want that one. So I’ll think about E and G. So ADE, ADG. By the way, stop me if

I make a mistake. It’s really embarrassing. OK, pop the end, ADG. Who’s the possible

children of ADG? ADG? Well, it could go back to

D, but that’s stupid. So ADGH seems to be

the only good one. Pop the last one, ADGH. And who’s his children, ADGH? ADGH has children E, G, and I.

But I don’t want G, because that’s brain dead. So ADGH, E or I. And that

one won, right? Because I got to A. Everyone see what I did? I tried to work out the

algorithm manually. So the idea, then, was that– so how much work I do? I visited 1, 2, 3, 4, 5, 6, 7. And then I found it. So I did 7 visits. And I got the right answer. So both of those are good– 7 is a small number, and getting

the right answer. Both of those are good things. And in general, if you think

about the way a depth-first search works– here’s a transcript of

what I just did. This will be posted on

the online version. So you can see it, even

though it’s not handed out to you now. So you can look this

up on the web. So in general, depth-first

search won’t work for every problem. It happened to work

for this problem. In fact, it happened to be very

good for this problem. But it won’t work for every

problem because it could get stuck. It could run forever in a

problem with infinite domain. This problem has infinite

domain. So if I were to choose my

start and end state judiciously, I could get it

stuck in an infinite loop. That’s a property of

depth-first search. Even when it finds a path, it

doesn’t necessarily find the shortest path. Well, that’s a bummer. But it’s very efficient

in its use of memory. So it’s not a completely

brain-dead search strategy, but it’s usually brain dead. So let’s think about

breadth-first search as an alternative. Again, all we need do is switch

the idea of thinking about stacks versus queues. Take off the beginning,

add to the end. That’s the way queues work. So now let’s do the same

problem with a breadth-first search. So I start with the agenda. I put in A. I pop off the head

of the queue and stuff the children at the end. I pop off the beginning and

stuff the children AB, AD. That looks right. That’s the end of path one. Now I pop off the beginning

and stick in the children. What are the children of AB? Well, AB could go to ACE. A is brain dead, so ABC– ABC or ABE. ABC or ABE, that looks right. Now pop off this guy, AD, and

put his children at the end. That’s ADE and ADG. I don’t think I made

a mistake yet. Pop off the first guy, ABC. Stick in his children ABC– ABC, it could go to B or F.

Looks like F is the only one that makes any sense. ABC, that looks right. ABE, put his children ABE– E could go to B– that’s

brain dead– D, F, or H. D, F– wait. AUDIENCE: AB. PROFESSOR: AB, thank you. I’m supposed to be doing ABE

followed by something, ABE followed by something. I don’t want B. D is fine,

F is fine, and H is fine. D, F, and H. OK so far? AUDIENCE: [INAUDIBLE] PROFESSOR: Oh no,

it’s not right? OK, what did I do wrong? AUDIENCE: Just AB. It’s the fourth– PROFESSOR: Oh, here. That’s up here. Is that [INAUDIBLE]. AUDIENCE: Yeah. PROFESSOR: Thank you. That would be embarrassing. OK, next pop off AD– This is why we have

computers, right? We don’t normally

do this by hand. OK, so ADE– if I had ADE, I could do B– that seems OK, D seems bad– F, or H. So it would look like B, F, H. OK, ADG. A, D, G. It looks like

H is my only option. ABCF. A, B, C, F. Looks like I could

do E or I. Finally. Now the only question

is whether I got the right number of states. Let’s assume I did. So 1, 2, 3, 4, 5, 6, 7, 8, 9,

10, 11, 12, 13, 14, 15, 16 — which happens to be

the right answer. At least it happens to be the

answer I got this morning when I was at breakfast. So what did I just do? I just did a breadth-first

search. Here’s a transcript. 16 matches, good. Breadth-first search has a

different set of properties. Notice that it took me longer. But because it’s breadth first,

and because each row corresponds to an increasing

path length, it’s always guaranteed to give you

the shortest answer. That’s good. So it always gives you

the shortest answer. It requires more space. I mean, you can see that

just on the chalkboard. And also it still didn’t take

care of your problem. So this still seems like

there’s too much work. I looked at 16 different

places. I did 16 visits. There’s just something

completely wrong about that, because there’s only 9 states. How could it take more visits

than there are states? So that just doesn’t

sound right. And it’s for exactly

your point. And so there’s another idea

that we can use, which is called “dynamic programming.”

The idea in dynamic programming, the principal is,

if you think about a path that goes from x is z through y,

the best path from x to z through y is the sum

of two paths– the best path from x to y and

the best path from y to z. If you think about that, that

has to be the case. And if we further assume that

we’re going to do breadth first, then the first time that

we see a state is the best way to get there. So what we can do then, in order

to take care of your case, is keep track of the

states we’ve already visited. If we’ve already visited a

state, it appears earlier in the tree, there’s no point in

thinking about it further. That’s the idea of dynamic

programming. And that’s also easy

to implement. All we need to do is keep track

of all those places we’ve already visited. So we make a dictionary called

“visited.” So I initialize before I start looking

at the children. I initialize right after

I set up the initial contents of the agenda. I create this dictionary

called visited. And every time I visit a new

state, I put that state in the visit list. Then before I add the child to

the agenda I ask, is the child already in the visit list? If the child’s already there,

well forget it. I don’t need him. Otherwise, just before you push

the new state, remember now that that’s an element

that’s been visited. So the idea, then, is that by

keeping track of who you’ve already looked at, you

can avoid looking– so if there’s a depth-3 way to

get to D, and a depth-2, then I don’t need to worry about

the previous ones, because it’s already in the visit list. Yes? AUDIENCE: Why do we still need

the new child states up there? PROFESSOR: Why do we still have

the new child states? AUDIENCE: The placement was

based [? on– ?] the [INAUDIBLE] state. PROFESSOR: I think

you’re right. I should think about that. I think you’re right. I think when I was modifying

the code for the different places I slipped and could

have removed that line. I think you’re right. I’ll have to think about it,

but I think you’re right. So if that line magically

disappears from the online version, he’s right. OK, so now one last problem. I want to see if I can figure

out what would happen with dynamic programming. So I want to do breadth first. And just as a warning, I’m

hypoglycemic at this point. So there may be more

errors than usual. So I need to keep track

of two things. I need to keep track

of the visit list. And I need to keep track

of the agenda. So there’s two lists I have

to keep track of. OK. Let’s start out by saying

that the agenda contains the start element. That’s A. That means we visited

A. It’s breadth first, so I want to take the first guy

out of the queue and add his children to the

end of the queue. So take the first guy

out of the queue. Add his children. A’s children are B and D, which

means that I’ve now visited B and D. Now I want to take out the first

guy from the queue, AB, and I want to put his children

at the end of the queue. AB’s children are A– that’s been visited, C– not visited, and E– not visited. So ABCE. But that visits C and E. Now I want to take out AD and

put its children at the end. AD is AEG. A is visited, E is visited,

which leaves just G, so ADG . And that visits G. Then I want to take out ABC and

put in its children, A, B, C. ABC, oh dear. AB– I’m looking up there. I said I’m hypoglycemic. ABC– ABC — could be B or F.

Well, B’s no good. Which leaves F, but

that visits F. So now, ABE. ABE, so that could be

B, D, F, H. B– visited, D– visited, F– visited, H– OK. That visits H. Now take out ADG. Children of ADG– ADG, two children, D and H. D

and H, they’re both there. That didn’t work. There are no children of ADG. ABCF– ABCF, three children– C, E, I. C– visited, E– visited, I– done. Found the right answer. 1, 2, 3, 4, 5, 6, 7, 8 — 8 visits. So I’ve got the same answer,

and it’s optimal, and I did fewer than 9 visits. 9 was the number of states. So this algorithm will always

have those properties. So the dynamic programming– oh, I forgot a slide. The dynamic programming with

breadth-first search will always find the best. It will never take longer than

the number of states. So in this problem that had a

finite number of states, even though I had an infinite

number of paths– because you can go around

in circles– it’ll still never take more

than the number of states. And all that it requires to

implement is to maintain two lists instead of one. So the point, then, is that

today we looked at a variety– we looked at two real different

kinds of search algorithms, depth-first search

and breadth-first search. And we looked at a number of

different pruning rules. Pruning rule (1) — don’t go

to some place that you’ve already visited. Pruning rule (2) — if you have

two children that go to the same place, only think

about one of them. You can consider dynamic

programming to be a third pruning rule, because that’s

the effect of it. And the final announcement,

don’t forget about Wednesday. Have a good week.

Great lecture.

Mistake at 57:34. Should be ADEH.