machine learning ?

The "camera's picture" is a little bit more complex than a simple binary input of 4 to 8 bumper switches.

In the simple example you give above which is "learn not to get stuck when hitting an obstacle" you have in fact set the system up as to what it can learn and how it is to learn it. The key is the "fitness rule" that determines what throws of the dice are to be accepted or rejected.

Although you might think that the input is simply the binary pattern at time N there are also sequences of patterns or the current action that can determine the next action.

An example is the alternating task. (move_left/food) -> move_right (move_right/food) -> move-left The input (food) is the same but the action is dependent on previous input/action combination. But if your program doesn't take these things as data they it will never learn to solve certain kinds of problems.

There are other things such as temporal factors. The time between bumps may be a "feature" of a room. All this and you still only have the same set of sensors but have constructed a much more complex environment. Actually I think "distance/direction travelled" between bumps might be more useful :)

It is hard to recommend books on these subjects. All I have done over the years is visit technical books shops, browse the books of interest to see if I can understand them. The last book I bought on AI was the 1993 edition of AI by Patrick Henry Winston.

There should be lots of stuff on the robot web sites. Subjects on ideas for simple robots such as Rodney Brooks subsumption architecture.

- John

True, but I can perform line detection on the image, and determine how far and in what direction the lines moved

Rich

And thus your "input" is no longer being used but some transformation of the input as regards any associations with the motor output. This moves beyond simple association of input/output into computation.

For example your lines will "move" in all sorts of ways depending on the orientation your robot has with respect to those lines.

And how will you use all this "line" data?

John

My idea is to associate input with randomly tweaking outputs.

I don't know if it would be useful. Just interesting.

The idea is for the robot to learn which input(s) change when the output(s) are changed. Also, HOW the input changes.

Robot randomly selects Left motor to send output to. Left motor turns on for a moment. Robot turns right.

The couch (4 lines, a rectangle) moved to the left on the picture.

The robot learns "when I send output to port 1 (left motor), 4 lines in vision move left.

I don't know what use it would be. The idea is to give the robot minimal programming and let it figure itself out.

In my original post, the program I mentioned is hardwired to learn how to get out of "bump" situations. You could also say it is hardwired to learn how to continue moving. It is a very simple program, but I like how the "memory" relates to the inputs (bumper detects) and to the outputs (direction to go to continue moving) they are all 9X9 matrix's :-) and they all relate to eachother perfectly.

It would be nice to have the robot learn that spot X,Y on the occupancy grid contains an obstruction. This obstruction will produce a BUMPER input if robot is at X+1,y and left + right motor outputs are told to go FD. This spot(x,y) on the occupancy grid is line (VX,VY) in the camera view.

Rich

Learning mechanisms within an individual requires extra machinery. So for simple things we might ask: is the extra machinery worth it or should we just build these elementary things in?

In the case of the evolution of organisms these inbuilt things are a result of a kind of evolutionary learning where the "results" are built into the genetic code. This means that once born the animal "knows" what to do. This is akin to dropping your robot on Mars and not having it to have to spend a year learning some basic useful behaviours like not driving over a cliff.

Imagine a young lamb has just been born and doesn't know how to find milk, or even stand up and walk to follow or find its mother? All that is required for a grazing animal is a little bit of fine tuning and away they go. Thus they have a survival advantage over an animal that would have to learn, by trial and error, all these things.

Learning in the individual can have a survival advantage. What can be learnt however is often genetically determined. Thus what you can teach a rat is not the same as what you can teach a pigeon. Such limitations would exist for any robotic learning system.

So what do we want to build into our machine and what is better left to a learning mechanism?

Evolution decided that for each individual species by the survival or not of each animal. For a grazer that just bends it head down to munch grass learning is limited and of less value then for a predator that has to deal with an ever changing environment where the extra effort to learn some things is worth it.

The idea of a robot starting with a blank slate and learning everything is like simulating evolution without the advantage of billions of individuals and eons of time.

Getting back to your "moving line" the first problem is how to get the machine to learn how to detect a "line" or "movement left/right" in the first place?

We might start with connecting the sensory inputs randomly to a set of logic gates and using some "fitness rule" to keep or reject the connections until we get one that "works". By "works" I mean is rewarded and selected for by some fitness rule.

The "fitness rules" (goals, what is pleasure or pain; tasty or not, smells good; smells bad) in animals is a product of evolution. Animals that liked jumping off cliffs didn't reproduce. Humans that liked the smell of rancid food tended to die out. Even so in a large population there is still a wide variation in "fitness rules" so should the environment change those lucky enough to have the best likes and dislikes will be more likely to survive.

That turning on the left motor will cause things to "move left" may be a case of "knowledge" being rewarding in itself. However to be useful that knowledge has to be used to say "home in on a target". How does the goal (home in on target) get connected with what has been learnt with respect to "left motor on" -> "lines move left"?

Take for example a network that can output the velocity of moving "lines". If you look out the side window of a train distant object move slower than object up close. Thus the faster the lines a moving the closer you are getting to a collision. Flying insects can use that for a perfect landing by adjusting their velocity with how fast the ground is moving beneath them. This is referred to as "optical flow" and can be used in robotics so a robot can whizz about and not collide with obstacles etc.

Natures "fitness rule" is reproductive success.

If you want your robot to "learn" you need a similar system. In this case survival is up to you or your inbuilt requirements as to what is good or bad.

This of course is a requirement not a formulae for building learning mechanisms.

So is this like a robot that moves, takes a picture and stores it, moves again, takes a picture and stores it, until it has a memory filled with pictures along with the movement required to go from picture(n) to picture(n+1)?

Thus if its goal is to have picture(245), a frontal view of say the fridge door two wheels turns away from physical contact, than it can use all these stored picture/ action associations to go from its current picture input to a desired picture input?

- John