the Fish and the Robot

This is a spin-off from the "where is BBR AI now" thread, related to my afternoon in the park experience.
<LONG STORY>
After work, I went downtown and happened to spot a fish in the water while crossing the bridge over Boulder Creek. Right there is a small 2' waterfall with a lot of rocks in the water below it. Also, a couple of sandy spots under the water. I could see 2 fishes lollygagging over the big sandy spot. I knew there were supposed to be fish in the creek, but didn't think there were really very many. After a minute or 2, I decided they where probably sitting there waiting for a drowned bug to wash over the falls, so they could have dinner. I really didn't see much else going on.
After a couple of more minutes, I decided to test my theory about the fish waiting for dinner, so I tossed a penny into the creek in the middle of the sandy spot. [if the Boulder council knew about the penny in the water, they'd have me shot, BTW].
Immediately, WHAM ... about 30 or more fish came shooting out from between rocks [where they blended in well and were difficult to spot] from everywheres in the creek, and all went straight for the penny drop. Coming in fast from 360 degrees around the drop site. Looked like spokes on a wheel. Well, guess there ARE fish in boulder creek after all. I also tried this a few more times over 10 minutes or so, and eventually most of the fish habituated to chasing fake bugs. 10-minutes and the 10th penny, just a slight commotion.
They figured it out pretty fast, although I'm not exactly sure how they did it. It didn't look like every single one of the fish had a chance to nibble on the fake bugs, but still the frenzy over new penny drops faded fast. Fish story-telling? Only react in case of positive feedback - ie, your buddy reacting? </LONG STORY>
In any case, to the point. Related to the other thread.
What's the difference between the fish and our level of robots?
What would happen if we tossed a penny or ball or whatever into the center of a bunch of mini-sumos? Pretty much, nothing. The mini-bots could easy run over and grab, or at least hit, the object. In this sense, they are not too different from the fish. They move, they turn, they bump into things, they can act autonomously, etc. They behave.
What they cannot do at all as well as the fish is to sense and perceive, and make something important of the sensory input. We have a lot of very simple sensors, but in essence our BBR bots are still blind and deaf like Helen Keller, but also very very dumb [in the IQ sense]. For small bots, the CMUCam is about the top end, but still that is hopelessly primitive compared to fish perception.
Attending properly to this problem is really the next level beyond BBR, I think.
Comments?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

Living creatures must learn their behaviors. Over its lifetime, a living creature will learn some percentage of the optimum set of behaviors that would make it a "perfect" specimen. Robots have the advantage that they can be programmed with 100% of their "optimum" behavior set at day 1.
Many living creatures' behaviors are associated with a group - the herd, flock, school, etc. - & behave according to what others in the group are doing. In the fish example, each individual fish may not necessarily be aware of the "bug" that was dropped in the water, but rather are aware that other fish are swarming aggressively, focused on some point. Each fish in the school reacts to the other fish in the school (at least 1 fish must have actually sensed the "bug" in the 1st place) & rush toward the point that all the other fish are rushing towards. Some lucky fish - it may not even necessarily be the 1st one to have sensed the bug - ends up with a meal, rewarding his behavior.
In a similar yet opposite context, individual sheep in a herd react to the other sheep by bunching together & fleeing, even though each individual hasn't actually sensed a wolf. Some number of individual sheep must actually sense the wolf, & their behavior triggers the "instinctive" behaviors of the other sheep in the herd, who haven't actually detected the threat. Any sheep who fails to exhibit the necessary degree of "correct" behavior may end up as the wolf's lunch.
In the case of the mini-bots, they are generally missing this "herd-instinct" behavior. They may react to the ball, but not to the other mini-bots' reactions to the ball. They need to not only sense their environment for "things" of interest, but also to determine what others of their ilk are doing, & react to that as well. If the desired outcome is to grab an object, those individual mini-bots who can't actually "see" the object need to be able to detect the fact that some of their bretheren are reacting to "something", & allow themselves to be drawn toward the center of that activity. An individual mini-bot may succeed in grabbing the ball if it's able to fight through the mass of its compatriots & be lucky enough to be in the right place at the right time, just like a fish that swims to the point where all the other fish are focusing their attention just might get rewarded with a bug to eat.
An interesting series of exercises fall out of this: How far might each individual mini-bot be programmed to go, to achieve its goal of grabbing the ball? How hard should each unit fight to get closer to the center of activity, & once it actually detects the desired object (as opposed to merely the commotion surrounding the object), how hard should it fight to secure the object? Should the effort stop when it's been determined that another min-bot has secured the ball, or should it try to wrest control away? Does it go so far as to "kill" another of its own kind, if necessary to achieve its goal? How might an individual mini-bot determine when to abandon its effort to grab the ball & go back to its normal "milling about with the herd" mode? How might a mini-bot secure the ball, defensively, to keep other mini-bots from taking it away, once grabbed?
Other variations include perhaps adding colored lights to indicate some sense of "purpose", such as green means "I'm heading towards something good," red means "I'm avoiding something bad,", etc. so that others detecting their light AND motion can behave appropriately. Also, different "types" of mini-bots could be mixed, with each type programmed to react differently to its own type vs. the other types. Programmed behaviors in reaction to other-than-own types could be "avoid", "flee", "attack", "ignore", etc.
JM
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
John Mianowski wrote:

Except of course, for the instinctual behavioral part.

You're kind of mixing regular schooling behavior with what happened in the case I cited. Regular schooling behavior involves a form of self-organization where each fish obeys basically simple rules to keep its distance from other fish, but to also follow the pack. IE, rule#1: go in the general direction of the entire school. Rule #2: maintain a more or less moderate distance from your nearest neighbors.
OTOH, what I observed was 30+ fish, who were spread out with each sitting in their own particular spots within the rocks, suddenly converging en-masse and at maximum warp-speed to a central point from 360 degrees of direction. Visualize spokes on a wheel, collapsing into the center hub. This is different from schooling behavior. Their intent to get to the dinner-spot overwhelmed any fish-thoughts about schooling behavior.

You kind of missed the point. You're talking here again about programming behaviors. I was talking about the fundamental problem of perception.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

I would suggest that the fish did not learn anything, but from instinct knew to start ignoring stimulus that wasn't food or a mate. Come back the next day and I'd bet they'd start out again just as curious about the money you're tossing into their water.
There is a good book that addresses much of this. It's called "Time, Love, Memory," by Jonathan Weiner. The book is about the origins of behavior, and Weiner uses scientific studies to look at just how much lower creatures (specifically fruit flies) retain such things as memory. It is a biographical book of the players involved that also discusses a lot of scientific underpinnings, and is a very entertaining read. The author won a Pulitzer prize for a previous book.
-- Gordon
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Gordon McComb wrote:

Clearly, a fish doesn't know the value of a dollar! Ha.
Actually, I think my experiment illustrated a couple of points.
First, I do agree with you that, if I went over and did the same thing today, the fish would probably act pretty much act the same, so they didn't in fact "learn" very much from yesterday's trial. Hmmm.
However, it's also clear that the fish do have some sort of short-term memory mechanism that prevented them from endlessly repeating the same behavior over and over. [and it wasn't just related to randomization of response].
To repeat the same useless thing over and over would obviously be survival-negative, as they would be wasting a lot of energy with their hit-speed attacks, but not be rewarded for their energy-expenditures. So, in the past, they evolved some sort of innate short-term memory mechanisms which reduces the useless behavior. IE, rule: if several repeated attacks produce the same bad result - penny-UGH! - then stop doing it. There are obviously many ways this could be implemented, but all require some sort of short-term "memory store".
Secondly, as I indicated before, the experiment also illustrates how advanced the fish's sensory and perceptual systems are, compared to our little robots, which is really important here.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

Again, see the book. It's specifically about manipulating the genes that control "memory." Now, in a fruit fly memory is different than what we have because we have so many more receptors, but it turns out the genes are strikingly similar between humans and fruit flies. He's the value of the work.

This is the real value of the experiment, IMO. Fish will follow most anything that catches their attention, as long as it doesn't exceed a size thresh hold. The sensory mechanism doesn't need to be complex, but are probably more exomplex than the sensor array that the average hobbyist can afford. I think it's important to bear in mind that there are sensors capable of picking up an incredible amount of data, but they are far beyond what we are willing to pay. Eyes are free to their owner.
(Curiously, through, they tend to ignore their own poop, and the poop of their buddies. They know it's not food, though it might behave -- slowly float or whatever exactly like food. Now *THAT* is worth a doctoral thesis.)
-- Gordon
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Gordon McComb wrote:

I'll look for it.

But our robots need this to advance to the next level.

Fish have excellent olfactory and I would imagine underwater taste sensing capability. They probably know which of their buddies the individual poop comes from. They know the pennies aren't food by nibbling, taste, and smell. But first, they have get sense the penny drop from 20+ feet away, and get over to it in jig time. That's the good part.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

Yes, but the point was this: If you have a high-end sensor capable of greater definition than what we commonly use today, you'll probably write some code to take advantage of it. The robot will seem to have more intelligence because it has greater information-gathering ability. There's diminishing returns writing behavior code for a robot that is basically blind: some touch switches, a couple IR sensors, and maybe an ultrasonic sensor on a sweeping turret. Compared to that fish with its robust vision system such a robot is the equivalent of an amoeba.
So, in order to get to the next phase we need either lower-cost sensors, or more money to invest in our robots. Or maybe both. That will make the next level of BBR much more meaningful.
-- Gordon
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

OTOH, he does know pennys are pretty much worthless.
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dan michaels wrote:

You didn't intersperse bugs, you don't know what they learned.
You didn't intersperse in some dimes. Maybe they learned and habituated only to the copper color.
You didn't intersperse some copper colored bugs. There are ways to refine your assumptions.
Great first experiment. Now where's your follow up research? Is it worth validating your claim? Or are we back on our simple little robots?
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
RMDumse wrote:

The follow-up research is obviously to build a sensory system comparable to that of the fish and which is small enough to fit on my small robot. That's the way to robot intelligence. :)
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Begging to differ, it isn't obvious to me at all. My point above was you don't know what the sensory system of the fish is doing. So how could you even tell if you were building something comparable?
It's wonderful to read other people's research, but hats off to someone who does and publishes their own.
A side note: I've run into habituation as a research subject before. Back in the days of the AIM-65 at Rockwell, I did a side project for a couple researcherx in San Diego, where we'd test baby mice by playing a loud noise, and checking their reaction to it, as measured by their jerking while balanced on a platform with a load cell.
(Did you know if you make a loud enough sound, like jingling car keys, you can kill a mouse? It will shock and go into cardiac arrest. Or at least so I am told.)
The rate of habituation was used to determine their mental soundness and abilities. By finding something to test at the very early stages of life, you could accelerate the rate of testing without having to let the rats mature to see if they had their faculties or not.
Anyway, one of the arguments of Brooks I find most compelling is introspect doesn't work. Our minds will make up lies, rather than admit it doesn't know something. And so we are prone to self delusions.
What our stupid little robots were to Brooks, was a way to check hypothesis against reality, and let reality arbitrate between what was useful/effective and what was not. Likewise, I think instead of trying to build something fishy, and comparing our robots to it, we should really learn what it is we should be building.
--
Randy M. Dumse
www.newmicros.com
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Forget fish sensors. They are overkill. Just go for better sensory processing algorithms. Try to create a generic learning system that can integrate the sensors you already have on a small bot, including data from what outputs your code is producing as extra "sensory" inputs.
For example, a bot with sonar should learn that if the wheels are being commanded to roll forward, but the sonar is not detecting thing moving towards it, then something is "wrong" and the bot is not moving.
For example, here's what I was thinking of trying to make work with my vex hardware. Reward the system for moving forward as quick as possible for as long as possible, without turning. I think a simple critic using the sonar data and wheel movement data could generate that reward signal. It would give the system more rewards as the it moved forward and the sonar sensed objects approaching. The faster the objects approached, the more rewards the critic would generate. Then, see how good the bot is at using all it's sensory data, to keep moving in a complex environment without constantly hitting things or getting stuck.
A really smart system, will learn it's way around the environment and find things like long halls where it can do a straight run down the hall without having to stop or turn. Or it might find a more complex route through the halls which maximises the long straight paths.
Simple sensors like the vex has is a hard enough challenge. Sonar ranging, combined with light level detection, combined with bumper switches is a lot to work with as it is.
For example, a good learning system might learn that when driving straight, if an object is approaching at the expected speed we are driving (aka the object is stationary), and the light on the left is a fairly constant level, but the light on the right takes a sudden jump higher, that the bot should turn right to get the most rewards (because the light jump was caused by the hall, or open door, or free path to the right which is more likely to yield a long path of rewards than continuing to go straight will in that context).
Or it might learn to use light sensors to help it stay in the middle of the hall and not get stuck hitting the wall which causes it to "loose rewards" before it gets going again. The point is that it should be combing information from all the sensors it has, and looking for correlations with past reward levels to trigger all it's behavior. It doesn't need to "know" what the sensory data "means". It just needs to be able to correlate temporal sensory data patterns, with the behaviors that have produced the most rewards (worked the best) when patterns similar to that were last seen.
I think simple problems like that will help lead to better sensory processing and learning systems and intelligence without having to deal with something like a high bandwidth video system. But, if you have the processing power and memory to process a video data stream, that's useful as well. I just don't think the simple sensors are being used as well as they should be yet.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Curt Welch wrote:

Then it could learn something wrong. There are many instances when a robot may move forward but there's nothing in front of the sonar within range, so all the sensor sees is normal noise and transients common in ultrasonic pings. It would appear to the robot that "I'm not moving!" when in fact it is. This reading alone is not sufficient sensory input.
Real life often screws up the best laid plans. You need to remember if things were always this easy, people would be doing them.
-- Gordon
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Right. This is why intelligence requires complex statistical learning. There are very few simple answers about how to make use of the sensor data. Though a sonar sensor can tell you important things about whether you were moving, it's very hard to correctly make use of that because of all the times it would be giving you false readings. The case you mention above is the "I see nothing" case where it would not be giving you any indication of movement. Other problems would happen when it's giving you a fix on something else which was moving. The system must recognize all the different exceptions to the case and use that to make estimates about what it meant about the world.
The reason few robots try to take advantage of this complex information in the sensory data is because it's so hard. My point above wasn't that real life data was easy to make use of, but that simply, there is useful information in the sensory data that humans, with brains, make use of all the time, but which robots seldom make use of.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Curt Welch wrote:

"Seldom" is not very good usage here, Curt. People have been looking at sensor integration problems for many years. The fact that it's hard hasn't meant that people haven't been doing it. Look at Arkin's book for dozens, if not 100s, of examples. Also ...
http://citeseer.ist.psu.edu/cis?q=robot+sensors
Also, Thrun's new book on Probabilistic Robotics. The real trick in driving an autonomous robot vehicle 130+ miles wasn't in controlling its behavior. It was in proper sensor processing.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

True.
That's something I would like to read. Much of the simple algorithms we tend to write use very simple boolean tests of data (is distance >80cm) for example. But much of how animals and humans react can be better understand as probability based because of the very fine resolutions the decisions in the brain are being made at.

Yeah, well, as I wrote in my previous message today, I don't think it's valid to force such hard distinction on the processing as being sensor or other types of processing. In the end, there's a lot of processing that happens between the raw sensor inputs, and the final outputs, and any decision to label part of the processing as "sensor" processing and part as "output processing" is totally arbitrary and mostly meaningless.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Curt Welch wrote:

If you look on the link cited, you will find one of Thrun's citations near the top of the page, and this will lead you to many of his papers online, which cover various parts of his work described in the book.

You might take a look at Braitenberg's book "Vehicles: experiments in synthetic psychology", as he describe what you can do by sensing analog signals and feeding processed analog to your motors.

You might take a look at Thrun's own words ...
http://robots.stanford.edu/papers/Thrun01h.html ==========In the 1970s, most research in robotics presupposed the availability of exact models, of robots and their environments. Little emphasis was placed on sensing and the intrinsic limitations of modeling complex physical phenomena. This changed in the mid-1980s, when the paradigm shifted towards reactive techniques. Reactive controllers rely on capable sensors to generate robot control. Rejections of models were typical for researchers in this field. Since the mid-1990s, a new approach has begun to emerge: probabilistic robotics. This approach relies on statistical techniques to seamlessly integrate imperfect models and imperfect sensing. =========If you check out what Thrun actually did on Stanley, the Darpa winning vehicle, you'll see the real key to success was in integrating short-range [out to 30' or so] Lidar maps with long-range video camera images to plot a clear course ahead for the next few seconds.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I think I've read those words before. The idea of using probability as a foundation of control systems extends at least back to Fuzzy logic in the '60s (and the ideas continue to be used in real world applications today). One of the guys I was chatting with at the DARPA meeting said he liked to use Fuzzy logic for the low level control systems which actually operated the various effectors combined with Bayesian probability to direct the higher level decisions.

Yeah, I read the summary paper and have seen various videos. The neat thing to me was that instead of trying to integrate the two sensors to construct an model of the terrain, he instead used the accurate Lidar data to identify safe driving areas on the lower portion of the video. He then made the assumption that the top (aka more distant) areas on the video, beyond the Lidar range, which looked similar, was probably also safe. I think he made it work by doing simple pixel level tests. So instead of using the video to try and construct an actuality 3D map of the terrain, he just used it to identify safe, and not-safe, areas of the video. That then translated directly into which direction the car would need to steer to stay in the safest areas, and, I assume, control it's speed based on how much "safe road" there seemed to be ahead.
The point here is that the output needs to reduce to only two basic things - which way to turn, and how fast to go. It doesn't need cm resolution terrain maps to make that decision. When we, as humans, drive down the road, we have very little clue about the actual terrain we are driving over. Instead, we have learned, through experience, what looks "safe" to drive on, and what looks questionable or out right dangerous. We drive by keeping the car in the "safe" zone. The total processing however, needs to reduce the huge volumes of data coming from the sensors, down to very simple outputs - turn a little left, or turn a little right. The mid level representations, (half way between raw sensor data and outputs) is likely to be mid way in complexity between the two as well. This mid level representations are what we know of as our "understanding" of what we are looking at, but yet, that understanding is nothing near what some of these DARPA cars attempt to create for their internal representation of what they are seeing. What was used by Thrun I think is closer to what naturally arises in our brains.
It seems to me that the answer to the question of which way to turn, is just a large probability problem where many clues are being picked up from the environment and each clue will add its bias to the answer. Something that looks like a car headed towards us on the left part of our vision will bias us to turn to the right. More of the same road we have been driving over to the left, will bias us to turn to the left. A shape that looks like a line painted on the road will bias us to not drive over it. All these things and a million more will bias our decision about which way to turn the car from second to second. All the processing from sensory data to effector data to me seems to be very probabilistic in nature - as does all behavior.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I ran across the following blog via a link from sensors mag,
http://www.sensorsmag.com/0706/GSThrun
http://www.autoblog.com/2006/06/09/head-of-stanford-darpa-challenge-project-speaks-about-the-future /
To read some of the comments, there seems to be some question as to how much work the Stanford guys did versus how much work the VW lab in Palo Alto did, which appears to be specifically doing research on autonomous vehicles ...
http://media.vw.com/article_display.cfm?article_id 16&uid2503 http://media.vw.com/article_display.cfm?article_id 01
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Polytechforum.com is a website by engineers for engineers. It is not affiliated with any of manufacturers or vendors discussed here. All logos and trade names are the property of their respective owners.