the Fish and the Robot

Living creatures must learn their behaviors.  Over its lifetime, a
living creature will learn some percentage of the optimum set of
behaviors that would make it a "perfect" specimen.  Robots have the
advantage that they can be programmed with 100% of their "optimum"
behavior set at day 1.

Many living creatures' behaviors are associated with a group - the
herd, flock, school, etc. - & behave according to what others in the
group are doing.  In the fish example, each individual fish may not
necessarily be aware of the "bug" that was dropped in the water, but
rather are aware that other fish are swarming aggressively, focused on
some point.  Each fish in the school reacts to the other fish in the
school (at least 1 fish must have actually sensed the "bug" in the 1st
place) & rush toward the point that all the other fish are rushing
towards.  Some lucky fish - it may not even necessarily be the 1st one
to have sensed the bug - ends up with a meal, rewarding his behavior.

In a similar yet opposite context, individual sheep in a herd react to
the other sheep by bunching together & fleeing, even though each
individual hasn't actually sensed a wolf.  Some number of individual
sheep must actually sense the wolf, & their behavior triggers the
"instinctive" behaviors of the other sheep in the herd, who haven't
actually detected the threat.  Any sheep who fails to exhibit the
necessary degree of "correct" behavior may end up as the wolf's lunch.

In the case of the mini-bots, they are generally missing this
"herd-instinct" behavior.  They may react to the ball, but not to the
other mini-bots' reactions to the ball.  They need to not only sense
their environment for "things" of interest, but also to determine what
others of their ilk are doing, & react to that as well.  If the desired
outcome is to grab an object, those individual mini-bots who can't
actually "see" the object need to be able to detect the fact that some
of their bretheren are reacting to "something", & allow themselves to
be drawn toward the center of that activity.  An individual mini-bot
may succeed in grabbing the ball if it's able to fight through the mass
of its compatriots & be lucky enough to be in the right place at the
right time, just like a fish that swims to the point where all the
other fish are focusing their attention just might get rewarded with a
bug to eat.

An interesting series of exercises fall out of this:  How far might
each individual mini-bot be programmed to go, to achieve its goal of
grabbing the ball?  How hard should each unit fight to get closer to
the center of activity, & once it actually detects the desired object
(as opposed to merely the commotion surrounding the object), how hard
should it fight to secure the object?  Should the effort stop when it's
been determined that another min-bot has secured the ball, or should it
try to wrest control away?  Does it go so far as to "kill" another of
its own kind, if necessary to achieve its goal?  How might an
individual mini-bot determine when to abandon its effort to grab the
ball & go back to its normal "milling about with the herd" mode?  How
might a mini-bot secure the ball, defensively, to keep other mini-bots
from taking it away, once grabbed?

Other variations include perhaps adding colored lights to indicate some
sense of "purpose", such as green means "I'm heading towards something
good," red means "I'm avoiding something bad,", etc. so that others
detecting their light AND motion can behave appropriately.  Also,
different "types" of mini-bots could be mixed, with each type
programmed to react differently to its own type vs. the other types.
Programmed behaviors in reaction to other-than-own types could be
"avoid", "flee", "attack", "ignore", etc.

JM

Except of course, for the instinctual behavioral part.



You're kind of mixing regular schooling behavior with what happened in
the case I cited. Regular schooling behavior involves a form of
self-organization where each fish obeys basically simple rules to keep
its distance from other fish, but to also follow the pack.  IE, rule#1:
go in the general direction of the entire school. Rule #2: maintain a
more or less moderate distance from your nearest neighbors.

OTOH, what I observed was 30+ fish, who were spread out with each
sitting in their own particular spots within the rocks, suddenly
converging en-masse and at maximum warp-speed to a central point from
360 degrees of direction. Visualize spokes on a wheel, collapsing into
the center hub. This is different from schooling behavior. Their intent
to get to the dinner-spot overwhelmed any fish-thoughts about schooling
behavior.



You kind of missed the point. You're talking here again about
programming behaviors. I was talking about the fundamental problem of
perception.

I would suggest that the fish did not learn anything, but from instinct
knew to start ignoring stimulus that wasn't food or a mate. Come back
the next day and I'd bet they'd start out again just as curious about
the money you're tossing into their water.

There is a good book that addresses much of this. It's called "Time,
Love, Memory," by Jonathan Weiner. The book is about the origins of
behavior, and Weiner uses scientific studies to look at just how much
lower creatures (specifically fruit flies) retain such things as memory.
It is a biographical book of the players involved that also discusses a
lot of scientific underpinnings, and is a very entertaining read. The
author won a Pulitzer prize for a previous book.

-- Gordon

Clearly, a fish doesn't know the value of a dollar! Ha.

Actually, I think my experiment illustrated a couple of points.

First, I do agree with you that, if I went over and did the same thing
today, the fish would probably act pretty much act the same, so they
didn't in fact "learn" very much from yesterday's trial. Hmmm.

However, it's also clear that the fish do have some sort of short-term
memory mechanism that prevented them from endlessly repeating the same
behavior over and over. [and it wasn't just related to randomization of
response].

To repeat the same useless thing over and over would obviously be
survival-negative, as they would be wasting a lot of energy with their
hit-speed attacks, but not be rewarded for their energy-expenditures.
So, in the past, they evolved some sort of innate short-term memory
mechanisms which reduces the useless behavior. IE, rule: if several
repeated attacks produce the same bad result - penny-UGH! - then stop
doing it. There are obviously many ways this could be implemented, but
all require some sort of short-term "memory store".

Secondly, as I indicated before, the experiment also illustrates how
advanced the fish's sensory and perceptual systems are, compared to our
little robots, which is really important here.

Again, see the book. It's specifically about manipulating the genes that
control "memory." Now, in a fruit fly memory is different than what we
have because we have so many more receptors, but it turns out the genes
are strikingly similar between humans and fruit flies. He's the value of
the work.


This is the real value of the experiment, IMO. Fish will follow most
anything that catches their attention, as long as it doesn't exceed a
size thresh hold. The sensory mechanism doesn't need to be complex, but
are probably more exomplex than the sensor array that the average
hobbyist can afford. I think it's important to bear in mind that there
are sensors capable of picking up an incredible amount of data, but they
are far beyond what we are willing to pay. Eyes are free to their owner.

(Curiously, through, they tend to ignore their own poop, and the poop of
their buddies. They know it's not food, though it might behave -- slowly
float or whatever exactly like food. Now *THAT* is worth a doctoral
thesis.)

-- Gordon

I'll look for it.


But our robots need this to advance to the next level.


Fish have excellent olfactory and I would imagine underwater taste
sensing capability. They probably know which of their buddies the
individual poop comes from. They know the pennies aren't food by
nibbling, taste, and smell. But first, they have get sense the penny
drop from 20+ feet away, and get over to it in jig time. That's the
good part.

Yes, but the point was this: If you have a high-end sensor capable of
greater definition than what we commonly use today, you'll probably
write some code to take advantage of it. The robot will seem to have
more intelligence because it has greater information-gathering ability.
There's diminishing returns writing behavior code for a robot that is
basically blind: some touch switches, a couple IR sensors, and maybe an
ultrasonic sensor on a sweeping turret. Compared to that fish with its
robust vision system such a robot is the equivalent of an amoeba.

So, in order to get to the next phase we need either lower-cost sensors,
or more money to invest in our robots. Or maybe both. That will make the
next level of BBR much more meaningful.

-- Gordon

OTOH, he does know pennys are pretty much worthless.

Randy
www.newmicros.com

You didn't intersperse bugs, you don't know what they learned.

You didn't intersperse in some dimes. Maybe they learned and habituated
only to the copper color.

You didn't intersperse some copper colored bugs. There are ways to
refine your assumptions.

Great first experiment. Now where's your follow up research? Is it
worth validating your claim? Or are we back on our simple little
robots?

Randy
www.newmicros.com

The follow-up research is obviously to build a sensory system
comparable to that of the fish and which is small enough to fit
on my small robot. That's the way to robot intelligence. :)

Begging to differ, it isn't obvious to me at all. My point above was you
don't know what the sensory system of the fish is doing. So how could
you even tell if you were building something comparable?

It's wonderful to read other people's research, but hats off to someone
who does and publishes their own.

A side note: I've run into habituation as a research subject before.
Back in the days of the AIM-65 at Rockwell, I did a side project for a
couple researcherx in San Diego, where we'd test baby mice by playing a
loud noise, and checking their reaction to it, as measured by their
jerking while balanced on a platform with a load cell.

(Did you know if you make a loud enough sound, like jingling car keys,
you can kill a mouse? It will shock and go into cardiac arrest. Or at
least so I am told.)

The rate of habituation was used to determine their mental soundness and
abilities. By finding something to test at the very early stages of
life, you could accelerate the rate of testing without having to let the
rats mature to see if they had their faculties or not.

Anyway, one of the arguments of Brooks I find most compelling is
introspect doesn't work. Our minds will make up lies, rather than admit
it doesn't know something. And so we are prone to self delusions.

What our stupid little robots were to Brooks, was a way to check
hypothesis against reality, and let reality arbitrate between what was
useful/effective and what was not. Likewise, I think instead of trying
to build something fishy, and comparing our robots to it, we should
really learn what it is we should be building.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear.

Forget fish sensors. They are overkill.  Just go for better sensory
processing algorithms.  Try to create a generic learning system that can
integrate the sensors you already have on a small bot, including data from
what outputs your code is producing as extra "sensory" inputs.

For example, a bot with sonar should learn that if the wheels are being
commanded to roll forward, but the sonar is not detecting thing moving
towards it, then something is "wrong" and the bot is not moving.

For example, here's what I was thinking of trying to make work with my vex
hardware.  Reward the system for moving forward as quick as possible for as
long as possible, without turning.  I think a simple critic using the sonar
data and wheel movement data could generate that reward signal.  It would
give the system more rewards as the it moved forward and the sonar sensed
objects approaching.  The faster the objects approached, the more rewards
the critic would generate.  Then, see how good the bot is at using all it's
sensory data, to keep moving in a complex environment without constantly
hitting things or getting stuck.

A really smart system, will learn it's way around the environment and find
things like long halls where it can do a straight run down the hall without
having to stop or turn.  Or it might find a more complex route through the
halls which maximises the long straight paths.

Simple sensors like the vex has is a hard enough challenge.  Sonar ranging,
combined with light level detection, combined with bumper switches is a lot
to work with as it is.

For example, a good learning system might learn that when driving straight,
if an object is approaching at the expected speed we are driving (aka the
object is stationary), and the light on the left is a fairly constant
level, but the light on the right takes a sudden jump higher, that the bot
should turn right to get the most rewards (because the light jump was
caused by the hall, or open door, or free path to the right which is more
likely to yield a long path of rewards than continuing to go straight will
in that context).

Or it might learn to use light sensors to help it stay in the middle of the
hall and not get stuck hitting the wall which causes it to "loose rewards"
before it gets going again.  The point is that it should be combing
information from all the sensors it has, and looking for correlations with
past reward levels to trigger all it's behavior.  It doesn't need to "know"
what the sensory data "means". It just needs to be able to correlate
temporal sensory data patterns, with the behaviors that have produced the
most rewards (worked the best) when patterns similar to that were last
seen.

I think simple problems like that will help lead to better sensory
processing and learning systems and intelligence without having to deal
with something like a high bandwidth video system.  But, if you have the
processing power and memory to process a video data stream, that's useful
as well.  I just don't think the simple sensors are being used as well as
they should be yet.

--
Curt Welch                                            http://CurtWelch.Com/
curt@kcwc.com                                        http://NewsReader.Com/

Then it could learn something wrong. There are many instances when a
robot may move forward but there's nothing in front of the sonar within
range, so all the sensor sees is normal noise and transients common in
ultrasonic pings. It would appear to the robot that "I'm not moving!"
when in fact it is. This reading alone is not sufficient sensory input.

Real life often screws up the best laid plans. You need to remember if
things were always this easy, people would be doing them.

-- Gordon

Right.  This is why intelligence requires complex statistical learning.
There are very few simple answers about how to make use of the sensor data.
Though a sonar sensor can tell you important things about whether you were
moving, it's very hard to correctly make use of that because of all the
times it would be giving you false readings.  The case you mention above is
the "I see nothing" case where it would not be giving you any indication of
movement.  Other problems would happen when it's giving you a fix on
something else which was moving.  The system must recognize all the
different exceptions to the case and use that to make estimates about what
it meant about the world.

The reason few robots try to take advantage of this complex information in
the sensory data is because it's so hard.  My point above wasn't that real
life data was easy to make use of, but that simply, there is useful
information in the sensory data that humans, with brains, make use of all
the time, but which robots seldom make use of.

--
Curt Welch                                            http://CurtWelch.Com/
curt@kcwc.com                                        http://NewsReader.Com/

"Seldom" is not very good usage here, Curt. People have been looking at
sensor integration problems for many years. The fact that it's hard
hasn't meant that people haven't been doing it. Look at Arkin's book
for dozens, if not 100s, of examples. Also ...

http://citeseer.ist.psu.edu/cis?q=robot+sensors

Also, Thrun's new book on Probabilistic Robotics. The real trick in
driving an autonomous  robot vehicle 130+ miles wasn't in controlling
its behavior. It was in proper sensor processing.

That's something I would like to read.  Much of the simple algorithms we
tend to write use very simple boolean tests of data (is distance >80cm) for
example.  But much of how animals and humans react can be better understand
as probability based because of the very fine resolutions the decisions in
the brain are being made at.


Yeah, well, as I wrote in my previous message today, I don't think it's
valid to force such hard distinction on the processing as being sensor or
other types of processing.  In the end, there's a lot of processing that
happens between the raw sensor inputs, and the final outputs, and any
decision to label part of the processing as "sensor" processing and part as
"output processing" is totally arbitrary and mostly meaningless.

--
Curt Welch                                            http://CurtWelch.Com/
curt@kcwc.com                                        http://NewsReader.Com/

If you look on the link cited, you will find one of Thrun's citations
near the top of the page, and this will lead you to many of his papers
online, which cover various parts of his work described in the book.


You might take a look at Braitenberg's book "Vehicles: experiments in
synthetic psychology", as he describe what you can do by sensing analog
signals and feeding processed analog to your motors.



You might take a look at Thrun's own words ...

http://robots.stanford.edu/papers/Thrun01h.html
===========
In the 1970s, most research in robotics presupposed the availability of
exact models, of robots and their environments. Little emphasis was
placed on sensing and the intrinsic limitations of modeling complex
physical phenomena. This changed in the mid-1980s, when the paradigm
shifted towards reactive techniques. Reactive controllers rely on
capable sensors to generate robot control. Rejections of models were
typical for researchers in this field. Since the mid-1990s, a new
approach has begun to emerge: probabilistic robotics. This approach
relies on statistical techniques to seamlessly integrate imperfect
models and imperfect sensing.
===========

If you check out what Thrun actually did on Stanley, the Darpa winning
vehicle, you'll see the real key to success was in integrating
short-range [out to 30' or so] Lidar maps with long-range video camera
images to plot a clear course ahead for the next few seconds.

I think I've read those words before. The idea of using probability as a
foundation of control systems extends at least back to Fuzzy logic in the
'60s (and the ideas continue to be used in real world applications today).
One of the guys I was chatting with at the DARPA meeting said he liked to
use Fuzzy logic for the low level control systems which actually operated
the various effectors combined with Bayesian probability to direct the
higher level decisions.


Yeah, I read the summary paper and have seen various videos.  The neat
thing to me was that instead of trying to integrate the two sensors to
construct an model of the terrain, he instead used the accurate Lidar data
to identify safe driving areas on the lower portion of the video.  He then
made the assumption that the top (aka more distant) areas on the video,
beyond the Lidar range, which looked similar, was probably also safe.  I
think he made it work by doing simple pixel level tests.  So instead of
using the video to try and construct an actuality 3D map of the terrain, he
just used it to identify safe, and not-safe, areas of the video.  That then
translated directly into which direction the car would need to steer to
stay in the safest areas, and, I assume, control it's speed based on how
much "safe road" there seemed to be ahead.

The point here is that the output needs to reduce to only two basic things
- which way to turn, and how fast to go.  It doesn't need cm resolution
terrain maps to make that decision.  When we, as humans, drive down the
road, we have very little clue about the actual terrain we are driving
over.  Instead, we have learned, through experience, what looks "safe" to
drive on, and what looks questionable or out right dangerous.  We drive by
keeping the car in the "safe" zone.  The total processing however, needs to
reduce the huge volumes of data coming from the sensors, down to very
simple outputs - turn a little left, or turn a little right.  The mid level
representations, (half way between raw sensor data and outputs) is likely
to be mid way in complexity between the two as well.  This mid level
representations are what we know of as our "understanding" of what we are
looking at, but yet, that understanding is nothing near what some of these
DARPA cars attempt to create for their internal representation of what they
are seeing.  What was used by Thrun I think is closer to what naturally
arises in our brains.

It seems to me that the answer to the question of which way to turn, is
just a large probability problem where many clues are being picked up from
the environment and each clue will add its bias to the answer.  Something
that looks like a car headed towards us on the left part of our vision will
bias us to turn to the right.  More of the same road we have been driving
over to the left, will bias us to turn to the left.  A shape that looks
like a line painted on the road will bias us to not drive over it.  All
these things and a million more will bias our decision about which way to
turn the car from second to second.  All the processing from sensory data
to effector data to me seems to be very probabilistic in nature - as does
all behavior.

--
Curt Welch                                            http://CurtWelch.Com/
curt@kcwc.com                                        http://NewsReader.Com/

I ran across the following blog via a link from sensors mag,

http://www.sensorsmag.com/0706/GSThrun

http://www.autoblog.com/2006/06/09/head-of-stanford-darpa-challenge-project-speaks-about-the-future/

To read some of the comments, there seems to be some question as to how
much work the Stanford guys did versus how much work the VW lab in Palo
Alto did, which appears to be specifically doing research on autonomous
vehicles ...

http://media.vw.com/article_display.cfm?article_id –16&uid2503
http://media.vw.com/article_display.cfm?article_id —01