Where is behavior AI now?



I think that fits well given my premise.

I don't think so. So far I think "behavior" as a word is so overloaded in meaning, it obscures what's going on, and the subcomponents of behavior are thereby going unnamed. Basic action states (music analogy notes) are what I think of as the real behaviors or as I am now calling them, atomic behaviors. A short fixed sequence of atomic behaviors which might be called an escape behavior as example (trill, or stanza, or maybe even a chorus, perhaps in the music analogy) I have no good name for to distinguish between atomic behaviors and this level. Mode doesn't really fit. Then there are long and complex things called behaviors, like "search for food", or "throw a tantrum", or what-have-you on human levels of action (music analogy melody or even opus). All these are simply called behaviors, and as such it is much to broad to be useful for the layering in behavior based robots.

Well, don't expect a large argument from me. I like reductionism in state machines, and I like to split state machines into even smaller and more insultated machines.
I find often the problem when someone designs a state machine, they make it too complex. Often a few simple machines, with intermachine communications, will have less states, less transitions, and be more robust as you say.
I've told the story several times before, but when I worked on a traffic light system, they wanted a state diagram of its operation. No one had ever been able to do it. I finally solved it by separating what what going on into several simple machines that interacted.
From problem too complex for anyone to solve, to problem solved with several small machines. If you didn't factor the machines like that, the number and complexity of the states just exploded.

Did you see my suggestion, in another post, though, that a robot that does an occassional or random change of state to escape, even if done at the wrong time, in the end looks more intelligent than a robot that does not?
A robot that always cruises will get trapped forever. A robot that occassionally throws in an escape sequence does not.

I think this is an argument of scale. While I get your point, what a curb is to nBot, a telephone pole laying on the ground is to jBot. You can build your robot for the expected environment, but it will still have to deal with those things at another scale, or it will not be robust.

So a robot with 200 foot wheels would be more robut than jBot, because there aren't many manmade structures it couldn't just roll over? What about a skyscrapper? Oh. It's just a mater of scale, and the problem returns at some point.

This is a really interesting derivation, because as I mentioned in another post, it can be used to reduce the number of states or atomic behavior. They way it does it is quite surprizing. By my definition of an atomic behavior, we have two variables instead of one, each with a gain function, and they are linear. Yet the output is not linear. It is a challenge to me to sort out if I think it is an atomic behavior or not.
I've run into a very similar thing in my walking robot. The center of turn when going straight is out at infinity. As it moves in toward the robot, the outer legs take their normal strokes, and the inner legs take shorter ones... until the center of rotation is under one leg, and it just stops and pivots. As the center of rotation mover under that leg and toward the center, the stroke of the leg reverses, and when the center of rotation is directly under the bot, the leg described is now making a full backwards stroke, and the robot's body rotates about a point.

This is attractive, yet fanciful. There are state changes going on. They have just been extracted from the output machine, and moved elsewhere.

But there must have been a mechanism (or transfer function if you prefer) that determined forward progress had stopped, and so rotational progress must begin.

Not knowing how the 0 value in rotate went up when the forward value went down, I am not able to fully comment, but the important point is, there was some function that did that transfer.
I would suggest, what you've done here is factor and hide your state information. Like the traffic light problem, you have reduced the complexity of the output machine, by separating the control to some other simpler machine.
Are you sure you haven't just moved the decision elsewhere?
The output routine uses two vectors. The issue is what fills the two vectors and how it does it.

I don't disagree, and I don't think it is heretical. I actually think it is pivotal. (no pun deliberately intended, but there's certainly one there now I see it written).
--
Randy M. Dumse
www.newmicros.com
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Randy M. Dumse wrote:

No. This is the misunderstanding I was attempting to correct. The normal navigation mode uses differential distance returned by the sonar array to steer the robot, and the combined distance returned by the sonar to control the robot speed. When the velocity vector is driven to zero, all that is left is the differential. Rotating in place is just normal steering with zero velocity. It need not be activated in a special circumstance because it is always active, steering the robot as a continuous, not descrete, function.
Again. there is no "mechanism that determined forward progress had stopped, and so rotational progress must begin." Indeed "rotational progress" need not "begin" because it is already, and continuously, active.
I think the fascination with continuous behavior as a series of static states may make it more difficult for one to understand, or even accept the possibility of, a different paradigm.
<snip>

Yeah. Nagle suggested that the IMU + Odometry navigation scheme on jBot was "straightforward." CRM is loaded with punsters.
Straightforward and pivotal. I think you have your "atomic" behaviors. As mentioned previously, seems like to make your argument you have to redefined "behavior" to mean "state," as you like to use the word. When I use the word behavior it has its own unique and richer meaning, not synonymous with the word "state" as you use it.
best, dpa
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Okay, but this is not what I'm asking about.

But that implies rotation is never zero itself.
I don't buy that. It doesn't fit your example.
In your example. The robot drove down the driveway straight. Otherwise it would have curved or careened into the alley. Instead there is an implication the differential is zero as it drove straight in.
As the forward velocity is reduced to zero as the combined ranges come down, it is not obvious there would be any differential information either.
If you drove in straight, then you are likely perpendicular to the walls, or so close to it, that the differential drive is not sufficient to turn the robot fast enough to get a response - unless your transfer function has weighting and is non linear.

Okay, then how does your robot ever go straight? (Perhaps you are saying it doesn't?)

I get that there is a continuous transfer, no problem for me in that concept.
What I don't get is what prevents them from both becoming zero and the robot stop?
In the Navy our missle control circuits had a little noise added to them. It was called dither. The purpose of this signal was to keep the system from getting wedged into one position. Previous posts you talked about putting a random element in the escape sequence to prevent sticking, same idea.
If you don't have some sort of mechanism to prevent 0 0, then your robot is very likely to get stuck looking at a close perpendicular wall. I haven't seen it do that, so I'm assuming you've done something to prevent that. If not, I'm surprized. But it isn't because I can understand your transfer function is continuous, it is because I don't think as stated it is robust.

To me, the word "state" implies something is "static". The atomic behaviors having variable outputs are somehow a more complex concept than something static. So I am suggesting the use of a constant linear alogrythm to calculate the output is the static part. So they are not truly states as I like to use the word.
Again, I feel the overloading of the word behavior is a road block to our progression in robotics now, so how behavior is amplified, split, augmented or replaced is not much my choice, only that new language which enables discussion of the various aspect of the behavior is my goal.
The reason I have chosen to keep the lowest level of behavior as its true meaning is not so much my choice, as I see it to be a Brooks/Jones choice. Brooks clearly prefers reactive behaviors. Jones clearly prefers servo behaviors. Both are the low level type. So I have been trying to name the low level types as the true behaviors.
If it should be that behavior maintains a meaning for complex sequences of atomic actions appearing to attain a common purpose or goal, I'm fine with that. Then, unscientific use of the word stands, and I can still spank a kid for bad behavior, without assigning which actual atomic action (touching the candy bar, gripping the bar, lifting the bar, unwrapping the bar, putting the bar into the mouth without paying for the bar, etc.).
But without the language to differentiate a big long behavior with variable branches from a little atomic behavior with no subparts and therefore also no branches, we aren't free to think about the problem with any precision.
--
Randy M. Dumse
www.newmicros.com
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Randy M. Dumse wrote: <snip>

Yes it is. You are asking how the algorithm works, and I'm attempting to describe it. Patience, grasshopper.

Yes. When there are that many sonar reflections rotation is never zero.

It exactly fits the example. This is not conjecture. I'm describing how the algorithm works to produce the actual behaviors that you yourself have witnessed.

There is always some differential with that many sonar detections.
Rotation can only go to zero (drive straight) in the absence of sonar reflections, which is also when velocity is driven to full speed -- out in the open. They can't both be zero.
If velocity is driven to zero because of many sonar reflections, they are ALWAYS asymetrical enough to garauntee non-zero rotation.
It's like balancing a pencil on its point. Sure it's possible, but in the real world it never happens.
(Given that this is all sonar driven behavior and not target acqusition of some kind, also running simultaneously but subsumed, driving the rotations and velocities).
I'm pretty certain I have described this all to you in real-time while watching the robot maneuver and herding it around with our feet. I guess I have not communicated what it's doing as well as I thought I had, which surprises me.
Or perhaps you were viewing it through the filter of "it must all be a series of discrete states," and came away with your own opinions of how it "must" be accomplishing these maneuvers?
We can try again! RBNO?
regards, dpa
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dpa wrote:

Now this helps, because I can remember seeing your robot come to a point and slow down to an "almost stop" and then slowly begin twisting away from some obstacle.
Talk about "emergence" and "in the eye of the beholder" - my external world view (more assignment of anima) to its action at that point, was, it is thinking of which way to go, and taking a very long time to consider it carefully with many sonar probings.
Instead you make me now aware, it was stuck against a flat spot and waiting to fall off one side or the other. Much less romantic.

This fits with the very slow turn away I saw.
As an aside, I hate the pencil balancing on its tip example, because it is offered as an objection to something I'm working on in my GR research.

Or I saw it's actions as louder than your words. :->

Yes, I saw that comment last time you made it, that I look at everything as a state machine and can't help myself, and grumbled at the suggestion, then too. I don't think you are correct on my bias here, but I can certainly see why you might think so.
Actually, I think I'm going to start a new thread, because I found a new comment from Brooks in Flesh and Blood that seems to apply to this discussion, and also fits my preset bias to see things in the light of state.
But in this case, I heard you say it was continuous, and I am willing to believe you, but wanted to ferret into it rather deeply to be sure there isn't some hidden trick that "tipped the pencil". So if I have a bias, it is probably completely antiopodal to yours. That is, I susect people are so biased about hidding state information, that they will do it, and swear up and down they haven't. Yet if you dig a little, you find it there.
For instance, I sitll wouldn't be surprised to find out you have a spot of code somewhere that says, if I've been sitting still over 20 seconds, see if we can't introduce some random noise, so we turn off to one side or another.

I was at the last two looking for a discussion, didn't see you. .
Randy
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Howdy,
RMDumse wrote: <snip>

No, "stuck against a flat spot" is not possible.

Alas, that's always the way it is with magic -- much less impressive once you know how the trick is done. I guess the lesson here is that a good magician never reveals his tricks... ;>)
<snip>

I understand that you have an intuition that the "missing piece" of artificial intelligence has something to do with state, and that the e=mc^2 of robotics may arise from this intuition. Perhaps you are right about this, I don't know. But I do know that this causes you to seek "state" -- even "hidden state" -- as a root principle of all you observe.
<snip>

Nope. Not needed.

Oops! My bad. Ok, next time for sure
regards, dpa
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
dpa wrote:

Not a flat spot then. Hummm... another thought occurs to me.
You know, you take a pencil, and push it into the ground an inch, and it will stand on its tip quite well. By the shape of the ground around the tip, through lateral resistances, it finds balance and stability.
Isn't there some shape if jBot approached it would find a similar support? For instance, if it happened on a parabolic U shape, wouldn't the differential term always favor turning toward the center of the U? Wouldn't it get stuck there? and require some other behavior to escape the stall?

Appologies for being absent at RBNO and contests. Some of my business dealings have turned south and requred attentions consuming all other interests.
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
RMDumse wrote:
<...>

Reminds me of a book I read as a teenager,
A.R. Luria on "The Functional Organization of the Brain"
http://www.comnet.ca/~pballan/Functsystems.htm
"A skilled movement is really a kinetic melody ... "
-- JC
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
JGCASEY wrote:

Thank you JC for that comment. It really fits beautifully, and gives me a platform to give an argument for my suggestion that behaviors are the smallest atomic components only of what now is too broadly called behavior, but also includes combinatioons of behaviors, which ought to have a different name.
I have suggested a new use of "behavior" is to mean "any static application of a output value (constant or linear variable) to an output device". Behaviors are reactionary, responses, and hold no inherent intelligence.
Again using the tone and the tune analogy and taking a ethological approach.
Consider a bird. What must a bird do to be heard in the world. Well, if the bird call is noise, it will not be heard, it cannot be identified, and it is without a very important survival tool to its species.
So what is the very most simple sound a bird could possible make to distinquish itself from noise, which would be useless. Well the opposite of noise, is a clear and pure tone. So any bird that can make a tone, any tone, is hugely more "situatied" than a bird that is silent, or one that makes noise. However better off, the bird is still not well identified by making a tone. Why? There are many unintelligent and non living things that can make a simple pure tone. For instance, if a wind blows over a hollow tube, a resonance will be set up and a steady tone ensue.
Like the silence in the OM chant, the bird can further show its life like ability by establishing a constant cycle (frequency of its tone) and then breaking that tone.
Now reach to see the point. A bird that can do nothing but cheep cheep cheep establishes indentity and a higher than inanimate intelligence this way. As other features are added, additional tones, additional silences, modifiers of the notes (trills, slides, etc.) the bird shows more and more identity, and increases its usefulness of this survival skill.
Closing the point: A tone alone is not enough to identify a sound as being animate, or inanimate origin. There is no intelligence in a tone. However, a sequencing of tones, a cycling of cycles, that shows intelligence. The intelligence arrises not from the tone, but from the melody (tune).
Likewise, there is no intelligence represented in atomic behaviors. It is the sequencing atomic behaviors that causes the observer to suspect and identify intelligence.
Looking to robots, a robot that can do one behavior, say "cruise" and drive straight ahead, we are not impressed. It has motion, but it is not very intelligent (in fact, we would likely say it has no intelligence at all).
Now a robot that has at least two behaviors, and sequences them, seems much more intelligent than the former. Say a robot has cruise and escape. This robot never runs into something and just stays there slipping its tires. This robot escapes.
Now a very interesting extension is that a robot that does random escapes between cruises, even when it isn't trapped, looks more intelligence than one with only cruise. A robot that does escapes when trapped looks even more intelligent.
Therefore I would say I have given a strong argument for the case, there is no intelligence in atomic behaviors, in and of themselves. There is intelligence in the sequencing of behaviors, whether they are appropraite or not, any sequencing of behaviors looks more intelligent than and atomic behavior which remains constant.
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Well, what I wrote wasn't an argument. It's simple mathematical fact. So I'm not sure what point you are trying to make, or what you didn't understand.

I really have no clue what you are thinking here.

If you read 40 bits of information, you have 40 bits of information. Is that really so hard to understand?
Now, if you want to talk information theory, then the information content of those 40 bits for a typical bot will in fact be quite a bit less than 40 bits because the bits you read are not statistically independent. Each time you read the switch at a high rate (like in my example) you have a high probability of it being the same as the last time you read it). And, if the bot has some algorithm for trying to avoid hitting things, then for most environments, the switch will be open far more than closed. This means the true information content of the bit stream will be far less than the 1 bit per bit read.

Yes, I didn't really get into the amount of information potentially embedded in the time domain for an application like that.

Sure, the information coming from a bumper switch on a bot is fairly useless above a certain temoral resolution. It's effectively noise.

There's far more useful information in it than that. Writing code to take advantage of the information might not be very easy, but the information is there none the less.
My point is not how much useful information might be in 1 second of bumper switch data. My point was that there is useful information in the history of what has happened over time. The bot can explore the entire world using only bumper switches and two wheels for driving to learn stuff about the world. The limit of the amount of information is really a function of the environment, and not the bot. It could drive around for 10 years, and build up detailed map data about the the entire world, having only binary bumper switch data to work with. It might have collected trillions of bits of raw bumper switch data in all that time, but the data might be compressed down to only a megabyte of map data because much of the switch data was redundant.
If the bot is located in a square room with nothing else in it, then all the data there is to collect, is the fact that you are in a square room. Once you figure that out, the information content from the bumper switches becomes effective 0 (because you can always predict what the bumper switch data will be). Now, for a real bot, that never happens, because there's all sorts of complex information about the bot itself mixed in the data (like maybe the motion batteries are running down so it's taking the bot longer to drive from one side of the room to the next - or the left wheel is starting to turn slower making it curve because a gear is wearing down). All that sort of data can be extracted from the bumper switch data given enough time in a real bot.

My point is the amount of data from the switches constantly increases based on how long you collect data from them. It's not limited to a few bits. If the bot drives around for years, it can might extract a megabyte of useful data from the environment though those simple binary inputs.

That could easily be true.

I can't really tell what point you are making here. If you put me inside a bot with two bumper switches for inputs, and two wheels for output (and lets say the wheels each have only three states - forward, stopped, or reverse - no variable speed control). I can still drive this thing around the environment, build up detailed map information, find out that I've been bumping into something that keeps moving, but other things that seem to be walls in a square room, then later figure out the thing that's been moving is actually another bot, with an intelligent drive. And even though they don't speak my natural language, after a few years of bumping into this other bot, we develop our own language, and start to communicate using a Morse-code like system. All that could be done with 2 outputs with 3 states each and two inputs with 2 states each. There's really no limit to the complexity of behavior you can generate if you have enough of the right type of logic inside the machine. As long as you have some outputs, and some inputs, then you can act intelligently.

BTW, I have no idea what jBot is, so maybe this is causing some confusion on my part. Should I look it up and become familiar with it to understand your points?

The complexity of behavior will be limited to the complexity of the controller inside the machine, not to the complexity of the I/O connections.

Ok, I'm a bit lost with what you thinking about. Are you talking about any machine, or the jBot?
In general, you can study the inputs and outputs and try to predict the behavior. But, depending on the machine design, it might be so hard to do that it's effectively impossible. I could for example, have a bot drive around a room in random directions where the "random" is defined by some standard PRNG. The machine could then record bumper switch hits, and collect bits of data using some technique such as recording a 0 if the left switch is hit first, and and 1 if the right switch is hit. It could then take about 1000 bits of collision data, and then run it though an encryption, or hash function, and then use the output of that, with some algorithm, to select the "random" directions it picks to turn.
You could spend 100 million dollars studying the behavior of bot like that and most likely, get no almost no where in being able to predict it's behavior, or create an equivalent machine to predict it's behavior. Now many, if you studied it for about million years, you could finally decode it's secretest. Some amount of study would no doubt do it. But the amount of study would be huge for what is in fact a very simple bot algorithm. Some machines are just inherently very hard to reverse engineer.

Yeah, I think that's true. Kind of. But I don't really understand what you think a behavior is. I would just say the intelligence is in it's outputs period.
A common technique for programming robots is to create a predefined output sequence and call that a "behavior". It might be something like, "drive forward for 1 second". Or "turn 90deg to the right". Then the system can use some higher level logic to trigger the selection of different behaviors. This allows bots to do useful things, but it doesn't make them look very smart. Humans and animals are far more flexible than that.
To me, that is just a two level system for the control of the outputs (the behavior generators at the low level, and the behavior selectors at the high level). In the end, the intelligence is in how the bot controls the outputs period - at all levels - not just the high level behavior selector.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Curt Welch wrote:

[...]
Could you, or would you, really do this? I rather doubt it. For a fair test you'd have to be confined to the bot with *no* other inputs. No food, no instructions, nothing. What seem like mere practical or implementation-specific obstacles may be more fundamental than we expect. If we cannot get a human to operate intelligently in a stimulus-poor environment, why should we expect a simpler system to do it?
Randy is onto something. The I/O complexity and properties are directly connected to the intelligence of the system. We can offer Helen Keller as counter-evidence, but she had a full set of senses up to the age of 1 1/2, later retaining touch, taste, olfaction, heat and cool, pain and a perfectly good set of muscular-skeletal actuators with built-in position and force sensors. Imagine if she had been born with only two single-jointed fingers bearing a few binary-valued touch sensors. She'd have wound up as a vegetable. Observe that even the simplest organisms are literally covered with sensors and muscular tissue. Complex senses and actions long preceded complex brains.
-- Joe Legris
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Ugh, I just wrote a long reply, hit the wrong button, and lost it. So now I have to write it again....
Humans do what they do, because they were motivated by rewards to do it. This is because we are reinforcement learning machines.
I didn't get into the subject of motivation in my example, but as you say, it is important to understand why we would do something.
If I locked a person in a bot control box which had only 2 lights to represent the bumper switches, and two buttons to make the bot move, but didn't tell the person what the switches were for, and gave them no motivation to figure it out, most people would never figure anything out.
But, if you told them that the only way they were going to get food, or anything else, would be to figure out the switches and lights, then that alone would act as motivation for them to start playing with the buttons.
But, all reinforcement learning machines must be rewarded for their effort, or else, the effort (behavior) will in time die out. So you will have to give the person some simple tasks that produce rewards to keep them going. Something they can figure out to cause food to drop into the room before they starve to death.
And to learn more, there has to be constant motivations to keep them experimenting and keep them learning. Maybe one set of actions gives them food. Another gives them water. Another turns the lights on or off. As the person plays with the buttons, and learns more about what they do, and what the lights mean, there has to be constant progress towards greater rewards, or else the person will simply stop developing greater bot-control skills.
No machine, or human, can learn a complex task in order to get a reward. You must learn the simple tasks first, and there must be a reward for doing it. You must learn to walk, before you learn to run, and there must be a reward for learning to walk, or else it will never be learned. And if walking is never learned, running will never be learned.
The behaviors we produce are always a result of the environment we are exposed to. And to produce typical complex human intelligent behavior, the environment must hand feed the complexity to us, step by step.

If you don't motivate it correctly, it will never learn. Simple as that. If you call it fair, to not give it a motivation, then of course it will not learn.

If you motivate the human correctly, it will do just fine in a stimulus poor environment.

Yes, she still had a large number of important sensors. But more important, is that she learned to behave "intelligently" because she had a teacher that worked with her to create an environment with the correct motivations to shape her behavior. Normal human environments (like a typical home or typical classroom) that work to shape our behavior, wouldn't have worked for her, because of her limited sensory ability. But, create the correct environment, with the correct rewards, and she learns to produce complex intelligent behavior like anyone.
Humans learn to do the complex things we do for the same reasons a dog learns to jump though a hoop on command. It's because we are motivated to learn these things. The only difference is that the human brain as the ability to learn far more behaviors, than a dog brain, and more important, we have hardware optimized for learning language behaviors.

Probably true. But not because she had limited senses, but instead, because a human brain is not optimized to deal with those senses.
The basics of intelligence is quite simple. It's just a problem of producing the correct behaviors, in the correct context. The context is defined by the sensory inputs (including their short term history). The "correct" behaviors, are the ones that produce the most long term rewards, for each context. The machine learns though experience, what type of behaviors, produce the best results, in a given context.
To create complex human like "intelligent" behavior, you simply need enough sensory inputs to create a complex enough context, and a machine with enough power, to create billions of different responses to that changing context.
With high bandwidth sensors, the system can create a very complex context with only a few milliseconds of sensory data (a picture is worth a thousand words and all that). The length of the short term memory time the bandwidth of the sensory systems defines how complex of a context it is responding to.
With simpler sensory systems, with less sensory information flowing in, the system simply needs to have a longer short term memory in order to create a similar sized context to respond to. A very limited sensory input can be adjusted for by having a very long short term memory. But the drawback is that it takes that much longer, to correctly understand the current context of the environment - and the quicker the context is changing, the harder it is to keep up with the changes if the sensory system is slow.

That's true. Because the more senses you have, the quicker you can understand the correct context of the environment. And that's a huge survival advantage. The quicker you can spot a predator (or any threat) that's going to do you harm, the more likely it is you are going to survive.
Imagine driving your little bot around for an hour bumping into a tiger about 100 times before you figure out it's a tiger like the one that killed your cell mate? The limited sensory inputs just mean it takes a lot long to collect enough data to recognize a complex object - to recognize the correct complex context of the environment and then produce the right behavior in response to that context (run for you life).
It's not that you can't make an intelligent brain with a limited sensory system. It's just that as long as you are going to have a complex brain, then it's optimal (for survival) to also invest in a large sensory system to speed up the system's ability to track the changing context of the environment. We need complex sensory systems because we are trying to survive in a complex environment that changes quickly.
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Yes. Apparently.
As you acknowlege concerning information therory, there really aren't 40 bits of information. There's less than one bit of information per read. I'm saying much less. So, there aren't trillions of inputs as suggested.
For example: Use as many processors as needed, say 40, and read a bump switch, all 40 at the same time. Compare the readings.
Are there really 40 bits of information there? or one bit of information copied/verified 40 times?
I am quite sure there is only 1 bit of information.
If there is more than 1 bit of information, (as per chance there "might" be) it is not 40 bits of information about the switch state. Some other number of bits about the system and its wiring (failure) might appear, but not a drop useful information more about the state of the switch itself.
You yourself made the rather insightful and critical comment earlier that you have to consider the time content.
"But, for anything interesting, the machine has to solve problems in the temporal domain. Which simply means, the current outputs, must be calculated not from just the current inputs, but from past inputs as well."
But it is quite wrong in general to assume the rate you sample something, like the bump switches, increases the information content in that input (if for no other reason than they have a mechanical limit in how fast they can even change state), which was my objection. And really, I'm thinking only the state transitions of the bump switch are of interest, and nothing of the detail of when they happen by reading ever and ever more often applies. The mechanical details of ths switch itself (wisker compliance, inertia, vibration modes, debounce, etc.) have much more to do with the timing on the details of the state changes, than the desired reading (did I bump something).
Again, my larger point was, our current robots seem to have very low content inputs, and limited states outputs.
And where I was going for the larger sake of the thread, is, we can probably map all the limited inputs to the limited outputs, list all behaviors (or even potential behaviors we haven't used) (oh, and behaviors in the sense I described in my post to dpa today) as a combination or permutation chart. Given we know all possible behaviors of our robots (being a limited set), then we can see the intelligence is not in having behaviors, but in enabling the transitions between the behaviors.
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Of course there is only bit of information when you sample it once. You seem to have missed the point of my message.

That is the main point of my post. When we have to map input, to output, we can not create a mapping function which maps one bit of input, to the output just because we have a one bit bump switch. We must instead read and record, the activity of the inputs for some amount of time, and then producing a mapping function, which takes the entire recorded history as input into the function.

Which is why I never said anything like that.

That's all fine and true. But, to produce "intelligent" behavior, we can't do it using only the last or current switch position to make an intelligent selection of what behavior to do next. The logic must be based on some history of what has been happening, even if that history is only the switch transactions and the order they happened in for the past 30 seconds.
For example, as was already mentioned, a simple robot that responds to only the current switch position, might get stuck in a corner, and bounce back and forth for a long time. But, if the robot is able to "remember" it's bounced back and forth 5 times in the past 10 seconds, that can trigger it to evoke a different behavior to escape the corner. But this means, the "trigger" that selects the correct current behavior, is not just based on the current switch setting (1 bit), but instead is based on something like the past 5 switch transitions (much more than 1 bit) of data.

And again, the low content is not a limit, because even though it's low, you need to record a history of that low content in order to act more intelligent. And once you start recording a history, the amount of content the system is using to make each behavior selection decision, is no longer so low. And the longer it can record input content (aka the further back in time it can remember), the more intelligent it can act (the more information it has to base its decisions on). So the limit to the intelligence the bot will be able to demonstrate, has as much to do with how much memory it has for past events, as it does with the flow rate of the sensors.

The size of that chart will grow exponentially with the amount of sensory data you choose to record. And even though you start with simple 1 bit input sensors, if you record 100 transitions, you don't have a table with 2 states, you have a table with 2^100 states. So looking at the input states alone is not enough. You have to also look at how much memory the system has to record and make use of history. The complexity of the behavior will be a function of how much memory the system has, and how much it makes use of to create triggers for complex behaviors.

Ok, but again, that wording is silly.
Lets say I have binary output that produces a constant strings of 1s and 0s. We will ignore what it actually controls.
This type of device has just 2 low level behaviors. It can send a 1, or it can send a 0. That's all it can do. All other higher level behaviors are created by stringing together different combinations of 1s and 0s.
So, the total intelligence of this box is determined by how it makes the decision to send a 1, or a 0 each time.
But we could look at each two bits sent, and say it has 4 behaviors. It can send, 00, 01, 10, or 11. Then we can say, as you said above, that the intelligence was not in the behaviors, but in the decision to transition to the next behavior. So, if it sends 0010, we would say that sending 00 was a behavior which was not intelligent, and 10 was a behavior which was not intelligent, but decision to send 10 after 00, was the intelligent act.
See how stupid that is? Why is the transition from the first to the second bit, "not intelligent" but the transition from the 3rd to 4th is? It's just silly to talk like that.
All outputs (aka all output decisions) are how the intelligence of the machine is demonstrated externally.

--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Curt Welch wrote:

Ah! I get the problem between us now. It is simply one of context. Since I'd opened with Brooks behavior based robots as a premise, I've been speaking according to Brooks rules. You are speaking from outside his rules. Otherwise, I think we are largely otherwise in agreement.
Brooks requires inputs to be horizontally applied to the behaviors. In the behaviors there can be state machines. But the idea of something outside a behavior which can "read and record the activity of the inputs for some amount of time, and then producing a mapping function" is completely alien in a Brooks architecture. In fact anything that is a mapping function of any kind is completely alien in a Brooks architecture.

Now for my part, I get state. I love state and the use of state. You'd be hard pressed to find someone more a fan of state as a concept than I. Brooks also admits state. Explicitly btw. However, Brooks would likely take exceptions the the "we can't do it" because his book _Cambrian Intelligence_ is practically at a persistently loud level all the way through diatribe of, "Oh Yes We Can! and this robot experiment proves it" I have never seen an instance where Brooks has said something cannot be done with anything but reactive responses. While Jones far favors reactive "servo" responses, he acknowleges you may have to use state (non-servo ballistic manuevers) sometimes for escapes. I can remember anything like that kind of concession in Brooks.
Yes, by recording past transitions we can make more intelligent decisions. We can agree on that. However, it is never necessary to record all past history. One of my own definitions of "state information" is that is the minimum amount of information to be captured to adequately permit all required future decisions.
Randy www.newmicros.com
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Right. I've not studied Brook's work. I've only scanned over some of it. So I'm talking general theory and not specific what Brooks's systems might suggest.

No, I think his architecture is a mapping function. Any system that takes data as input and produces some result dependent on the data is a mapping function to me.
So, are you saying the system can change state in response to any input at the same time it's producing some output? If so, then that state change _IS_ short term memory which is recording some historic aspect of the inputs. If there are no limits to what the state changes can include, then his system does in fact allow a complete recording of state for the last N inputs.
As a simple example to show you what I mean, assume you have a single binary input as the sensory, and you wanted the system to record the last 3 inputs. How many states does that take? 8 (aka 2^3). So if the machine has 8 internal state, then every state can represent one unique pattern of the last three inputs. You then specify the state changes as a function of the current state, and the current input to accurate reflect the last 3 inputs at all times. That gives you a system that has a short term memory that records the last three inputs.
If you want the last 1000, you just need a few (ha) more states. :)
Is that consistent with his use of states?

If he allows state that changes as a function of the current state and the current inputs, then he's talking about the exact same thing I am just using different names.

For sure. And in most real world cases, it's not even possible for any reasonable life span because there's just too much data. The state however tends to summarize a large amount of past inputs however - by calculating things like an average for example.

Yes, that's always key. As a matter of fact, many of the techniques I've played with used that same assumption in a slightly different direction. If you have a learning machine with finite amount of state memory (aka short term memory), it must maximize the amount of potentially useful sensory data it stores in the memory in order to maximize the odds of sorting out the useful state information from the non useful state information as it learns. A key part of that process, is the removal of duplicate information, and the equalization of the amount of information stored in each state variable. You don't want to store what amounts to the same information 10 times, because some fact about the environment came to you through 10 different sensors at the same time. This type of thinking is important in understanding how you must (or can) transform and combine sensory data before you try to use it to control behavior.

--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Yes, I can see mapping from that view. I should quote from Brooks to give you a sense of his point of view.

I've never thought of state information as short term memory (STM). Why would it be STM vs. LTM? What would LTM be, if not state information?

Agreed.
But usually you don't need to record inputs exactly as presented.

Is it consistent with _my_ use of states? Yes.
Is that consistent with his use of states? Oh, how I wish I could tell you! Brooks gives frustratingly little insight to his AFSMs.
In the first paper in "Cambrian Intelligence" the Augmented stands for added variables. The fullest explanation I've found is on page 13 in 3.1 of the first paper:
"Each module, or processor, is a finite state machine, augmented with some instance variables which can actually hold LISP data structures.
"Each module has an number of input ines and a number of output lines. Input lines have single element buffers. The most recently arrived message is always available for inspection. Messages can be lost if a new on arrives on an input line before the last was inspected."
He shows a little code there, but I'm not familiar with its syntax.
By the second paper it stands for Timers added to FSM's, where the communications wires between states have limited durations.
Trying to penetrate more deeply into what a FSM is to Brooks, if his model resembles what I understand them to be, is difficult.
To give you a flavor of what I've been saying about Brooks two of his papers are entitled "Intelligence without Representation" and "Intelligence without Reason". In the latter of representations he says,
"I make it clear in the paper that I reject traditional Artifical Intelligece representation schemes... I also made it clear that I reject explicit representations of goals within the machine."
and later,
"Her representations were totally decentralized and non-manipulable, and there is certainly no central control which build, maintain, or uses the maps."

Yes, averaging, or just remembering the last pass compared to the current one, or keeping counts, which is a way of condensing state information, however it also hides state information, and therefore should be done with care.

Well, I look at the state information a bit different way. I try to only pick up the essential data to begin with, and ignore the irrelavant. So I generally don't track inputs, and later remove duplicate information.
Let me give an example of this. Say I am implementing a combination lock with a telephone like keypad. The main input are the numbers 0-9. Let's say there is a four digit sequence necessary to open the lock, followed by a # to enter and try the combination.
The way I hear you are suggesting is I need a 10,000 state machine to track this input, 9999 and #. Actually the * and # complicate this matter, so perhaps I need a 12^5th number of states. Now I see this as an "input centric" approach to the problem.
My approach which I would see as a "data centeric", "content centeric" or perhaps a "minimal-necessary-historic content" approach, would be to have 5 states. The first state would have a transition out to the second state when the first correct number was pressed. Each following state would have a similar transition to the next state for a correct press, and a transition back to the first state if the wrong number were pressed. Four correct presses is the only way through the states to the final #-key test state that would operate the bolt.
In essense, the state information would only contain a very limited range (for 5 states, or 5 different values only) rather than 10000+. The wrong strokes are deliberately forgotten as irrelavent. No memory of their being made is kept. (Although we could make another machine that counted wrong attempts, if we wanted a fancier lock, but unless extra features are required, there's no sense in remembering undesired inputs. If I really needed to retain the inputs in original form, a circular rotating pointer buffer would be far more efficient than a state machine.)

I understand this processing to be sensor fusion. Do you use the same term? There's a nice series of articles on sensor fusion and multiple sensors running in Servo magazine just now. (Note though it has some terrible errors in it. For instance the conversion rate of Sharp sensors are stated to be 1000x what they actually are.)
--
Randy M. Dumse
www.newmicros.com
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Both STM and LTM are state. The only difference is how far back in time the state is representing a past input.
For your example of a machine that responded differently to one light flash or two, it would have to record state information about the last flash. But if two flashes have to happen within one second for the machine to consider it two flashes instead of one, the state doesn't need to record anything more than one second old. I would state that only recorded information bout what happened in the last 1 second sort term memory.
Now, on the other hand, if a machine was building up a map of the environment it was exploring, and that map was going to stay around forever, then I would call that long term state.
The distinction isn't absolute, it's fairly arbitrary. How long does the state have to stay around before you call it LTM? Pick a number you like. :)
But in the context of the learning machines I've been looking at, it takes on a slightly different meaning to me. If you want to build a general purpose learning machine that can learn to react differently to two light flashes, than one it must have generic state memory to work with. And, unlike us, as programmers who know what we want to make the machine react to, a learning machine doesn't know ahead of time what state information it will need to store. Should it store the number of flashes? Or the length of the last flash? Or how long it's been since the last flash? Or what? A generic learning machine needs to try and store "everything" about past stimulus events to some extent and then produce behaviors which are a function of the current state. Through experience, it must learn that only the number of flashes is important and not for example, the amount of time each light was on, or the exact time between the two flashes.
This type of machine needs memory to store information about past sensory conditions, but since memory is always limited, there's always a limit to how far back in time the state will cover. This state that defines the context the machine is reacting to, is what I call short term memory - because even with a very large machine, the memory is only going to stare a very short amount of sensory information.
But the same type of learning machine needs to learn the value of each reaction it's producing. This is information which it needs to be tracked long term - meaning, for the entire life of the machine what is the average amount of reward each behavior ended up being associated with? This long term statistical information about rewards, is what I call the long term memory of this type of learning machine.
So a reinforcement learning machine that can learn to respond to different temporal sensory patterns, will need a form of short term memory which is the state that defines the current sensory context the machine is responding to, and long term memory of the statical value of every atomic behavior the machine can produce.

Right. They are normally summarized because the behaviors you want to produce are seldom a function of all input data, but just some limited amount of information in the inputs. The light sensor might tell you exactly how long the light has been on, and off, and what the light level is. But, if the robot only needs to count light flashes, it can turn the light level into a simple ON or OFF value by comparing it to a threshold, and it can track on and off transitions and count up to 2. All other information in the sensor can just be filtered out, or thrown away, and not recorded in the state.
Most systems will have more information flowing into it from the sensors, than flowing out in terms of output behaviors. This is very true for humans and I assume most if not all animals. The data following into the system though the sensors is far greater than what it needs to send out - so much data is filtered in the process - we see things and just ignore them - we don't even realize we saw them.

Yeah, he's really just getting into semantics there. His approach is a different way of structuring the specification, but like object oriented vs structured programing, the difference is only semantic and not substantive - they are Turing machines either way.
If there are no limits on the number of states his modules can have, then they are complete Turning machines - even if it might quickly become impractical to specify the module when it has 2^1000 states. (aka 1000 bits of state information).

Yes, exactly. The purpose of the states are only to allow the machine to produce the outputs it needs to produce. If you know exactly what conditions it needs to produce the different outputs, you only need enough state information to track the sensory information which is important to those output decisions. And for many practical problems, that means very little state information is needed. But if you want to look at how much is available, then you see there is huge amount of information in the sensory inputs, which for a given need, we don't care about, but is there none the less (like the timing of the keys hit (measured to micro second accuracy) and how long he held down each key). All that is in the sensory data - but since we don't need it for a given application, we don't save it - we filter it out.
But, as I talked about above, if you are trying to build a learning machine (like I play with), and don't know ahead of time what sensory information is going to be needed, the problem gets a bit more interesting. The system must save as much information as it can.

It's a term I'm learning to use because of my recent exposure to robotics. It's a good term.

Yeah, this is where getting into Robotics is exposing me to some useful new ideas and techniques. It's why I think robotics is good for AI because it puts some simple real world problems in front of you to solve that are important stepping stones to harder AI problems (such as how to solve generic sensor fusion problems for a generic learning machine when you don't know ahead of time what the sensory data "means" and what needs to be fused or how it needs to be fused).
--
Curt Welch http://CurtWelch.Com /
snipped-for-privacy@kcwc.com http://NewsReader.Com /
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Hey Randy, since you never acknowledged my earlier post regards CiteSeer, I thought I'd do some homework for you to show you how useful and powerful it is .... note the ability to download PDF files, etc.
http://citeseer.ist.psu.edu/cis?q=state+machines --> 4606 docos http://citeseer.ist.psu.edu/alur92hybrid.html http://citeseer.ist.psu.edu/alur94theory.html http://citeseer.ist.psu.edu/henzinger98about.html
http://citeseer.ist.psu.edu/cis?q=robot+navigation --> 1619 files http://citeseer.ist.psu.edu/16153.html
http://citeseer.ist.psu.edu/cis?q=genetic+algorithms --> 8361 docos http://citeseer.ist.psu.edu/floreano96evolution.html http://citeseer.ist.psu.edu/yao93review.html
Also http://citeseer.ist.psu.edu/ye02characterization.html --> SICK http://citeseer.ist.psu.edu/spero04review.html
On and on.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Acknowledged.
Wow, I keep a separate section here for books on state machines, and I've had a hard time filling an "apple crate" full of them. I count 14 at the moment. I subscibed to FSA-research to look for activity, and have seen less than two dozen posts (other than my own trying to start conversations) in two years.
And here you find more references than I can hope to read from your CiteSeer search.
Thanks.
--
Randy M. Dumse
www.newmicros.com
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Polytechforum.com is a website by engineers for engineers. It is not affiliated with any of manufacturers or vendors discussed here. All logos and trade names are the property of their respective owners.