DARPA Grand Challenge

On Tue, Oct 11, 2005 at 11:22:11AM -0700, Padu wrote:


Similar experience here - our team, Insight Racing, launched about 10 minutes after Terra Max. I was video taping our truck moving off in the distance, probably about a mile out when all the sudden I hear the crowd "ooooohhhh" off to my right and turned to see Alice nosed up on the berm looking skyward.
It took forever for DARPA to clear Alice from the course and we were paused for a while along with Terra Max and the others behind here. When we did finally come back by the spectator stand, you might remember use - the white Chevy Suburban that was weaving and canyoning down the road. Our magnetic compass was apparently off by 10 or 15 degrees though we had just calibrated the day before and it was on perfect. Perhaps all the power lines down that 8 mile dry lake bed had a bad effect on it we don't know.
Regarding Terra Max saying the wind was high - it was very much so. We were experiencing heavy dust clouds affecting our LIDAR and the high winds also broke a strap tha supports the LIDAR sun screen, causing it to flap down in front of the LIDAR intermittently. The LIDAR is used for obstacle avoidance and the flapping was causing an "obstacle" to termittantly appear right in front which was also contributing to the weaving.
When we saw it pass us by the spectator area in that condition we thought it won't be long now. We really didn't want to go down like Alice, though, right in front of all those people. But the "Desert Rat" was tenacious and did not give up - we went another 20 miles like that, passing the likes of Princeton, UCLA, Cornell and Cal Tech.
The frustrating thing is that if we had not had the above problems, we could have gone quite a bit farther. But that's the breaks. We aren't complaining.
Regarding Ensco, we were in the bay next to them at the NQE and got to know some of their team. Very nice folks. The funny thing is, or not so funny, is that when we were filling our tires with Fix-A-Flat, they were shining theirs up with Armorall. Later, after the race, one of their team members told us that they wished they had been using Fix-A-Flat instead :-)
The web site was not so hot at tracking. But DARPA had a 3-D display in the "Map Room" where the whole course region was modeled on a 3-d terrain relief surface map. Above the map, projectors projected the whole course down from above along with each vehicle's position and status, updated in real time. You could literally watch the vehicle number's move along the course at their course-relative speed. While you could not tell if a vehicle was paused (its color did not change or anything, it just didn't move), the color would change to red if the vehicle was e-stopped and out of the race. The "Map Room" was very cool, but its downside was that it was too limited by space and a long line formed outside that tent limiting access to it.
Also, there was a long time when Terra Max was paused. The reason I heard on pretty good authority was that their chase vehicle broke down - it overheated, or they had radio problems, or something like that. So Terra Max was paused for a very long time due to no fault of their own. It was always part of DARPA's plan, announced in team leader meetings prior to the race that if they needed to shut down vehicles and continue the next day to complete, that they would. Terra Max and the Gray Team both were subject to this. One of the reasons given was that in westerly directions during the late afternoon/evening hours, the sun is very bright in the desert with it being low on the horizon and presented a safety problem for both the robots and chase vehicles. So the plan was to shut down at 6pm and start up again at 6am the next morning. DARPA actually announced that The Gray team and Terra Max would be stopped and restarted the next morning. But the Gray team convinced DARPA to let them continue, that they did not have any sensors that would be affected by the setting sun and DARPA let them continue and finish on Saturday. Terra Max stopped and finished up the next day.
-Brian
--
Brian Dean
ATmega128 based MAVRIC controllers
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Hi Brian, How do you feel about a civilian robot race? Would you enter your vehicle?
thanks Ed
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
On Sun, Oct 16, 2005 at 07:01:13AM +0000, newtype wrote:

Well, technically it's not my vehicle - I'm just 1 of a 9 person team. Regarding a follow-up race, I doubt it. First off, I'm tired. It was a lot of effort to do the DGC, and I could use a break. Second, a lot of the appeal to me was the challenge of doing something that no one had ever done before. But now that it has been done, a lot of the appeal for a follow-up is now gone as well. But if a follow-up was significantly different and presented new challenges, then yes, maybe.
-Brian
--
Brian Dean
ATmega128 based MAVRIC controllers
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
"Brian Dean" wrote

Cool, so you guys were the Desert Rats! I was cheering for you guys and for Monstermoto. I love to see people with very small budgets beating over engineered limitless budgets. Congrats to your team.

I wonder if magnetic compasses are really worthy for this type of application. If you had to redesign your vehicle, would you still include it in it?

Indeed, one of the guys from the Terramax team was wearing a hat. We were at that back fence waiting for Terramax to be unpaused and the wind took his hat away... I was wondering how funny (or not) and unprobable it would be to terramax to think the hat was an obstacle, take a detour and crash onto the barriers just like alice. 8^)

Kudos!
<snip>
Exactly my thoughts... One easy thing they could've done is to position a camera on top of that map and broadcast the image of that map, that would be a thousand times better than that one they were using.
Final question to you guys: was it worthy? Would you enter a competition like that again?
It would be very nice if you guys could write a short post-mortem of your project. I know I'd be very interested.
Cheers
Padu
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
On Mon, Oct 17, 2005 at 11:56:50AM -0700, Padu wrote:

Thanks. We think we won if you measure by miles per dollar :-)

We're not yet sure that was the problem, it's just a theory. We need to analyze the log data to be sure. We just got that back a day or so ago, so we can dig into it and see what was up. But the unit we used is usually quite good, although can be a little finicky at times - it is very sensitive. It's an AHRS with builtin magnetometers so it actually gives us quite a bit of information besides just heading. It gives us pitch, yaw, roll which we can use to determine whether we are on an incline, detect vibration (in order to slow down on rough sections), etc. We are quite happy with the device.
Our Novatel GPS also gives excellent heading when the truck is in motion, but poor heading when sitting still. The AHRS is reliable regardless of motion. Also, the GPS heading flips when we reverse, unlike the AHRS unit which always gives the heading the truck is pointed in. We should probably fuse the two sensors so that the best of both complement each other.

Irony for sure. That guy would be very unpopular on the team after that, I'm sure!

That's a great idea - I wish DARPA would have thought of it.

While I'm tired from the effort, my participation on the team is something I will remember for the rest of my life. I made some good friends and learned a lot of things I thought I already knew. It was a great experience. Those things are priceless, no matter how you slice it.
Would I do it again? Not a competition exactly like that, but maybe something similar in that it is in a field that is of interest to me and the nature of the event is interesting itself. The GC was interesting in part because it was so crazy to think anyone could create a fully autonomous ground vehicle that could successfully navigate 140 miles through an unknown rugged off-road region at useful speeds. That had never been done before, which was part of its appeal to me.
But now that it has been done, doing it again would be sort've anticlimactic, IMHO.
But if another competition came along that had similar characteristics as the DGC, then yes, most likely I would be interested in that as well, just like the original DGC was so interesting to me.

We most certainly will do this.

Likewise.
-Brian
--
Brian Dean
ATmega128 based MAVRIC controllers
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Brian Dean wrote:

I think this characterization is a little misleading, considering the precisely specified GPS course laid out. GPS was really the sine qua non of the whole venture.
This is not to belittle the achievement, but I think some of the exultations of success ( which attracted my attention ) were overblown. This was being touted as an AI breakthrough, but consider this from the "The Tartan" on the CMU "Red Team(s)" :
While there is an extensive amount of attention to hardware, [the] most powerful technology is preplanning the curves in the hours or so before the race when the route is revealed, Whittaker said. We compute the terrain and the details of where were going to drive and how fast were going to drive, and that is definitely the thinking part of the race.
"definitely the thinking part of the race ..." Compare and contrast the humble bee, plunging into the UNKNOWN, sans GPS, sans LIDAR, with only its vision and its ever so tiny brain to rely upon.
    Some say that unto bees a share is given Of the Divine Intelligence ...
- Virgil Lew Mammel, Jr.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
"Lewis Mammel"

<snip>
Much has been said here lately on this subject. I agree that GPS was one of the most utilized sensors in the competition, but so it was last year and results were very different (although I also agree that this year the route was a little bit simpler than last year's). For the sake of robotics field, it would be nice to see vehicles with more intelligent vision systems, and I deeply believe that the real intelligence is in vision.
But look at the two other points of view:
1) DARPA has a very concrete goal in mind. Given a convoy of trucks that must deliver supplies from Bagda to Basra, make the trucks completely autonomous. GPS will most likely be available (and military GPS is less succeptible to jamming), aerial photography will most likely be available (and military aerial photography has much better resolution than the ones available to CMU or any other team for example). Roads will probably be even better that the ones we saw at the GC. So, in DARPA's point of view the event was really a success. For us half assed scientists (or the eventual full assed ones 8^) ) it was not enough. Of course I wanted to see a course with mud, snow, climbs, descents, forests and what not... but I don't think that's really pragmatic.
2) Although human vision system still cannot be compared with machine vision, for a course like the GC, we would also need a GPS or at least a map. Although most of navigation was done using GPS, there was collision avoidance present in all vehicles, and we were able to see a demonstration of it on the NQE.
Writing this second view, one question raised on my mind. I believe that according to the course DARPA chose, it was possible to two vehicles to be on the same road on opposite directions. How road sharing was managed when vehicles were about to cross each other? If such thing happened and both vehicles were allowed to be in movement, I take my hat out for them. Such capability is fundamental for real world autonomous navigation.
Padu
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Padu wrote:

I wonder about the utility of autonomous truck convoys to the military though. For the foreseeable future they're still going to need human guards of some kind. I think the utility for an autonomous device would be more in the nature of a recon or weapons delivery platform--something that goes in harm's way in lieu of a human.

--
--John
to email, dial "usenet" and validate
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Your forgetting GPS data was a requirement last year and this year. A CD was given out with GPS data and you had to pass over the GPS points, at the required speed for that GPS data point.

You can see in the dark, unless you use ultrasonic, FLIR? SONAR? SAR? Light seeable edge detection is not that easy and how are you going to see a water puddle or pond? that water puddle cause a car to wreck itself last year. What about driving in a sand storm, or fog when you can't see 1 ft. in front of you, heavy rain?
I'm not clear about your definition of intelligent vision.
thanks
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
newtype wrote:

Starlight scope.

Which is one of the reasons that vision is the real challenge.

Another reason it's the real challenge.

Then you use alternative sensors. Just remember, though, on the modern battlefield he who radiates is lost.

--
--John
to email, dial "usenet" and validate
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Not sure I understood what is your point here, but anyway, my point is that I don't see the use of GPS as "cheating" or a bad thing. Although, if you only rely on GPS for navigation, sooner or later you will be doomed.

You are misundertanding vision (as in vision algorithms) with human vision. We can only see in a limited spectrum of light. Cameras can use a much broader spectrum. My point is: it doesn't matter what sensor you are using, the real challenge is making good sense of the data you are receiving. We humans can see a water puddle with our vision, so the computer should be able to do it, it all depends on the intelligence is givem to it. As mentioned above, there is an additional advantage to robots, they are not limited to one type of sensor as we are. How do you drive through a sandstorm or through heavy fog?

That's because I did not define it. What I asserted is that the real intelligence [yet to be found] is in vision. You see, identify that there is a blob in a picture frame is already not a simple thing. Ok, in ideal conditions it is, but the real world is far from mathematical ideal. Light conditions change, lenses get dirt, etc. But assume you got that part nailed and you have the image nicely structured into objects. Now the real problem is to recognize what are those objects and what do they mean. For example, it would be nice for an autonomous tank vision system to identify bushes or soft objects from untraversable ones. I think all falls into the discussion if one day we will be able to have a strong AI, able to implement our horizontal knowledge (or learn its own) and common sense.
Cheers
Padu
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Good to hear a first hand report - I was rooting for CMU, being an alumnus of them myself and while I was disappointed in their failure to take the win, I feel they have nothing to be ashamed of.
I feel a lot of teams hit upon some hard luck, and as I said "those are the breaks".
I don't hold any ill-feelings towards Stanford at all - they had a very fine machine and they finished first, and that may probably had a lot to do with their anticipating whatever breaks may have come, be those breaks dust storms, glaring sunlight or flat tires.
Even though I had nothing to do with Red Team itself other than having been a student there at one time, they performed admirably.
As did the other teams.
I was a bit disappointed in the "webcast" as it was - I was hoping for some video or at the very least more real time updates, and as I have mentioned in another post, the "real-time" standings weren't very realistic - I imagine everyone involved was too busy to update all of us spectators on the web.

From the (quasi) real-time standings, they were on pace to finish within minutes of Stanford and the Red Teams and possibly even beat them - it was hard to tell since the "webcast" didn't take into account pauses in the race.
It must have been very disappointing for them to have been shut out by a failure of a tire.

I believe a guy from Stanford (if not him, someone else) mentioned that last year's race had one of the tougher stretches early on, while this year's race left the toughest (Beer Bottle Pass) for those who could travel 125 miles in the first place.

Only 10 less though

I can only imagine.

and IMO kudos to everyone who qualified

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
"Ben" wrote

Nothing at all! The results were very close indeed.

As that Montmerlo guy said on the webcast, their success comes mostly from lots and lots of testing under different circunstances. If you never achieved 20 miles of autonomous drive, you cannot expect to achieve 132.

ditto
Well, the live webcasts at the spectator tent weren't much better. If they really wanted to do a live webcast, they would have live video in each one of the shadow cars. That would really be great.

Specially when the first place is an almost stock SUV while you are running on a custom made off-road vehicle.

Yes, that was Montmerlo from stanford.

I don't know why but I was expecting 175 miles, and I was not the only one. But, from the results, even if it was 200 miles, chances are that results would not be too different.
Cheers
Padu
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
On 8 Oct 2005 07:29:18 -0700, snipped-for-privacy@gmail.com wrote:

Did anyone notice around noon PST yesterday all the standings were revised to reflect fewer miles?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
On Sun, Oct 09, 2005 at 01:01:25PM -0700, Ben wrote:

I am on one of the teams and our chase vehicle guy said that the gas company dug up a section of the course and it was too late to fill it back in and/or the fill was too soft for the vehicles to cross or something like that. This was all unexpected and so DARPA had to scramble to short circuit an 11 mile section of the course to avoid that area. But it appears that the vehicle tracking system did not account of the 11 mile shorter section until later and that is the discrepancy.
-Brian
--
Brian Dean
ATmega128 based MAVRIC controllers
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
These competitions seem like a far better investment of tax dollars than the traditional method of doling out funds to researchers based on proposals.
I have been involved in several projects funded by grants, including one multi-million $ NSF grant. There was a big celebration when we got the grant, but after that, there was little urgency in actually doing anything. Much of the money was spent on administration and bureaucracy. The only results were a few papers printed in some obscure journal that is now gathering dust on a library shelf. I wonder how different it would have been if we only got the money after getting concrete results, and we knew there were other highly motivated teams trying to get there first.
The Ansari X Prize is another good example of competition vs central planning (even though it was privately funded).
-bob
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I couldn't agree with you more, it often seems that big governments main objective is to spend as much as possible on as little as possible. Keeps the economy going.
It's great to have some concrete results, it's even nicer to see a competition where anyone can be involved.
Joe
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Oh! Yes, I think it's quite a good rip off of the public, to offer a $2 million dollar prize, reap the benefit of maybe $40 million in expenditure of engineering hours, vehicles, equipment and supplies, that would have possibly been spend on useful projects, and spend $9 million just putting on this year's event. So the government got to waste millions on adminstration as they always must, and bilked millions out of the hopeful.
Then they under-rewarded only one of the hundreds who tried.
And just like any good government project, now that it is over, nobody has anything useful or comercial in hand, to show for their efforts.
Plus, with their careful manipulation of who was "selected" and who was culled, all we had was a bunch of comercial vehicles with computers strapped on. None of the "who knows what may slither across the starting line" they originally envisioned.
As you say, it was a "far better investment of tax dollars" but as a whole, the robotic community has been literally impoverished by it, maybe to the tune of $40 million. This is not the kind of dealings I think any honorable government should engage in. How can you?
--
Randy M. Dumse

Caution: Objects in mirror are more confused than they appear.
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

The public also benefits from the research done. You have to balance the cost against the benefit, but your analysis only mentions the cost. What other robotics projects are there that these researchers would be working on instead? Are those projects more worthwhile? I'm sure the Universities involved would have spent the money one way or the other. The private teams had sponsors which used advertising money to further robotics research. This is a good thing, because without the competition, this money would not have been available. This means more roboticists get more money, and more time, to work on what they love doing.

Yes, government bureaucracy is inefficient. Unfortunately they have the same budget every year, so at least we should be happy when they do something useful with it.

The results are useful, and commercial. It takes a leader with vision to advance these ideas. I hope the teams who lost don't underestimate the value of what they have. I wouldn't be too surprised to see automated electic carts in use in airports in the future, rather than the manually driven carts they have now. All you need is a flashing lights and a beeping sound and they expect everyone to get out of the way anyway. Could it be all that difficult?

What do you mean? It wasn't open to everyone?

Why ignore nearly a century of research of what works best crossing a desert. SUVs are very good at crossing the landscapes found on earth because thats what they were developed to do. The emphasis really was on autonomous navigation, and the vehicle only had to be able to cross the landscape. Perhaps if the challenge were retrieving molten magma from a volcano, then you would get what you might be imagining. Its not always necessary to reinvent the wheel, especially when your objective has the same requirements as the first wheel had when it was invented.

Your ignoring the sponsors, the value of the knowledge gained, and the advertising value of having the event. A lot of attention and press has been spent on these teams. I'm sure there will be many oportunities which come from thier participation alone.

I think you are ignoring more than half of the picture. I see it as the great success it is. I'm sure you will benefit from this research in ways only a non-pessimist can envision.
Brent S.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Funny you choose to summarize that way, that I am ignoring more than half the picture. On the contrary, I think I'm the only one posting here giving balance to the whole picture.
Wonderful book by Henry Hazzlitt, "Economics in One Lesson" where he opens in Chapter 1 this way:
"Economics is haunted by more falacies than any other study known to man. This is no accident. The inherient difficulties of the subject would be great enough in any case, but they are multiplied a thousandfold, by a factor that is insignificant, say, in physics, mathematics or medicine - the special pleadings of special interests. While every group has certain economic interests indentical with those of all groups, every group also has, as we shall see, interests antagonistic to those of all other groups. The group that would benefit by such policies, having such a direct interest in them,will argue for them plausibly and persistently. It will hire the best buyable minds to devote their whole time to presenting its case. And it will finally either convince the general public that its case is sound, or so befuddle it that clear thinking on the subject becomes next to impossible."
Henry goes on in Chapter 2 to apply his lesson in Chapter 1, and talks about a baker who has a window broken out by a vandal with a brick. Good or bad? The baker replaces the window, so some glasier makes $250, and we see that money going round and round the community making business for everyone.
The problem is, that is ignoring more than half the picture. The transaction with the window is visible to all. What is invisible, is that the baker was going to spend that $250 on a suit. He didn't. So a tailor went without business that day. And the baker, rather than having a window and a suit, has only a window, so he is poorer too.
Stanford and CMU spent $20M collectively on their two entries. That's visible. That dozens or even hundreds of other research projects, for who knows what, at those universities, got shelved, is the invisible part. Maybe one of those students would have made a nano bot that could clear arteries. Many university programs are working on such technologies right now. Maybe we'd have a cure for strokes and heart attacks by that. But we'll never know. The half that is visible, the half that is the "great success", is we now have demonstrated a few vehicles which can raise a lot of dust in the desert, follow a very carefully plowed out course, and many carefully provided waypoints, but have to be followed by chase vehicles lest they become a threat to public safety, and have no other immediate use.
Maybe there will come an industry from this. Wonderful. That too will be visible. I will reinforce the idea this was a great success. But we will never know that part that is invisible, that was lost, because it never happened. Because $20M from two Univ. alone, and hundreds of other groups in like manner was drained out of the robotics industry to chase a $2M prize. The issue is not if something good might come from it. The issue is what was sacrificed for it, and it is a truth and an expense we will never know.
--
Randy M. Dumse

Caution: Objects in mirror are more confused than they appear.
  Click to see the full signature.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Polytechforum.com is a website by engineers for engineers. It is not affiliated with any of manufacturers or vendors discussed here. All logos and trade names are the property of their respective owners.