Telepresence - Big deal?

There is some info on google.

http://www.google.com/custom?&q=robot+telepresence

Also, see iRobot ...

http://www.google.com/custom?&q=irobot+telepresence

You will notice that the packbots that iRobot sells to the military are
camera equipped for remote surveillance.

More locally, I've equipped my RAD-hack platform, on which I'm testing
various sensors and subsumption routines, with both zigbee for remote
I/O, and also a wireless cam so I can watch what the robot is doing
while sitting at my PC. Unfortunately, I've been a little caught up in
other matters recently and progress has slowed, but I think this makes
a nice test platform, where you can basically command the robot to do
something, like drive into a wall, and then watch how the sensors and
control routines deal with the problem. So maybe call this
"soft-telepresence".

http://www.oricomtech.com/projects/jpillbox.htm

Wow, I didn't know Cisco saw this as such a hot market. Very
interesting.


Yes, and some like Talon and Sword are packing real firepower. Better
under human control than autonomous.


Yes, a nice new phrase. Oversight remotely.

There's another thread here by a "roboDNA" about remote driving. Seems
he's doing something like Microsoft Robot Studio, but with a more
direct route. It will be interesting to see if he replies and gives
more details on what he's got. However, this looks to me like
hard-telepresence.

I have a little RF Video Camera. It's about 1/2 inch on side. I was
going to insert it into the front of my Mini Sumo, and show the video
on an external monitor. Again, that fell to the "not enough hours in
the day" problem.

My current use of telepresence is in the class I taught at UNI over the
internet last spring. This spring I'm assisting in a new Computer
Interfacing and Signal Processing class. I hope to also attend a DSP
course in another department.

Last year I used PVX software. Polycom is one of the biggies in the
Teleconferencing world. The software was good, but very problematic.
Prone to lock ups and needing rebooting. It was also a resource hog. I
ran on a 2.8GHz laptop, and barely had anything left over to open
slides or see my notes.

This year, we're experimenting. Windows Live Messenger has come along
very far, and is almost as good as PVX was last year.

But I am among the first to say, this is not a mature technology. Sound
levels vary. Camera position control just isn't there. It's not like
being a teacher in the room. When it is, then I can see this being a
prefered way to teach. It has the advantage of being digital and can be
captured.

I see a future with educational software being "rentable" and you being
able to choose the best instructor for your needs, no matter where
he(she) or you are physically located.

But in general I think I agree with Brooks on telepresence. It will be
a big deal. A very big deal.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

This is what I have too. All mine took to hook up was a 9v battery on
the xmtr and hooking the receiver to an AV monitor. I actually had it
on my hexapod Nico-6 that I sent to Larry Geib for Robothon 2006, but
they apparently didn't have an AV monitor in the hall. Otherwise, you
could see ahead from Nico's perspective, as he was walking around.

Also, George Matsuoka had an article in the first issue of Robot mag, a
year ago winter 2005-6, about putting a web cam on his robot tank. In
his case, he used a rabbit wifi server, plus wiport and USB cam on the
tank, and controlled it via HTML script from his PC-wifi router. His
solution is a lot more integrated than zigbee + separate wireless-cam,
but also cost him about $500 for the electronics, plus plenty of setup
and programming time. True telepresence will require the 2-way
integrated solution, like this and what you discuss below.

I'm expecting to take a break from normal activities for a couple of
years, if things work out, and when I get back into robotics, I expect
all you guys will be doing T-P :). I imagine M$ will have this included
in their Robotics Studio.  I started playing with RF on my robots last
year, and it's great stuff.

How do you get the video signal reliably back to the receiver at all times?
Don't you lose it behind certain barriers?  

I made a remote controlled lawnmower I would drive from in front of the TV,
but it was never safe because I could lose the video when it went behind a
big tree.  I tried several transceiver pairs in the 900 MHz and 1.2 GHz
bands, and found none that would work reliably.  I had hoped some
inexpensive product would emerge that broadcast in numerous bands, with a
receiver that searches the bands for the best signal, but I've seen nothing
like that so far.  Perhaps someone here has a pointer?

Mike Ross

I think has less to do with the transmitting band than with the power
output of the transmitter and the quality of the receiver. There are
some high-end microwave video transceivers that have 1+ mile (line of
sight) range. I'd be very, very surprised if a little old tree bothered
the signal all that much. However, this kind of rig will cost you -- a
couple hundred bucks, easy. Try the covert wireless online outfits, and
stay away from the low-end consumer stuff. There is a healthy demand for
these types of tranceivers, so there is no real incentive to market
something like this on the cheap.

-- Gordon

That has an ominous sound to it... Inquiring minds want to know. If you
aren't around, who am I supposed to have these long winded pendantic
discussions I enjoy so much with? Not to detract from any of my other
faithful correspondents here, but you would be missed.

Today was a heck of a frustrating day for telepresence for me.

First, I arranged for a live web broadcast of a physics colloquim at
4PM. I was able to log on and see the room set up, then, nothing. Email
came 2 hours late. Cell phone calls went to voice mail. Nothing. Even
IM failed everyone about start up time. A couple hours later, I got an
email where I could watch the recording on the web, but the live part
of it was long over.

Then this evening, I was supposed to sit in on a DSP class being taught
for the first time at the university. Again, starting time I had a
phone call. Then nothing. The laptop couldn't connect to the network.
So I again missed out.

Telepresence may be a great idea, but it is in the beginning stages of
functionality.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

Regards your story as below, hopefully, you guys - or Bill.Gates - will
have this all worked out by time 2 years come and go. Just a few
technical problems, like any new endeavor :). From what I've heard
about B.G's talk in Las vegas [wherever] last week, he wants to have
his Vista OS in control of ALL your electronics needs 24-HOURS every
day, and this means you guys hiding out on the range in east Texas too.
The new OS'es are geared more towards multimedia - the big big thing
now - so I should imagine wireless is in there too.

Another story. A couple of months ago I bought a new WinXP notebook.
Did some shopping locally for a couple of weeks. Walked into Office
Depot and they had 20 notebooks in there, all with "glossy" screens,
all running DVD movies. Not one OS window or actual "work" application
in sight. With the glossy screens, mainly what I saw where bright
reflections of the overhead lights, and I could also read the signs on
the back wall 50' away reflected in the screen. Not good.

Not too difficult to see what Office Depot thinks the market for
notebooks is. Glorified $1000 DVD players.

Another story. I ended up buying the lowest end Compaq notebook I could
find, with matte screen. I actually wanted it to do work on. I bought
it online from the hp.com site, and would you believe it, it shipped by
fedex from Kunshan in "red" china, near Shanghai. Turns out 100s of
american companies have plants over there.

http://www.google.com/custom?&q=kunshan+china+outsourcing

That's where Carly Fiorina sent the jobs. Since everything in the
notebook was already made over there, and labor is cheap, imagine how
much greater the profit is than say for Dell, when the bits and pieces
are separately shipped to texas for assembly by amercian wage-earners.
Plus, I, and not hp, paid the $50 for shipping the assembled notebook
to the US. So, HP wins on all accounts.

Also, regards my travel plans, nothing too ominous, just a minor change
of venue. I've heard rumors they have internet all over the world now.

I have two Dells.  The back of one says "Made in China".  The back
of the other says "Made in Mexico".  Dell also has large assembly
plants in Malaysia and Ireland.  So I think your belief that Dell makes
everything in Texas is mistaken, or at least outdated.


Last year I was working in Shanghai, and I tried to buy a Dell laptop
and have it shipped directly from the plant in Shenzhen.  They told
me they couldn't do that, it had to ship to Texas, and then back to
China.  I didn't buy it.

Offhand, were the 2 Dells you bought preassembled deals, or specialized
orders? My compaq was specialized per my requirements, and it also took
2 weeks for them to put it together, and I did have to pay the freight
from China. It's possible of course that Dell is assembling many of
their boxes in MX or CN in order to keep up with the competition.

BTW, I did try shopping Dell online several times, but every time I
came back to the website the options were all changed around and
eventually I just gave up. BTW, all I wanted was  a basic box, without
all the wizzbangs, and the compaq was $300 or so less than the cheapest
similar Dell.

I bought 5 of the bottom end Dells for the class, since they didn't
have enough in the department to go around. After class I got them
back, slightly used, and sold them to family and friends.
Unfortunately, one of them failed two months after I sold it to my step
daughter. That was disappointing because they were out of warrantee. So
I got her another Dell as a replacement, with a longer warrantee this
time.

For simple things like web brousing, editing source, downloading as a
terminal, these low end units seemed to do just fine.

My personal computer is a Sony VAIO. I try to buy at the front edge of
technology, and keep it for 3 yrs before replacement. They usually last
about that long. Again extended warrantee will run out a few weeks
before the display goes out, by my experience.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

Yes, must be the same unit. I saw them today on Cyberguys site.

http://www.cyberguys.com/templates/searchdetail.asp?productIDY31&ta=prod_info

They say 7/8" on a side.

When we first got it, we hooked it to Cocoa, the office Pomeranian.
Watching things from a small dogs point of view can be a very
disorienting sight. Further, she does all sorts of body motion dances,
and it doesn't take long to get a sea sick feeling from watching.

In fact that's something else I remember from the class. When they took
the remote camera and tried to position it for me to see something they
were doing with their hands, or at their computer screens, I really did
get a bit nosiated at times from the visual cues of the motion. Very
unsettling and unexpected. Particularly if the guy holding the camera
forgot what he was doing, and started swinging it around as he walked
off. Whoa!

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

Huh, looking through the links, it seems clear to me, while
telepresence is a well accepted idea, there aren't that many
applications currently (as opposed to the teleconference market which
is burgeoning already). I take it to mean this field is wide open.

Did quite a bit of reading ahead in my LabVIEW text last night. Seems
they are big on web publishing. So I think I'm going to put an
interface into my MiniSumo to let it be driven remotely through the
Bluetooth or XBee serial channel. Really though, my main interest is
not control but data collection.

I want to watch the MiniSumo linefollow, or sumo fight, and watch the
variables, the motor outputs, the analog readings from the range and
line sensors, and maybe even the state changes as it goes. A higher
challenge would be to put that camera in there and merge that stream
into LabVIEW, and have a complete remote panel to see the world as the
sumo sees it.

I am very intrigued by telepresence. I may do a startup or spinoff. My
first interest is in the classroom presence, for instructors or
students. But I think the future in the field is open ended.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

In areas where humans cannot go, specifically deep ocean work,
telepresence is becoming increasingly important. The company that I
work for makes remote operated vehicles (ROV -
http://www.ssaalliance.com/quest_uhd.html ) and telerobotic manipulators
(http://www.ssaalliance.com/conan.html ) and has been adding increasing
intelligence to the vehicles. Our current ROV offering uses inertial
sensors and doppler velocity loggers to either hold position and
altitude above the seafloor (called Stationkeeping) or perform offset
movements from a fixed position, with a resolution of several
centimeters. It can perform these functions even when disturbed silt
has made visibility drop to zero. The control system allows the
operator to independently control forward/reverse and yaw, while
keeping pitch and roll to a minimum, even when using the manipulator to
pick up something weighing several hundred pounds.

These types of ROV systems are used for just about every task that
takes place below the depth that divers can reach, most of which is
related to the oil and gas industry, but we have done some work for
research universities also. The types of jobs that I hear about most
are drill rig support, pipelay touchdown monitoring, and pipe
inspection.

The manipulators are hydraulic and extremely powerful and surprisingly
responsive. The more expensive versions are controlled by moving a
miniature version of the arm, called the master arm. The larger arm,
the slave arm, matches the position of the master arm allowing simple
and intuitive control.

-Robotguy
http://robotguy.net/blog

RMDumse wrote:

That's awesome.  This is exactly the sort of thing we're going to need
to develop the Moon, both before astronauts get there, and even once a
base is established (since going outside in a spacesuit is a lengthy
procedure and exposes the astronaut to radiation, so they'll want to
minimize that as much as possible).  Of course when the operator is on
Earth you have a several-second lag, but with the kind of smarts you
guys are putting into your ROVs, and a bit of practice on the part of
the operator, I don't think that'll be a big deal.

The first task I think is well-suited to a lunar ROV is bulldozing
regolith to bury the habitat (either digging a trench first, or just
heaping the regolith up on top of a habitat sitting on the surface).  A
layer a meter thick or so keeps out the radiation and also provides
thermal insulation, keeping the inside nice and toasty.  This should be
pretty easy, don't you think?

But if you could also make teleoperated robots that could do simple
maintenance tasks -- brushing dust off solar panels, plugging and
unplugging connectors, moving equipment from here to there -- then this
would be a HUGE benefit.  So I'm very excited to hear that these things
have moved out of the research lab and into serious commercial use,
since that both indicates that they're real-world-ready, and also tends
to drive further progress much faster.

Best,
- Joe

Very interesting. You have quite the manipulators there. So far, all I
have been able to effect remote distance is turning a camera on the
Polycom unit so I can look around the room, but even that is very
tedious.

What I would really like as an instructor (or even as a remote student)
is the ability to "move my head around" or translated, be able to move
the camera to different perspectives, like looking over a students
shoulder, or moving to the other isle.

I'm afraid where I'd like to go will allow me to swing a camera around
like a club and clunk students in their heads. Sensing the surroundings
to avoid dangerous movements is something we rather take for granted,
but are very difficult remotely.

Is there force feed back on your manipulators? or is it purely visual
feedback? How difficult is it for an operator to learn to use that arm?

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

No, no force feedback. I have heard of other companies that have it,
but not ours. It is very intuitive to control if you can see it (if it
is right in front of you), but gets tougher when you only have the view
from a pan & tilt camera or a wrist camera
(http://www.ssaalliance.com/cameras.html  - bottom of the page). I just
had to write firmware for a lighting controller that, besides being
able to handle the pressure at almost 20000 feet of seawater, can also
handle repeated short circuits, because the operators keep whacking the
lights with the manips while it's underwater. SIZZLE!

-Robotguy
http://robotguy.net/blog

A company I work for is planning on putting some small Manufacturing
plants in South America to expand our operations. We would plan on
bringing the new employees to our U.S. location for training.  Of
course, we can't train them 100% on all that might go wrong, or what to
do in unusual situations.  Therefore we will need to assist them on
these things from our U.S. office.

On other job's I've done 3-D design of large weldments. Sometimes I
would be called by the shop to answer a question, and was able to
answer it just talking on the phone.  But, it was usually easier to
physically go up and help resolve an issue.  In this case, the shop was
only a 50' walk.  Physcialy going to another country is certainly a
barrier to quick resolutions of these things.

I was thinking that a simple mobile base 4 or 5 foot high, with a
laptop and webcam on top, could be a big help.  It could be connected
to Internet via WiFi, and remotely operated in our office in the U.S.
While the user in the U.S. could remotely drive it into the shop, I
suspect that it would be more practical to just let the S.A. user wheel
it to the area in the shop where they have the question.  In the
simplest implementation, the base isn't even motorized, but the camera
is an off-the-shelf tilt and pan.  The only non-standard thing I would
add is a motorized laser pointer... and perhaps that could simply be
glued to the camera and  just aimed at a downward angle, so that when
you aim the camera at a person, you aren't shooting the laser at their
face.

In usage, the S.A. guy wheels it to an area in the shop where they are
having a problem with a machine, and then initiates a video call to the
U.S. office.  The guy in S.A. can describe the problem while pointing
with his hand to the problem area saying "There are sometime bubbles in
the material when it comes out here."  The guy in the U.S. can use the
laser pointer to point out where he thinks there needs to be
adjustment.  The cameras could be aimed down at a paper so a user can
sketch solutions as well.

The above is certainly cheap to do, since it only involves existing
stuff that an office or shop may have.  When I've attempted to use
other existing web-cam robots, the time lag and slow framerate made it
useless in my view.

I may report back here when I get a chance to try a simple unit out on
my own.  But, at the moment, things like firewalls and other stuff to
protect our network have become barriers outside of my direct control.

Joe Dunfee

Having spent what seems like a life time of answer questions over the
phone, going back to Rockwell as an Apps engineer in 1980, then for my
own customers since. I know there are many ways to improve remote
assistance. However, from my experience with tough cases and cross
culture situations, like starting the company in the Philippines, I
know there are moments where just a hand full of face to face grunts
and hand waving could have saved hours of otherwise misdirected effort.


It's funny you mention that. I've worked quite a bit with the director
of research in their services department at the university. When i was
talking to him of the idea of a robot base he got very excited. One of
the things we'd discussed was the idea of a problem with the "Germany"
plant (just happened to pick a country for sake of discussion), how can
I go out on the line and look to see what the problem is? Well if I can
log onto one of the remotes there, I can drive right out to the
production line and look around for myself. I can go to the worker on
the line and ask him questions. I can look at gauges and see if I think
parameters are okay or not.


We also talked about pointers, but we were thinking arms, where you
could actually point like you would if you were face to face. The laser
pointer is even a better idea. I'd probably make it independent from
the head though. Another neat feature was if the remote worker had a
laser of his own, perhaps modulated, where he could point, and the
optics locally would cause the head camera to move there. So he could
highlight features he wanted you to look at, and then ask questions as
well.


I've done that before, where I wrote on a piece of paper and held it up
to the camera where my face was. Problem was I only had a pencil. When
I blackend in the lines making them about 1/4" thick it worked fine. So
think "Marksalot" tip when doing that.

Some systems have what they call white boards, where one or both ends
can draw on a screen and the other can see it. I've had no success to
date using a white board, but tried several times.


That's the commercial market that Cisco and Polycom are seeing and
going after.


Have you tried IM lately? Now called Windows Live Messanger. It does
much better today than just a year ago. Also Skype, a free VoIP
solution. Call anywhere in the world. They brought out a video version.
It was too preliminary then, but it's been about a year since I tried
it again, it may have improved.

I first tried to teach the class from my ranch. The latency got up to 4
- 5 seconds. That was simply annoying. Very choppy and disconcerting.
Unnatural. However, from the office (T1) it worked well. About a 1/4 to
1/2 second. Very tolerable.


Yes, I had to open ports on my firewalls as well. Surprizingly, I
didn't for Messenger. It was much better about translating past them.

Did you know you can buy PVX software that runs on your PC (>1.4GHz
ones) for about $120? (Tigerdirect.com for instance). A copy on each
end would get you close to what you are describing without the
mobility, of course. It would be laptop to laptop then with no pan and
tilt.

In their more expensive purpose built units, Polycom has pan and tilt
cameras built in. My experience with PVX to Polycom 3000 in the class
was that the camera would only move about one in four times of pushing
the controls button, and rather later than I'd hope. I am told two
Polycom units do very well with remote pan and tilt, and they have
remotes to tell the camera where to go on the far end. Let's your
distant operator point the camera where he wants to, to show something.

Contact me if you care to Joe. We can try a video IM message session
and you can see how useful it is. I also just loaded the updated PVX
software, but haven't used it yet. We'll probably be using it later
today (this evening) in my next classroom appearance.

--
Randy M. Dumse
www.newmicros.com
Caution: Objects in mirror are more confused than they appear

Randy and I had a video conference yesterday, using Microsoft's Live
Messenger.  However, I was quite disapointed with the results.  The
main issue to me was lag time, which I roughly estimate at around 1
second. I would also have liked the frame rate to be improved.

I should note that a significant portion of the lag time was not
internet related, because a lot was there when I viewed my own image
from my camera.  I should point out that I have a relatively old USB
type 1, which has a slower speed than the current version 2.  I was
able to look at a simlar set-up at a store, using a new camera, but
even then the lag was an issue.

The Ping times for internet inside the U.S. are typically around 0.1 to
0.2 seconds.  I don't think that is a major problem... but leaves
little time for other processes like video compression.

In my observation, the ability to see each other in near real-time is
critical. The flow of a technical discussion involves seeing how the
other person responds to what you say, so you can see if they are
understanding and agreeing.  People tend to interrupt each other, but
mostly do so when they hear a good place to jump in. The lag I
experienced in our test was bad enough that we tended to interrupt each
other at inappropriate places.

I would have eagerly traded resolution and color in the video for less
of a lag.  But, neither Randy or I could find a way to adjust this.
The only solution seems to be the high end type of video conferencing
(costing well over $1,000) which has MPG compression built into the
video hardware.

For telepresence using any kind of physical object, the problem has
only got to be compounded. Sight is fundamental to being able to do
this well.

Does anyone know of a good solution to the lag problem?

Joe Dunfee