Low cost vision system, thoughts

"mlw"

Not exactly cheap, but cheapest firewire camera I could find:
http://www.1394imaging.com/en/products/cameras/firewire_color/

I bought the DFK 21AF04 and shutter speed goes as fast as 1/10,000s. I also
bought a prosilica (1/3" sensor instead of 1/4" sensor), but I've still not
compared the image quality on both of them.


I'm not sure, but take a look at what I call the coffe makers (sick ladars),
it was my understanding that they use some similar technique. And by the
way, everybody at the DARPA GC was using lots of them. Actually, from my
perception, there were far more SICKs than traditional vision on the GC.

Cheers

Padu

This is also a problem for biological eyes.
When we move our head and body the eyes balls
move to compensate. When the eye balls move
the data is ignored.

If you watch a bird of prey hovering, its head
is still relative to the earth while its body
moves about. Hold a chook and move it sideways
and the head stays fixed.

Wave your hand back and forth in front of your
face and it is "blurred" but we seem to cope with
that ok.

You might find it practical to simply stop the
robot, take an image, process it, and make the
next move. I think you will also find that the
bluring problem is more in relation to the
camera moving sideways (such as when the robot
rotates) not when it is looking in the direction
the robot is moving.

Get some real data. Take some images while your
robot is moving about. See how bad the blur is
when it is moving in a straight line at the
speed your robot moves.



I presume here you are thinking about vision for
navigation, (positional recognition?) and obstacle
avoidance?

I have not used IR laser light, I am not even
sure a web cam can "see" IR light. I think some
have IR filters so as they are not swamped out
by IR light. Others are designed to just see
IR light from hot objects such as cars and people.

You might even consider white light using a
lens to focus it. Although it expands with
distance the resolution of your camera also
decreases to match. White light also has the
advantage of returning the color of the object.

But this is not armchair thinking stuff it
requires getting real data and looking at the
images and thinking about how you can extract
the information you need.

In reference to the Padu post if LMS units from
SICK that helped a vehicle map the terrain. Are
they cheap? It says the "helped" not that they
were sufficient by themselves and they used
five of them.



I think you need to get some real data and think
about how you are going to process it so you are
not trying to solve an imaginary problem or failing
to solve a real problem you didn't know existed.

--
JC

To a point, if you watch someone's eyes when they move, they stay focused on
one object as they move, and jump and refocus. Try it yourself, it takes
effort to not to do this.


Obviously objects that are moving fast will be blurred, but things moving at
a reasonable speed need not be. The current field of inexpensive cameras do
not behave "fast enough." If I may use a photographic analogy, say the eyes
have an ISO speed of, say, 50, a lot of web cams have a speed of, lets say,
1 or 10. (And of course, the effective shutter speed follows)

I have no benchmarks to test this, but this is what I see when I process
images. There is no reason a slow moving object should be blurred.

As a start, yes.


You should get a web cam and point your IR remote control at it. It comes
through clear as day as "white" light.


Actually, the advantage of an IR LED is that color is not returned.


True, but I have done an amount of looking at images taken from my stupid
logitech quick cam, enough to theorize about what I want to try next.

I have done a bit of thinking (and investigating with my quick cam) about
it, and the first problems to be solved is navigation and object avoidance.
The IR laser should help this, it would be sweet if we could get an IR
laser grid projector, but I digress.

 ...

jgc wrote:

mlw wrote:

 ...


Well blow me down you are right I had never actually
tried that before :)


Why? Color is invariant to position and rotation
and many things can be identified by color alone.

Or do you mean the advantage is they don't absorb
color the way a red laser does?

Is IR evenly reflected despite the color?

 ...


If you can get a cheap IR laser maybe the lens used
in the red laser pens for making a line would work?
That would be a start. Have you thought about what
you would do with the line data and how you would use
it to recognize a position?

--
JC

What's really fun is using a bunch of high output IR LEDS as illumination
for a "night vision" webcam.


Well, actually, very little color info is returned.

Using color to identify things is tricky, if the lighting varies, i.e.
illumination with an incandescent light, fluorescent light, or even candle
light, or even if the position of the sun and the weather all affect color.
(I used to do a lot photography and contracted at Polaroid for their
electronic camera) The issue of color mapping is huge in imaging, you can't
rely on color at all unless you have control over illumination and color
response of the imaging device.

To use color reliably, you would have to do an amount of imagine processing
on the image just to normalize the RGB data into some known color space.



I haven't done any spectral analysis, but it looks almost monochrome in that
the light colors are reflected lightly (more), and dark colors are
reflected darkly (less).

I don't see why they wouldn't. "IR" is just light that our eyes can't see.


Say, we have a 320x240 camera, we would have 320 Y references, one for every
X or horizontal pixels. Take this data and normalize it to zero, or treat
it like a series of differentials, something along those lines.

The rest is all vague and conjecture:

It should be possible to hash this data into a really reduced quick search
format that can be close with a probability of error.

Ideally, if the hash algorithm is good, the same view from acceptably
different distances and/or perspectives should be the same. (That's the
hard part)

why would this be difficult?

in greyscale image processing, you use a threshold to reduce to
monochrome

16 grey levels to monochrome:

threshold = 8
for y =1 to height
for x=1 to width
if pixel (x,y) > threshold then pixel(x,y) =1
if pixel (x,y) < threshold then pixel(x,y) =0
next x
next y

if you have 256 colors, and want to "normalize" colors,
why not:

color red = 0 to 16
color green = 17 to 32
color blue = 32 to 48
color yellow = 49 to 64
color orange = 64 to 80

etc...  depends on how the color's of your image format work...

for y =1 to height
for x=1 to width
if pixel (x,y) < green then pixel(x,y) = red
if pixel (x,y) < blue then pixel(x,y) = green
if pixel (x,y) < yellow then pixel(x,y) = blue
if pixel (x,y) < orange then pixel(x,y) = yellow
next x
next y

so different levels of red = RED
different levels of green = GREEN

Jen

Well, it *isn't* difficult if you know from what you are converting. As
mentioned, the lighting varies color dramatically, you would have to have
some way to calibrate instantaneous color.

The hard part is figuring out the calibration.


Yea, that fine but really slow. REALLY REALLY slow. Your best bet is to use
lookup tables:

for(int y=0; y < MAX_Y; y++){
  for(int x=0; x < MAX_X; x++){
    red_ref[y][x] = translate_red[red[x][x]];
    green_ref[y][x] = translate_green[green[y][x]];
    blue_ref[y][x] = translate_blue[blue[y][x]];
  }
}

This way, you avoid the conditional expression for each color in each pixel.
If you have 640x480 pixels, and a conditional (conditional expressions
usually take longer because predictive caching of processors evaluate them
differently than non-branching instructions) need to takes about 0.0000001
seconds, think about it. (640*480*3*0.0000001) is about 1/10 of a second.
Non branching instructions go faster because they can be pipelined. Then
again, I'm talking C/C++ not java or perl.

The brute force of translating the pixels is a challenge, but the *real*
challenge is figuring out what those translation values are.

What happens when a light red reflects fluorescent light vs when it reflects
incandescent? Fluorescent light is typically very "white" while
incandescent is typically fairly yellow (YMMV). The weather makes a
difference to, cloudy weather vs sunny weather change color as well.

In photography you usually use blueish flash, and the flash overrides the
ambient light and provides a more consistent lighting.

Color is a huge problem, especially if you want to do something with it.
Comparative color may be something to look at, but right now I will settle
for monochrome.

mlw wrote:


 ...

I am aware of the problems you mention above.

The human vision system of course uses neighbouring
colors to decide what color something is thus yellow
(in RGB values) can look brown, and shade also is
a relative value not the absolute pixel value.

Yet color can be used,  see D Herring post below,
"Use CMVision to quickly find colored objects"
http://www.cs.cmu.edu/~jbruce/cmvision/


In an experiment I did I used the R:G:B ratio
to locate (blob and thus outline) balls of
different colors. The light source was constant
and I guess it might not have worked in sunlight
or another type of light source without a different
set of ratios.

--
JC

That is sort of the problem, variable light sources create different color
ratios, if you have more yellow light, your objects look yellow. If you had
some sort of reference in the camera view at all times, then you could do
it.

we don't need no stinkin structured light.  If a human
sees a GREEN apple under intense RED light, it
is a RED apple!  we can pick the apple up and examine
it under the red light...  it is still red.

when we turn off the red light, and switch to white light
instead, we have a green apple!

You could have a "reference" from the objects in the field
of view perhaps..
IF pixels = VERY BRIGHT then object= light source

If lightsource present, other colors in view = other colors + light
source color

otherwise the objects are the color they appear to be, under
the non-white light.

Rich

 ...


A white reference card to view?

--
JC

Maybe, but the human eye uses both BW and color, using IR cross hairs for
one type of vision and color for another.

A grey card, white card, or better yet, several shades of a known neutral
color.
You can get a macbeth color checker for a better idea of what you are
rendering, if you really want to get serious about this stuff.
http://www.gretagmacbeth.com/index/products/products_colorcheckers/products_cc-overview/products_colorchecker.htm
They cost around $100(USD) for a 36 patch chart. They also make a 140 patch
unit for ~$300(USD) but that is excessive for my uses.
The macbeth chart is calibrated, and the actual chromacity values are
provided for your reference.

Auto white balance is still a best guess, and doesn't cut it all the time.

A diffuse illumination sensor can help you make better guesses of ambient
illumination.

THere is a lot of information out there regarding color science,
illumanants, color temperature etc.

Mike

The concept you're talking about is called structured light, and it's
been around for, oh, decades. There's plenty of data out there,
searchable via Google, and a number of patents, which are available at
the uspto.gov site. Variations include scanning a laser beam with a
mirror or polygonal mirror, splitting it with a diffraction grating (the
basis of CD focusing, research on Foucault), using an animorphic lens,
and several others. A number of factory automation systems use this
approach for stereoscropy, and it's a method that has applications in
medical imaging. There are commercially-available DirectShow filters
that rely on coded light structures of one shape or another. Somewhere
there's probably a page on how someone did their own filter.

Do a search on Google for structured light vision and you'll find
thousands of hits. Most texts on machine vision will discuss the many
approaches as well.

-- Gordon

...


I actually don't like the structured light idea
for aesthetic reasons :-)

As for Direct Show I would most likely need to
get up to speed with VC++ .NET and get some good
tutorials on using Direct Show.

I just googled Direct Show tutorials went to the
msdn site and it was unclear what to do with the
code snippets and vague descriptions.

However it would make any software Windoze dependent.

I have been looking at learning Java which seems
to have a lot of media support but is it fast enough
for my own processing code?


--
JC

There is little to choose from. There is one (older) book on DirectShow,
but that's about it. The market is VERY small for this, so it's not the
kind of thing book publishers will fall over themselves for. There are
some FAQs and mini guides at some of the Microsoft MVP sites -- The
March Hare has one -- you can find it by going through the DS-related
Microsoft groups.

If you're a book learner, DirectShow can be very frustrating. It's
mostly an architecture for people with infinite patience and a knack for
experimenting. I have one of those qualities...

-- Gordon

Gordon McComb wrote:


Not me I'm afraid. I like people to get to the point
and make it simple. A product should be easy to use
and not require years of training or infinite patience
be it hardware or software if they expect a wider
usage of the product.

An example in another area might be the sale of a new
kind of fruit or vegetable. Give people recipes and
instructions how they can use the product and suddenly
it becomes a desirable item.

All I wanted was access to the data of a web cam at
a usable rate and an easy to learn language that can
compile code fast enough to deal with data in real
time so I could concentrate on what really interests
me. I figured a VC++ shell using DirectX or whatever
would do the job. That is not going to happen so I
will just have to accept it and go do something else.

--
JC

JC said :

Most webcams have a filter for infared light, a small amount of hacking
can remove it

My old monochrome ccd based Connectix camera
used an IR filter so as not to be swamped by the
IR outdoors.

--
JC