1. Home
  2. General Robotics Forum
  3. learning from digital inputs + vision

learning from digital inputs + vision

here's a killer app idea for a robot.. I'm very suprised it hasn't been done before..

webcam pointed at the TV screen, software tracks movement of sprites on screen.

Video game system connected to TV.. Start with the Atari 2600 for simplicity of machine vision code..

Computer connected to webcam, and also has access to the current state of the joystick (5 1-bit inputs. Up, Down, Left, Right, Button)

Software watches screen, relates movement on screen to joystick state..

Software learns goal of game. Restart in 2-player mode, and play the computer. If you program it correctly, it should be impossible to beat the computer.

You could also do it without joystick feedback. Let the computer randomly cycle the joystick bits, and relate movement on the screen to the joystick state. It would learn how joystick relates to the screen, but not the concept of the game.

The point: machine learning, and a friend to play games with.

Anyone interested in trying to get this work with me? I have a webcam, A PC, a TV, and an atari 2600.

You could allow the user to lasso parts of the screen, IE score and remaining lives. Perhaps a small programming language within the software:

If lasso1 = previouslasso1 - 1 then BAD --for lives GoLearn endif

if lasso1 = previouslasso1 + 1 then GOOD GoLearn endif

If lasso2 = previouslasso2 - 1 then BAD --for score GoLearn endif

if lasso2 = previouslasso2 + 1 then GOOD GoLearn endif

Learn: what just happened on screen?

Color change of objects? which objects? store colors, shapes of objects, Good/BAD and add them to a rule table collision between objects? which objects? store colors, shapes of objects, Good/BAD and add them to a rule table timer reached a certain count? which count? store count and Good/BAD rule in rule table life indicator reached a certain count? which count? store count and Good/BAD rule in rule table

then in the main loop:

check for Closeness to Good rule move joystick to achieve good check for closeness to Bad Rule move joystick to get away from bad

PacMan seems like a good place to start

Rich

Reply to
aiiadict
Loading thread data ...

If you program it correctly then it isn't "learning" it simply embodies your solution to the problem.

What software tool do you use to capture and process the images?

[ snip pseudo code ]

How about the first games that could be played on your TV? It consisted of bat/s being a vertical bar (of variable length) and a ball that moved with variable speeds.

=====================================================

bat * * * ball bat * O * * * *

======================================================

Each player could move their bats up or down to prevent the ball going off the screen. The angle the ball left the bat would depend on what part of the bat it hit.

-- John

Reply to
JGCASEY

Pong is perfect.. only 3 objects to keep track of with machine vision...

BTW, I've seen them referred to as "paddles" instead of "bats"

Rich

Reply to
aiiadict

And what software tool do you use to capture and process the images?

Reply to
JGCASEY

Have you tried this? I expect the TV refresh/flicker will cause tearing of the image that you will need to eliminate before proceeding. However, tracking computer-generated images will be easier than generic image processing since they tend to be more consistent.

Daniel

Reply to
D Herring

perhaps have VBL trigger shutter open and close... (or exposure time)

kinda takes the fun out of it if you need access to video refresh signals.

I'll experiment with changing the exposure time with the cams I have hooked up.

Rich

430 No such article 222 6007 body perhaps have VBL trigger shutter open and close... (or exposure time)

kinda takes the fun out of it if you need access to video refresh signals.

I'll experiment with changing the exposure time with the cams I have hooked up.

Rich

Reply to
aiiadict

My assembly language compiler. I'm not running windows, I have capture software running and working (wrote a driver that talks directly to the imaging chip), and I have line detection, shape detection (assembly)...

My tool is a compiler and my mind.

Rich

Reply to
aiiadict

Your first post wrote of a computer connected to webcam, not an imaging chip. What is the imaging chip? I would be interested to know more. I have only used web cams. I can also program in assembler. Are you using a PC?

-- JC

Reply to
JGCASEY

You could use an emulator to run the game - modern computers can emulate an atari many times over.

Reply to
Brendan Gillatt

mitsubishi artificial retina

Rich

Reply to
aiiadict

Do you have a web site or some such where you show all your robotic stuff?

I googled and found,

formatting link
Did you used the Artificial Retina Evaluation Board?

I noticed the modification date was back in June 2001

Has anyone else used the retina?

Cheers,

John

Reply to
JGCASEY

It will be complete in the next few weeks.

No, I did not use the eval. board....

A/D converter, with the CPU clocking out the the data.

Rich

Reply to
aiiadict

PolyTech Forum website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.