here's a killer app idea for a robot.. I'm very suprised it hasn't been done before..
webcam pointed at the TV screen, software tracks movement of sprites on screen.
Video game system connected to TV.. Start with the Atari 2600 for simplicity of machine vision code..
Computer connected to webcam, and also has access to the current state of the joystick (5 1-bit inputs. Up, Down, Left, Right, Button)
Software watches screen, relates movement on screen to joystick state..
Software learns goal of game. Restart in 2-player mode, and play the computer. If you program it correctly, it should be impossible to beat the computer.
You could also do it without joystick feedback. Let the computer randomly cycle the joystick bits, and relate movement on the screen to the joystick state. It would learn how joystick relates to the screen, but not the concept of the game.
The point: machine learning, and a friend to play games with.
Anyone interested in trying to get this work with me? I have a webcam, A PC, a TV, and an atari 2600.
You could allow the user to lasso parts of the screen, IE score and remaining lives. Perhaps a small programming language within the software:
If lasso1 = previouslasso1 - 1 then BAD --for lives GoLearn endif
if lasso1 = previouslasso1 + 1 then GOOD GoLearn endif
If lasso2 = previouslasso2 - 1 then BAD --for score GoLearn endif
if lasso2 = previouslasso2 + 1 then GOOD GoLearn endif
Learn: what just happened on screen?
Color change of objects? which objects? store colors, shapes of objects, Good/BAD and add them to a rule table collision between objects? which objects? store colors, shapes of objects, Good/BAD and add them to a rule table timer reached a certain count? which count? store count and Good/BAD rule in rule table life indicator reached a certain count? which count? store count and Good/BAD rule in rule table
then in the main loop:
check for Closeness to Good rule move joystick to achieve good check for closeness to Bad Rule move joystick to get away from bad
PacMan seems like a good place to start