Keyboard / Mouse Input Device Design??



Good, what about sending keystrokes and mouse actions to Linux?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

...
The C language Xlib "SendEvent" protocol request does that, on systems where X Window System is in use. For a Perl version see http://search.cpan.org/dist/X11-SendEvent/SendEvent.pm
Jan is correct that not all linux systems and applications use X. I suspect that more than 95% of end-user Unix and linux systems use X, but imagine that a large fraction of servers do not. Eg, the tens of thousands of machines in google's server farms don't need displays. If you plan to address machines such as those, you'd need another set of system calls. -jiw
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

So you can't simply get the pixels comprising the display screen on Linux?

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

Not sure if you are asking this question rhetorically, but yes you can, using XWD or, programmatically, using XGetImage on the root window.
The question is whether you intend to track all screen state, or take snap shots. If you intend to take only snapshots, then on both Windows and Unix, you're looking at 20 lines of code or less.
I noticed that your application allows the specification of font family, font face, etc., when scanning for text. This has implications on the scope of what you are doing. Naturally, you're not proposing to be able to recognize any piece of "graphical text", are you?
-Le Chaud Lapin-
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I can't tell what you are asking, if you are asking can my technology recognize text from screen shots, the answer is yes. Here are some more details: http://www.seescreen.com/Unique.html
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

Ok, so even though you're using a DFA in your algorithm, the overall model is still stochastic. I see in many places 100% recognition, which, naturally, makes anyone skeptical. To get 100% recognition of arbitrary text, you have to know a priori the Bezier sets of not only all font families currently known, but those that have yet to be made, which seems absurd.
I think you should be more clear about the effectiveness of your tools, as in how it works. Instead of saying, "it recognizes, say a bit more." Since you already have a patent, it does not hurt to be more complete in your description.
-Le Chaud Lapin-
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
wrote in message

No it is not stochastic at all, the whole process is completely deterministic.

You must provide it with the means for knowing the precise pixel pattern of every Glyph that it must recognize, this is typically done by specifying a FontInstance: (a) Font Typeface Name (b) Point Size or PixelHeight (c) Style including (Bold, Italic, Underline, and StrikeOut) (d) Foreground Color and BackGround Color
It can process many different FontInstances simultaneously. This part of the system is operational and fully tested. It can provide 100% accuracy on any FontInstance that is not inherently ambiguous. The default FontInstance for much of MS Windows, Tahoma, 8 point is processed with 100% accuracy. Simple Heuristics can be applied to get very close to 100% accuracy on most FontInstances.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
wrote in message

Actually it could be set up to process all font families currently known. The simplest way to do this would be to build the DFA for the lower case vowels of every FontInstance in the colors of black on white. Then the text would be required to be transformed to black on white. Now it could quickly determine the correct FontInstance on its own, and then load up the appropriate full DFA(s). This assumes machine generated text that is not dithered or anti-aliased. With dithering, the problem of transforming the text to black on white becomes more complex, yet still feasible.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

Ok, I see what you are doing now. I hate to rain on anyone's parade, especially one where the objective is ambitious, you should know that what you are doing, the ultimate result, could be done in a way that is probably superior in many respect than the image based method.
One example is simple. Let's say that a programmer wants to use your software to know whenever the string "You Have Mail" appears anywhere on the screen, knowing that there is a mail application that pops up a window with this message. He specifies the font family, point size, style, and background/foreground colors of the little window that contains this message. To get this information, he spends 10 minutes repeatedly sending mail messages to himself to force the window to popup, and when it does, he eyeballs the message to ascertain the parameters. Finally he goes to your software and enters arguments for these parameters. Then he tells your software to run, and specifies a rate-of-grab of frame buffers so that the window, which pops up for only three seconds, is not missed.
Compare that to not having to force anything to popup or eyeball anything, simple typing in "you have mail", checking case-insensitive box, and being done with it. Not rate-of-grab would be necessary because there would be no frame grabbing. The monitoring software would simply "know" the state of entire GUI system at any point in time.
Certainly you will agree that, if this is what your software does, the latter method has significant advantages?
-Le Chaud Lapin-
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

And significant disadvantages, for example a false positive match. For something as simple as that, my system might be able to process as many as 100 frames per second. In fact that may be the biggest problem with the approach you are proposing over my method. Another problem is that there are times when this "text" message is displayed using a bitmap, rather than text itself.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

The method I was suggesting could not generate a false positive for text that is not regarded as simply an image. The reason is that, the very objective of your software - to determine what text is being rendered, is actually acoomplish before the text even hits the screen. If there is any program anywhere on the computer that tries to display "MOSFET" using any DRAW-TEXT primitive in the system, my method would catch it. So in fact, I would get a 100% hit rate on text that is normally rendered by the system.
For text where the programmer first converted it to an image and told the GUI subsystem to render it, my method would fail with OCR. But then, the problem reverts to OCR anyway.
Now consider: we do not have an exhaustive list of fonts to be used, so your method would have to have that to approach a hit rate of 90% without help from the user. Of course, if the user tells you what the font face is, etc, and all of these things, then yes, your software would approach 100%.
However, as mentioned, my gut feel is that "in-line-interception-of-text" versus "snapshot-of-graphics" is superior. One has to imagine the headache vs. % effectiveness of using each model.
Which would you rather have? 100% hit rate on 95% (perhaps) of the situation by simply declaring what text needs to be sought or 98%+ hit rate on 98% of the situations with painstaking determination of color, font face, pitch, and foreground back ground color each time, not to mention the possibility that you will miss an "easy" true positive because you're taking snapshots?
-Le Chaud Lapin-
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Including all the cases where the string "MOSFET" was not placed on the right place of the right window to be the correct trigger even for the required action.

So my system can ALWAYS work, whereas yours only works some of the time. SeeScreen is inherently compatible with every system, platform and application.
The next best alternative is a hodge podge conglomeration of many different complex technologies limited to simulating user actions on far fewer applications and operating system platforms.

I already posted one of several ways that my system can determine with certainty the exact set of FontInstances.

Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Now that I have read your other posts I can more easily understand your question. My system can recognize any machine generated text. It must be provided with exactly what to look for, this is typically done by specifying one or more FontInstances: (a) Font Typeface Name (b) Point Size or PixelHeight (c) Style including (Bold, Italic, Underline, and StrikeOut) (d) Foreground Color and BackGround Color
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

With those parameters, it is indeed possible to find matches. How could you not? If your software runs on the same computer as the windows that it is monitoring, then certainly if you render a piece of text using the parameters that match what is displayed, you will have an exact match, even with effects of anti-aliasing, transformations, etc.
However, I should point out again. Given that the user of your software has to specify these parameters anyway, and given that text that was not generated by the underlying font system will not, in general, be recognized by your software, it remains that the most important elements of recognition are pieces of text that is generated by the GUI system.
But it is possible to intercept _all_ rendering of such text through well-defined API's. In other words, if I were interested in knowing if there were a window that had the word "JFET" in it, I have to options.
1. Use your system and enter the above information. 2. Use my hypothetical system, and just specify "JFET".
Do you see? By interposition into the GUI subsystem, it becomes far easier to describe what you are looking for. Font face, point size, styling, and color become irrelevant, if it doesn't matter.
There is something else that is important. With your system, it seems that you are taking snapshots. The problem with snapshots is that there is a chance you will miss something, unless you are planning to bump up the rate of frame-grabbing so fast that you miss nothing. With my hypothetical system, there would never need to be a need to take a snapshot. You'd always know the state of the system.
-Le Chaud Lapin-
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

My system is the only possible way that is inherently compatible with every system , platform, and application. There are many cases where the required information is unavailable from the system internals. My system handles all of those cases. Now that we have dual core machines, it is possible, using a DFA to process many screens very quickly. I expect that my system could even play and win fast paced video games.
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

While I don't actually 'like' the keyboard interface approach being asked for, these devices are readily available, and so it should be easy to test how it does all behave. Look at the Hagstrom KE72 http://www.hagstromelectronics.com /, or the PI engineering X-Keys control board (the USB versions, may well be the better long term solution) http://www.piengineering.com/ . The Vetra systems VIP module, might also be worth a look (allows direct RS232 input to the keyboard port) http://www.vetra.com/Elimina2.htm .
Best Wishes
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
On a sunny day (Tue, 31 Oct 2006 19:24:17 -0600) it happened "Peter Olcott"
I ma sure you can, but because of the large amoun tof stuff that potentially _can_ run, X11 (own drivers), text console (own drivers), vgalib, you'd first have to find what is running and how I think, before you can access any display buffer[s]. There may be more then one graphics card too :-)
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload
Peter Olcott wrote:

What do you mean by *force* ?
You can get USB keyboards and mice anyway !
A USB mouse saved the day a while back on a system I was working on when for some unknown reason the standard mouse 'froze'. We just substituted a USB mouse and it was fine again.
Graham
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Peter,
You might want to look at
http://www.hagstromelectronics.com /
Don Cleveland
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

I am already working 16 hours a day, what I need is someone that already knows this off the top of their head. Feasible / Infeasible ?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
<%-name%>
Add image file
Upload

Polytechforum.com is a website by engineers for engineers. It is not affiliated with any of manufacturers or vendors discussed here. All logos and trade names are the property of their respective owners.