How to extract string from where mouse has pointed to?

[Li kai Liu Angus]

Hi, I found some translation software can extract string from where the mouse points to and translate the word on the fly. I was wondering how can this be done and is there any Win32 API can archieve this function. A brief explanation will be appreciate:) Thank you very much Angus Liu

[Christian Graus]

This SO needs to make the FAQ, I know I've answered it 15 times. The two ways to do this are using WindowFromPoint and GetWindowTest ( advantages: can find text that is obscured, disadvantages, will not work with text in bitmaps, is complex in other ways ( what word am I over in a sentence, etc. )). Second, write some OCR software. Advantages - works with everything, disadvantages - you have to write some OCR software. Christian The tragedy of cyberspace - that so much can travel so far, and yet mean so little. "I'm thinking of getting married for companionship and so I have someone to cook and clean." - Martin Marvinski, 6/3/2002

[Li kai Liu Angus]

Hi Christian, Thank you for your quick response. I've not been awared that I'm asking a common question like this. But I'm a causal visitor of this site and didn't roll back to find out where your other 14 answers appear. Anyway, Thanks a lot for your 15th answer:) Angus Liu

[Christian Graus]

That's cool - it's not that I'd expect you to read the rest of the site, only that it would make a good inclusion to the FAQ ( which I would hope people read ), because it is asked so often. Christian The tragedy of cyberspace - that so much can travel so far, and yet mean so little. "I'm thinking of getting married for companionship and so I have someone to cook and clean." - Martin Marvinski, 6/3/2002

[moliate]

Just an addition to CG:s answer: GetWindowText will not work on passwords on some versions of Windows (NT+ ?) if used from outside process. Use dll-injection to get your code inside the process... GetWindowText will not work in a WebBrowserCtrl. Use elementFromPoint/get_innerText in this case... /moliate

[Christian Graus]

Cool - my answer is designed to show how big a task it is ( and therefore beyond the scope of some thing that could be answered here in one go ). You've made me realise it's even worse than I thought. I guess OCR is the only viable option. Christian The tragedy of cyberspace - that so much can travel so far, and yet mean so little. "I'm thinking of getting married for companionship and so I have someone to cook and clean." - Martin Marvinski, 6/3/2002

[Li kai Liu Angus]

Cheers, Moliate. You explained my worries about retrieving a string from WebBrowser Control. But doesn't getting innerText return all text that an element has? Say, I have a piece of text "Hello! World" inside a pair of tag. And, my mouse is hovering over the word "World". Any more specified function can return the word "World" instead of "Hello! World"? Or, even more fussy, tells me my mouse is over the character 'W'.? Well, Sorry about my nagging questions:P Angus Liu

[moliate]

ykliu wrote: But doesn't getting innerText return all text that an element has? Unfortionally, it does.... (similar to GetWindowText) I really don't know how to get a specific word or character. A possible (but probably not the best) solution would be to make a copy of the element and gradually update the text so it removes one word after another. At the same time reduce the mouse coordinates by the difference between the original elements lower right corner, and the copys lower right. When the mouse goes out of the bounding box, you have the word pointed at... (hopefully) /moliate

[Michael Dunn]

Christian Graus wrote: it would make a good inclusion to the FAQ ( which I would hope people read ), because it is asked so often. It is in the FAQ. :-O --Mike-- Rollin' in my 5.0 With the rag-top down so my hair can blow. My really out-of-date homepage Sonork - 100.10414 AcidHelm Big fan of Alyson Hannigan and Jamie Salé.

[Li kai Liu Angus]

That's all right, your explanation has been very helpful. I really appreciate it:) Angus Liu

[Li kai Liu Angus]

Thanks! I read that:) However, I still think there is a way out other than using the OCR technique... I got an English-Chinese translation software here called Dreye 2002. Well, the name of the program isn't important. It does the trick of extracting the word in the sentence from where your mouse's pointed at. I don't think it involves OCR because it can't read text in the bitmap. Another reason is that recognition of Chinese character is so complicated that can't be always 100% correct. But this program can read chinese character with no errors and translate it into English on the fly, and vice versa. It looks to me that it can read any piece of text drawn by Windows, including text in RichEdit control, WebBrowser Control, Menu, Title Bar of a window, etc. virtually every piece of text so long as it is not in bitmap format. I've looked up in MSDN for the solution. My attempt wasn't very successful. That's why I come up with this question:) Anyway, it's so nice to have so many experts around and tries to answer my question. Thank you all! Angus Liu