how to extract non ascii charachters from a text.
-
hi, i have a text in an "unknown text encoding" and I need to extract all non ascii charachters from it; replace them with some ascii tags; do some processing and then; replace the original non ascii chars back. *(non ascii charachters like japanese and chinese charachter sets) well basically i have a vrml scenegraph and i need to render it using the open inventor renderer, which has problems with non ASCII charachters. so I am doing this .. get the sceneData in memory as binary data void *sceneData= ; scenedata(sizeof(filesize)); then read each BYTE from memory buffer; check the ASCII value to each BYTE; if outside ASCII range -> then replace it with a TAG. ELSE copy as such; NOW is this check a correct way to do it .. i mean do both the bytes of the multibyte charachters need to be outside the ASCII range ??? is there a better alternative way than this HACK !! please suggest !! :-( i am totally stuck
-
hi, i have a text in an "unknown text encoding" and I need to extract all non ascii charachters from it; replace them with some ascii tags; do some processing and then; replace the original non ascii chars back. *(non ascii charachters like japanese and chinese charachter sets) well basically i have a vrml scenegraph and i need to render it using the open inventor renderer, which has problems with non ASCII charachters. so I am doing this .. get the sceneData in memory as binary data void *sceneData= ; scenedata(sizeof(filesize)); then read each BYTE from memory buffer; check the ASCII value to each BYTE; if outside ASCII range -> then replace it with a TAG. ELSE copy as such; NOW is this check a correct way to do it .. i mean do both the bytes of the multibyte charachters need to be outside the ASCII range ??? is there a better alternative way than this HACK !! please suggest !! :-( i am totally stuck