Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
  1. Home
  2. General Programming
  3. C / C++ / MFC
  4. Very weird problem...

Very weird problem...

Scheduled Pinned Locked Moved C / C++ / MFC
helpquestion
18 Posts 5 Posters 0 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S Offline
    S Offline
    Sprudling
    wrote on last edited by
    #1

    While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

    020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
    20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
    7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
    2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

    Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

    R N T 3 Replies Last reply
    0
    • S Sprudling

      While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

      020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
      20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
      7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
      2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

      Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

      R Offline
      R Offline
      Rick York
      wrote on last edited by
      #2

      Have you checked for virii lately ?

      S 1 Reply Last reply
      0
      • R Rick York

        Have you checked for virii lately ?

        S Offline
        S Offline
        Sprudling
        wrote on last edited by
        #3

        As a matter of fact I just did earlier today. Using NAV 2002 with the most up to date virus definitions, and no virus found. I really think there is a logical explanation to this. Sprudling

        R 1 Reply Last reply
        0
        • S Sprudling

          As a matter of fact I just did earlier today. Using NAV 2002 with the most up to date virus definitions, and no virus found. I really think there is a logical explanation to this. Sprudling

          R Offline
          R Offline
          Rick York
          wrote on last edited by
          #4

          You are probably right. I was grasping at straws. :)

          1 Reply Last reply
          0
          • S Sprudling

            While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

            020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
            20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
            7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
            2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

            Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

            N Offline
            N Offline
            Nish Nishant
            wrote on last edited by
            #5

            Check your Notepad default font settings Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

            S 1 Reply Last reply
            0
            • N Nish Nishant

              Check your Notepad default font settings Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

              S Offline
              S Offline
              Sprudling
              wrote on last edited by
              #6

              That was of course the first thing I checked and it's the WinXP default courier. It's only that file (and others starting with the same numbers in hex) that shows up in chinese. Have any of you even tested yet? Sprudling :omg:

              N 1 Reply Last reply
              0
              • S Sprudling

                That was of course the first thing I checked and it's the WinXP default courier. It's only that file (and others starting with the same numbers in hex) that shows up in chinese. Have any of you even tested yet? Sprudling :omg:

                N Offline
                N Offline
                Nish Nishant
                wrote on last edited by
                #7

                What is the file extension you are using? The way I understand notepad is that if the extension is TXT it simply reads the file as ASCII and dumps it on the multi line edit control which is it's main control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                S 1 Reply Last reply
                0
                • S Sprudling

                  While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

                  020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
                  20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
                  7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
                  2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

                  Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

                  T Offline
                  T Offline
                  Tim Smith
                  wrote on last edited by
                  #8

                  Probably what is happening is that Notepad is using an auto-detect to see what encoding you are using. With those binary characters at the start, it is getting fooled into thinking it is something like UTF-16 (or whatever they call UNICODE). Tim Smith Descartes Systems Sciences, Inc.

                  N 1 Reply Last reply
                  0
                  • T Tim Smith

                    Probably what is happening is that Notepad is using an auto-detect to see what encoding you are using. With those binary characters at the start, it is getting fooled into thinking it is something like UTF-16 (or whatever they call UNICODE). Tim Smith Descartes Systems Sciences, Inc.

                    N Offline
                    N Offline
                    Nish Nishant
                    wrote on last edited by
                    #9

                    I seriously doubt that Notepad does anything of that sort. It's only functionality is to open the file and read the contents into a multi line edit control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                    T 1 Reply Last reply
                    0
                    • N Nish Nishant

                      I seriously doubt that Notepad does anything of that sort. It's only functionality is to open the file and read the contents into a multi line edit control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                      T Offline
                      T Offline
                      Tim Smith
                      wrote on last edited by
                      #10

                      BZZZZZ.... Wrong. Save something from Notepad. On the save dialog you get to select the encoding. Now, since there is no magical encoding flag in the file system, that information has to be stored in the file. When you open a file in notepad, it goes through some nasty work to try figure out the encoding. For example, UTF-16 can commonly be detected by looking for alternating 0 bytes. Tim Smith Descartes Systems Sciences, Inc.

                      N 1 Reply Last reply
                      0
                      • T Tim Smith

                        BZZZZZ.... Wrong. Save something from Notepad. On the save dialog you get to select the encoding. Now, since there is no magical encoding flag in the file system, that information has to be stored in the file. When you open a file in notepad, it goes through some nasty work to try figure out the encoding. For example, UTF-16 can commonly be detected by looking for alternating 0 bytes. Tim Smith Descartes Systems Sciences, Inc.

                        N Offline
                        N Offline
                        Nish Nishant
                        wrote on last edited by
                        #11

                        Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                        T S 2 Replies Last reply
                        0
                        • N Nish Nishant

                          Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                          T Offline
                          T Offline
                          Tim Smith
                          wrote on last edited by
                          #12

                          heh... COPY CON Been there, done that. GOD I AM GETTING OLD!!! Tim Smith Descartes Systems Sciences, Inc.

                          N 1 Reply Last reply
                          0
                          • N Nish Nishant

                            What is the file extension you are using? The way I understand notepad is that if the extension is TXT it simply reads the file as ASCII and dumps it on the multi line edit control which is it's main control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                            S Offline
                            S Offline
                            Sprudling
                            wrote on last edited by
                            #13

                            The extension doesn't matter. I can rename the file to whatever I want, and it shows the same chinese signs. Anyway, I think I've figured out why this happens how. Then the WinXP parses a file it tries to find out if it's ASCII or Unicode. An experiment: - Create a new .txt file in Notepad, enter "Sprudling", and save it as usual. - Open the file in an hexeditor and insert 2 bytes in front og "Sprudling". - Set the 2 bytes to FFFE, and save the file again. - Now open it again in Notepad. It's now shown in unicode as "灓畲汤湩", which is 4 chinese letters for those who actually have installed those letters. I guess Notepad is more than you thought... :] Sprudling :)

                            N 1 Reply Last reply
                            0
                            • T Tim Smith

                              heh... COPY CON Been there, done that. GOD I AM GETTING OLD!!! Tim Smith Descartes Systems Sciences, Inc.

                              N Offline
                              N Offline
                              Nish Nishant
                              wrote on last edited by
                              #14

                              Tim Smith wrote: GOD I AM GETTING OLD!!! Yes, I agree ;P Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                              1 Reply Last reply
                              0
                              • S Sprudling

                                The extension doesn't matter. I can rename the file to whatever I want, and it shows the same chinese signs. Anyway, I think I've figured out why this happens how. Then the WinXP parses a file it tries to find out if it's ASCII or Unicode. An experiment: - Create a new .txt file in Notepad, enter "Sprudling", and save it as usual. - Open the file in an hexeditor and insert 2 bytes in front og "Sprudling". - Set the 2 bytes to FFFE, and save the file again. - Now open it again in Notepad. It's now shown in unicode as "灓畲汤湩", which is 4 chinese letters for those who actually have installed those letters. I guess Notepad is more than you thought... :] Sprudling :)

                                N Offline
                                N Offline
                                Nish Nishant
                                wrote on last edited by
                                #15

                                Yeah. Tim Smith cleared that up for me :-) But he says he's getting old :-) Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                                1 Reply Last reply
                                0
                                • N Nish Nishant

                                  Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                                  S Offline
                                  S Offline
                                  Sprudling
                                  wrote on last edited by
                                  #16

                                  You could always use edit.com ;P Sprudling

                                  N M 2 Replies Last reply
                                  0
                                  • S Sprudling

                                    You could always use edit.com ;P Sprudling

                                    N Offline
                                    N Offline
                                    Nish Nishant
                                    wrote on last edited by
                                    #17

                                    edit.com was introduced with DOS 5.0 [it was part of the qbasic set] later it became stand-alone Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                                    1 Reply Last reply
                                    0
                                    • S Sprudling

                                      You could always use edit.com ;P Sprudling

                                      M Offline
                                      M Offline
                                      moliate
                                      wrote on last edited by
                                      #18

                                      You can use IsTextUnicode and see what it returns. This is probably what notepad does.. /moliate

                                      1 Reply Last reply
                                      0
                                      Reply
                                      • Reply as topic
                                      Log in to reply
                                      • Oldest to Newest
                                      • Newest to Oldest
                                      • Most Votes


                                      • Login

                                      • Don't have an account? Register

                                      • Login or register to search.
                                      • First post
                                        Last post
                                      0
                                      • Categories
                                      • Recent
                                      • Tags
                                      • Popular
                                      • World
                                      • Users
                                      • Groups