Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
  1. Home
  2. General Programming
  3. C / C++ / MFC
  4. Very weird problem...

Very weird problem...

Scheduled Pinned Locked Moved C / C++ / MFC
helpquestion
18 Posts 5 Posters 0 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S Sprudling

    While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

    020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
    20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
    7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
    2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

    Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

    R Offline
    R Offline
    Rick York
    wrote on last edited by
    #2

    Have you checked for virii lately ?

    S 1 Reply Last reply
    0
    • R Rick York

      Have you checked for virii lately ?

      S Offline
      S Offline
      Sprudling
      wrote on last edited by
      #3

      As a matter of fact I just did earlier today. Using NAV 2002 with the most up to date virus definitions, and no virus found. I really think there is a logical explanation to this. Sprudling

      R 1 Reply Last reply
      0
      • S Sprudling

        As a matter of fact I just did earlier today. Using NAV 2002 with the most up to date virus definitions, and no virus found. I really think there is a logical explanation to this. Sprudling

        R Offline
        R Offline
        Rick York
        wrote on last edited by
        #4

        You are probably right. I was grasping at straws. :)

        1 Reply Last reply
        0
        • S Sprudling

          While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

          020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
          20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
          7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
          2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

          Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

          N Offline
          N Offline
          Nish Nishant
          wrote on last edited by
          #5

          Check your Notepad default font settings Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

          S 1 Reply Last reply
          0
          • N Nish Nishant

            Check your Notepad default font settings Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

            S Offline
            S Offline
            Sprudling
            wrote on last edited by
            #6

            That was of course the first thing I checked and it's the WinXP default courier. It's only that file (and others starting with the same numbers in hex) that shows up in chinese. Have any of you even tested yet? Sprudling :omg:

            N 1 Reply Last reply
            0
            • S Sprudling

              That was of course the first thing I checked and it's the WinXP default courier. It's only that file (and others starting with the same numbers in hex) that shows up in chinese. Have any of you even tested yet? Sprudling :omg:

              N Offline
              N Offline
              Nish Nishant
              wrote on last edited by
              #7

              What is the file extension you are using? The way I understand notepad is that if the extension is TXT it simply reads the file as ASCII and dumps it on the multi line edit control which is it's main control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

              S 1 Reply Last reply
              0
              • S Sprudling

                While I was coding a fileparser I stubled upon something _very_ weird. My file started like this (in hex):

                020102416E6369656E74277320506C6564676501536869656C64730152616C034F72740354616C012B353025
                20456E68616E63656420446566656E736504436F6C6420526573697374202B34332504466972652052657369
                7374202B343825044C696768746E696E6720526573697374202B34382504506F69736F6E2052657369737420
                2B343825043130252044616D61676520476F657320546F204D616E6100426C61636B01436C7562730248616D

                Try pasting this into a new file using a hexeditor, save it, and then open it in Notepad. If you have chinese character-set installed it will for some reason show up in chinese. Why?! This is mostly plain english... I've tested this lots of times, and there seems to be no way around it, except uninstalling the chinese character set! Sprudling :omg: :eek: :-O PS! I'm using WindowsXP Pro

                T Offline
                T Offline
                Tim Smith
                wrote on last edited by
                #8

                Probably what is happening is that Notepad is using an auto-detect to see what encoding you are using. With those binary characters at the start, it is getting fooled into thinking it is something like UTF-16 (or whatever they call UNICODE). Tim Smith Descartes Systems Sciences, Inc.

                N 1 Reply Last reply
                0
                • T Tim Smith

                  Probably what is happening is that Notepad is using an auto-detect to see what encoding you are using. With those binary characters at the start, it is getting fooled into thinking it is something like UTF-16 (or whatever they call UNICODE). Tim Smith Descartes Systems Sciences, Inc.

                  N Offline
                  N Offline
                  Nish Nishant
                  wrote on last edited by
                  #9

                  I seriously doubt that Notepad does anything of that sort. It's only functionality is to open the file and read the contents into a multi line edit control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                  T 1 Reply Last reply
                  0
                  • N Nish Nishant

                    I seriously doubt that Notepad does anything of that sort. It's only functionality is to open the file and read the contents into a multi line edit control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                    T Offline
                    T Offline
                    Tim Smith
                    wrote on last edited by
                    #10

                    BZZZZZ.... Wrong. Save something from Notepad. On the save dialog you get to select the encoding. Now, since there is no magical encoding flag in the file system, that information has to be stored in the file. When you open a file in notepad, it goes through some nasty work to try figure out the encoding. For example, UTF-16 can commonly be detected by looking for alternating 0 bytes. Tim Smith Descartes Systems Sciences, Inc.

                    N 1 Reply Last reply
                    0
                    • T Tim Smith

                      BZZZZZ.... Wrong. Save something from Notepad. On the save dialog you get to select the encoding. Now, since there is no magical encoding flag in the file system, that information has to be stored in the file. When you open a file in notepad, it goes through some nasty work to try figure out the encoding. For example, UTF-16 can commonly be detected by looking for alternating 0 bytes. Tim Smith Descartes Systems Sciences, Inc.

                      N Offline
                      N Offline
                      Nish Nishant
                      wrote on last edited by
                      #11

                      Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                      T S 2 Replies Last reply
                      0
                      • N Nish Nishant

                        Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                        T Offline
                        T Offline
                        Tim Smith
                        wrote on last edited by
                        #12

                        heh... COPY CON Been there, done that. GOD I AM GETTING OLD!!! Tim Smith Descartes Systems Sciences, Inc.

                        N 1 Reply Last reply
                        0
                        • N Nish Nishant

                          What is the file extension you are using? The way I understand notepad is that if the extension is TXT it simply reads the file as ASCII and dumps it on the multi line edit control which is it's main control. Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                          S Offline
                          S Offline
                          Sprudling
                          wrote on last edited by
                          #13

                          The extension doesn't matter. I can rename the file to whatever I want, and it shows the same chinese signs. Anyway, I think I've figured out why this happens how. Then the WinXP parses a file it tries to find out if it's ASCII or Unicode. An experiment: - Create a new .txt file in Notepad, enter "Sprudling", and save it as usual. - Open the file in an hexeditor and insert 2 bytes in front og "Sprudling". - Set the 2 bytes to FFFE, and save the file again. - Now open it again in Notepad. It's now shown in unicode as "灓畲汤湩", which is 4 chinese letters for those who actually have installed those letters. I guess Notepad is more than you thought... :] Sprudling :)

                          N 1 Reply Last reply
                          0
                          • T Tim Smith

                            heh... COPY CON Been there, done that. GOD I AM GETTING OLD!!! Tim Smith Descartes Systems Sciences, Inc.

                            N Offline
                            N Offline
                            Nish Nishant
                            wrote on last edited by
                            #14

                            Tim Smith wrote: GOD I AM GETTING OLD!!! Yes, I agree ;P Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                            1 Reply Last reply
                            0
                            • S Sprudling

                              The extension doesn't matter. I can rename the file to whatever I want, and it shows the same chinese signs. Anyway, I think I've figured out why this happens how. Then the WinXP parses a file it tries to find out if it's ASCII or Unicode. An experiment: - Create a new .txt file in Notepad, enter "Sprudling", and save it as usual. - Open the file in an hexeditor and insert 2 bytes in front og "Sprudling". - Set the 2 bytes to FFFE, and save the file again. - Now open it again in Notepad. It's now shown in unicode as "灓畲汤湩", which is 4 chinese letters for those who actually have installed those letters. I guess Notepad is more than you thought... :] Sprudling :)

                              N Offline
                              N Offline
                              Nish Nishant
                              wrote on last edited by
                              #15

                              Yeah. Tim Smith cleared that up for me :-) But he says he's getting old :-) Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                              1 Reply Last reply
                              0
                              • N Nish Nishant

                                Oh Boy!!! Thanks Tim S. Thanks for the info. But this is bad. This means notepad is not a pure text editor anymore. Now what do we do? Windows comes with not a single pure text editor. Do we all have to go back to the COPY CON days??? Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                                S Offline
                                S Offline
                                Sprudling
                                wrote on last edited by
                                #16

                                You could always use edit.com ;P Sprudling

                                N M 2 Replies Last reply
                                0
                                • S Sprudling

                                  You could always use edit.com ;P Sprudling

                                  N Offline
                                  N Offline
                                  Nish Nishant
                                  wrote on last edited by
                                  #17

                                  edit.com was introduced with DOS 5.0 [it was part of the qbasic set] later it became stand-alone Nish Sonork ID 100.9786 voidmain www.busterboy.org If you don't find me on CP, I'll be at Bob's HungOut

                                  1 Reply Last reply
                                  0
                                  • S Sprudling

                                    You could always use edit.com ;P Sprudling

                                    M Offline
                                    M Offline
                                    moliate
                                    wrote on last edited by
                                    #18

                                    You can use IsTextUnicode and see what it returns. This is probably what notepad does.. /moliate

                                    1 Reply Last reply
                                    0
                                    Reply
                                    • Reply as topic
                                    Log in to reply
                                    • Oldest to Newest
                                    • Newest to Oldest
                                    • Most Votes


                                    • Login

                                    • Don't have an account? Register

                                    • Login or register to search.
                                    • First post
                                      Last post
                                    0
                                    • Categories
                                    • Recent
                                    • Tags
                                    • Popular
                                    • World
                                    • Users
                                    • Groups