Great information! I just wanted to add that BOMs (byte order marks) aren't always present in a text file as well. There no requirement for BOMs. While there's no reliable way to detect encoding - like you said - web browers and other applications (like Word) do try to detect the encoding. If you - the original poster - needs to do something like that, a simple (but probably not the most efficient way) is to take a random sampling of strings within the text file and use StringInfo.GetTextElementEnumerator and enumerate the text elements. With either all of those or a random sample, call TextElementEnumerator.GetTextElement (returns a String) and check the Length. If it's greater than one, you at least know you're dealing with a multi-byte character set (MBCS), like UTF-8. If all of them were 2 bytes, then it's likely it's a double-byte character set (DBCS), like UTF-16 (there's also 4-byte characters, known as UTF-32!). If they're all 1 byte, then you've probably got an ASCII (or other single-byte encoding) file. From there you can make certain assumptions. You see browsers doing this when they start displaying question marks for chracters in odd places (this also happens when the specified encoding is wrong).
Microsoft MVP, Visual C# My Articles
