FindFirstFileEx() and Unicode

[treddie]

Howdie. I am researching how to use the Unicode version of FindFirstFileEx() in vb.Net and am having a heck of a time getting this to work. I had downloaded an example and it used the standard version, FindFirstFile(). I then modified the code to work with FindFirstFileEx(), instead, and got that to work, too, but only for ANSI characters (Alias "FindFirstFileExA"). But when I attempted to use the Unicode version (Alias "FindFirstFileExW"), it fails miserably. The best I have been able to do is get a non-zero return value from the function (which means it found the folder), but all the returned WFD structure contains are null strings for the folder name and alternate folder name. I have googled the problem (MSDN is of no help in this regard) and did find one reference on how to alter the function parameters to provide pointers instead of a string and the WFD structure, themselves. At any rate, here is my code for just the top half that looks for folders. The bottom half deals with searching for files, but it is just a repeat of this example, essentially. If I can get this top half to work with your help, I can certainly get the bottom half to work. I would have liked to attach a zip containing the project and the test folder I used, but it does not look like attachments are possible, here. Thanks for any help!

Option Strict Off
Option Explicit On
'Option Infer Off

Imports VB = Microsoft.VisualBasic
Imports System.Runtime.InteropServices

Friend Class Form1
Inherits System.Windows.Forms.Form

'Private Declare Function FindFirstFileExW Lib "kernel32.dll" Alias "FindFirstFileExW" (ByVal lpFileName As String, ByVal fInfoLevelId As FINDEX_INFO_LEVELS, ByVal lpFindFileData As WIN32_FIND_DATA, ByVal fSearchOp As FINDEX_SEARCH_OPS, ByRef lpSearchFilter As Int32, ByVal dwAdditionalFlags As Integer) As Long
Private Declare Function FindFirstFileExW Lib "kernel32.dll" Alias "FindFirstFileExW" (ByVal lpFileName_IntPtr As IntPtr, ByVal fInfoLevelId As FINDEX_INFO_LEVELS, ByRef lpFindFileData_IntPtr As IntPtr, ByVal fSearchOp As FINDEX_SEARCH_OPS, ByRef lpSearchFilter As Int32, ByVal dwAdditionalFlags As Integer) As Long

Private Declare Function FindNextFileW Lib "kernel32" Alias "FindNextFileW" (ByVal hFindFile As Long, ByRef lpFindFileData As WIN32_FIND_DATA) As Integer

Private Declare Function GetFileAttributesW Lib "kernel32" Alias "GetFileAttributesW" (ByVal lpFileName As String) As Integer

Private Declare Function FindClose Lib "kernel32" (ByVal hFindFile A

[Dave Kreskowiak]

May I ask to what end are you doing this? Why go through all of this trouble when the functionality of it is wrapper in the System.Io.FileInfo class? Your return type for the Declare is wrong. You've got it as Long when it returns an Int32. You may want to look at this[^] before proceeding. You don't need to specifically say your using the W version. The declare on the linked page will selected the W version if its available.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[treddie]

Dave Kreskowiak wrote:

May I ask to what end are you doing this? Why go through all of this trouble when the functionality of it is wrapper in the System.Io.FileInfo class?

System.Io.FileInfo is limited to the MAX_PATH =260 barrier, as far as I can tell.

Dave Kreskowiak wrote:

Your return type for the Declare is wrong. You've got it as Long when it returns an Int32.

I tried Int32, but the function returns -1 in that case. However, using Long seems completely wrong too, because the function is inside a 32bit set of libraries...win32. So it can't deal with a 64bit Long.

[Dave Kreskowiak]

treddie wrote:

System.Io.FileInfo is limited to the MAX_PATH =260 barrier, as far as I can tell.

Yeah, that's because it's the limit in Windows, not .NET. The current path limit is 248 characters and the filename limit is 260.

treddie wrote:

I tried Int32, but the function returns -1 in that case. However, using Long seems completely wrong too, because the function is inside a 32bit set of libraries...win32. So it can't deal with a 64bit Long.

The return value is either going to be a valid file handle, or INVALID_HANDLE_VALUE if the search fails for any reaons. IIRC, INVALID_HANDLE_VALUE is 0xFFFFFFFF, -1 decimal. It's been telling that that the call to FindFirstFileEx has been failing this entire time. Check the return value before you try to use it. If it's -1, call GetLastError for the error code and it'll give you a clue as to why it failed. But, I think you're wasting your time doing it the really hard way.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[treddie]

Dave Kreskowiak wrote:

that's because it's the limit in Windows, not .NET. The current path limit is 248 characters and the filename limit is 260.

But there are easy ways to accidentally create pathnames much much longer than that. When I try the FileInfo class, I run into that barrier. FindFirstFileEx() allows me to get around that problem. I have that working fine. But the Unicode part of it is what is biting me. Incidentally, I also found through tests that the TRUE limit for Windows, is 240 characters max. When you subtract the 8.3name from the 260 and get 248, you still have to remove another 3 for the root folder...That leaves you with 245, but then you cannot use another 5 characters. This may be a bug in Windows, but in some cases, if you try to go beyond 240, you can open/save a file OK, even move it or change its name, but you may not be able to right-click it without crashing Windows and forcing it to restart. Fortunately, you do not lose any of your running apps, but all the desktop windows you had open, get closed. UPDATE: Error code is 2, file not found. Which repeats what the returned value (-1)of the function says. So no help from GetLastError, there. :(

[treddie]

I found at least part of the problem. A string variable cannot contain Unicode characters. So Dim lpFileName_str As String = "\\?\UNC\" & path & "\*" will not work and comes back with, for example, "\\?\UNC\d:\?????????\*". Now I think I see why the string parameter needs to be changed to a pointer to the path, rather than a string representing the path. Also, when I put the path into memory, and then retrieve it, everything is correct except the "\*" at the end...It gets interpreted as "[foreign character]*". So I think I'm finally on to at least part of the solution. I'll take a look at how to handle backslashes in Unicode strings and see what happens. That also explains why the return value from the function was "-1", file not found.

[Garth J Lancaster]

this might not do all you want - but Uwe Keim from here on CP & 'Zeta' did some work on this Zeta Long Paths[^] http://zetalongpaths.codeplex.com/[^] hth 'g'

[Dave Kreskowiak]

treddie wrote:

I also found through tests that the TRUE limit for Windows, is 240 characters max.

Not quite. A fully qualifed filePATH to a file (including filename) is 248 characters max. A fileNAME (with no path information at all) is 260 characters max. Think of setting the "current directory" first before trying to access a file with a name like that. That's one way to get a filepath longer than the 248 character limit.

treddie wrote:

This may be a bug in Windows

Considering that code has been around for, literally, decades, its not a bug. These are hardcoded limits in NTFS.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[Dave Kreskowiak]

treddie wrote:

string variable cannot contain Unicode characters

Considering every single string in .NET is stored as Unicode, that statement is not true. The encoding is UTF-16.

treddie wrote:

for example, "\\?\UNC\d:\?????????\*

The problem with what your looking at is that not every font has glyphs for every Unicode character, so you get boxes for those. In addition, if the string is being displayed by a control or other rendering device that doesn't support the encoding, characters that are not recognized are replaced with question marks. Read this[^].

treddie wrote:

Now I think I see why the string parameter needs to be changed to a pointer to the path, rather than a string representing the path

Hehe. You never pass a "string". Strings are always passed by pointer. You can only put one value, per parameter, on the stack when function call is made. In the case of strings, or other object that is not an intrinsic type, it's always a pointer.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[treddie]

Bummer...It's in some form of C. I'm not a C guy. I can struggle through it and eventually figure it out, but the hardest part is converting to vb and getting the function declarations correct. One oddity in his code, is I did not see any references to FindFirstFileEx()...Only FindFirstFile() which cannot deal with wide path names.

[treddie]

Dave Kreskowiak wrote:

string variable cannot contain Unicode characters

Yah, that was sloppy reasoning on my part. Although, I also can't explain why I was getting inconsistent behavior in what was being reported in the string. Sometimes, when I would pass the pathname to the string, it would come back with ?'s. So without having a stable situation that I can depend on, I can't come to any conclusion on what that was all about. Especially since I can't duplicate the problem. But now, what I have found is that the string holds the foreign characters just fine, but something odd happens when I send the string to an RTB and a plain TextBox. The RTB displays "\*" as "*", but the TextBox had no problem at all and correctly shows, "\*". But if I hover my mouse over the "rtb1.text =" statement in the code, it reads fine! Weird.

Dave Kreskowiak wrote:

You never pass a "string".

But why do they declare strings in any function that accepts them? I've used ANSI strings before with no problem, just as MSDN suggests. The headaches happen when I get to Unicode strings.

[treddie]

Dave Kreskowiak wrote:

I also found through tests that the TRUE limit for Windows, is 240 characters
max.

I'm not so sure about that, Dave. I ran some extensive tests many months ago when working on this program and found that there are 4 Windows limits. Their numbers are as follows, according to MS: Folder name = 260 characters File name (w/ extension) = 260 characters Folder path w/o file name = 260 characters Full path w/ filename = 260 characters Obviously you can't have 260 in all categories so it's a matter of whatever the actual amounts are for each category, that they don't exceed any of the 4 limits. At any rate, I tested all 4 cases and found that when you subtract out the twelve 8.3-filename chars, the 3 root characters, and the null character at the end of any path (all the characters the user has no control over), you are left with at that point only 244 characters, including backslashes, that you can play with. But, you have to actually limit yourself to 240 maximum, user-controlled characters to prevent the Windows crash on folder/file name right-click -> properties. I think that last case classifies as a bug in that your Windows session gets trashed if you go over 240 characters when getting a file's properties. This was true in my tests for both WinXP (latest SP), and Win7 (6.1.7600 Build 7600), although the manner of the crash was different for each system. The WinXP crash was really bad and would hang the computer.

[Dave Kreskowiak]

treddie wrote:

But why do they declare strings in any function that accepts them?

Because you're definfing the TYPE of the pointer. What kind of object does it point to? What are the rules for incrementing that pointer if needed?? If you go back to C, a "string" in it's most basic form is just a pointer to an array of type char (character). Under Windows, you can have ASCII strings and wide-character strings. Incrementing the pointer for an ASCII character just adding 1 to the pointer, but, for multi-byte characters, such as Unicode, you increment the pointer by 2 or more.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[treddie]

That's logical.

[MicroVirus]

Actually, NTFS is not at all that limited. It's just that the kernel32.dll API interface is limited to those path limits. From MSDN, CreateFile page: In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path. For more information, see Naming Files, Paths, and Namespaces. By adding \\?\ you tell the kernel32 API to NOT do any parsing or processing on the path, and instead pass it directly on to the filesystem driver. From MSDN page on Naming Files, Paths, and Namespaces (Windows): For file I/O, the "\\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system. But not all functions can work with the extended paths, because some functions inherently need to do processing on the path.

[Dave Kreskowiak]

Yeah, I knew that. Slip of the brain when I typed it.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak

[treddie]

OK, I'm finally getting this to work. I took the suggestion more seriously that GetLastError() might provide some insight into the problem. It wasn't anything that explicitly told me what was wrong, but it did cause me to take a much closer look at all of my declarations, and realizing that using "\\?\UNC\" was not correct...It needs to be just "\\?\". But also took to heart your comments on strings and pointers. In that regard, I found some typing problems here and there, and missed some lines that were still not rewritten for pointers. I'll post the resulting code here, shortly, after I copy my test code over to my final demo program and fix some remaining issues.

[TnTinMn]

Hi treddie, Based on my understanding of your code, I see no reason to use FindFirstFileEx versus FindFirstFile. Also, the directory filter flag option can fail silently if it is not supported and you need to check the attributes anyways if it was your intent to just recover directories. But it appears as your code is also doing these checks. Just a word of caution about interop examples you find on the web; many of these were originally done in VB6 and may not have been properly modified for DotNet types. Here are links to a reference and a tool that you may find useful in working with interop. Miscellaneous Marshaling (look at the links under "See Also") Samples[^] PInvoke Interop Assistant[^] I was bored so I tried putting together a simple TreeView file explorer that bypasses the 260 character limit. It may serve as an example for what you are trying to do. Just place a TreeView control on new WinForm project and paste this code. I used the command prompt "Subst" command to create a drive reference to a path that reached the 260 character barier and then added some more directories under that substituted drive to get to a file that had 400+ characters in it's path to test this. It appears to work. Edit: There appears to be a length limit on codeblock code.

Imports System.Runtime.InteropServices

Public Class Form1
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
TreeView1.Nodes.Add(DriveToTreeNode("c"c))
End Sub

Private Sub TreeView1_AfterCollapse(ByVal sender As Object, ByVal e As System.Windows.Forms.TreeViewEventArgs) Handles TreeView1.AfterCollapse
e.Node.Nodes.Clear()
e.Node.Nodes.Add(New TreeNode With {.Tag = New NodeData With {.IsDirectory = True}}) ' add dummy node
End Sub

Private Sub TreeView1_BeforeExpand(ByVal sender As Object, ByVal e As System.Windows.Forms.TreeViewCancelEventArgs) Handles TreeView1.BeforeExpand
' only directories can have nodes
e.Node.Nodes.Clear()
EnumerateDirectory(e.Node)
End Sub

Private Sub TreeView1_MouseDoubleClick(ByVal sender As Obje

[TnTinMn]

Here is the remainder of the code.

Private Shared Function GetShortPathName(ByVal path As String) As String
Dim sb As New System.Text.StringBuilder(1000)
Dim len As Int32 = GetShortPathName(LongPathPrefix & path, sb, sb.Capacity)
If len > sb.Capacity Then
sb.Capacity = len
GetShortPathName(LongPathPrefix & path, sb, sb.Capacity)
End If
Return sb.ToString().Replace(LongPathPrefix, "")
End Function

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode), BestFitMapping(False)> _
Private Class WIN32_FIND_DATA
<MarshalAs(UnmanagedType.U4)> Public dwFileAttributes As IO.FileAttributes
<MarshalAs(UnmanagedType.U4)> Private ftCreationTime_dwLowDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Private ftCreationTime_dwHighDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Private ftLastAccessTime_dwLowDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Private ftLastAccessTime_dwHighDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Private ftLastWriteTime_dwLowDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Private ftLastWriteTime_dwHighDateTime As UInt32
<MarshalAs(UnmanagedType.U4)> Public nFileSizeHigh As UInt32
<MarshalAs(UnmanagedType.U4)> Public nFileSizeLow As UInt32
<MarshalAs(UnmanagedType.U4)> Public dwReserved0 As UInt32
<MarshalAs(UnmanagedType.U4)> Public dwReserved1 As UInt32

  ' Note: by changing the cFileName size constant from 260 to 32767 to handle long
  ' path names, it appears that cAlternateFileName always returns blank
  ' if the ShortPathName is needed, then use the GetShortPathName function
  ' http://msdn.microsoft.com/en-us/library/windows/desktop/aa364989%28v=vs.85%29.aspx
   <MarshalAs(UnmanagedType.ByValTStr, SizeConst:=32767)> Public cFileName As String
   <MarshalAs(UnmanagedType.ByValTStr, SizeConst:=14)> Public cAlternateFileName As String

  Public Function CreationTime() As DateTime
     Return DateTime.FromFileTime(MergeToInt64(ftCreationTime\_dwLowDateTime, ftCreationTime\_dwHighDateTime))
  End Function
  Public Function LastAccessTime() As DateTime
     Return DateTime.FromFileTime(MergeToInt64(ftLastAccessTim

[treddie]

But FindFirstFile() cannot handle Unicode, as far as MSDN seems to suggest.