memory mapped

sarfaraznawaz

can any one explain about the memory mapping of file what it does actually and how to use it... please provide a example........ thanking you sarfaraz

Niklas L

Memory mapping allows you to treat a file, or portion of a file, as an array of bytes, just like any other array in C/C++. I suggest you read about it here[^], since it can be quite a complex topic.

home

sarfaraznawaz

can i used it for reading the 64gb of data should i have to create the MMF and the i have to read ?...... here i am trying to take the c drives which of 64gb displaying in windows client area .for this how to create MMF and to Read. i know the function of MMF but not able to implement if in proper way give me example to create and read the data for particular on drives ........... thanking you sarfaraz

Niklas L

This is almost impossible to tell without knowing your data. Do you have one 64GB file, or many smaller with a total of 64GB? Are they text or binary files? If you want to display one big file in a window, you can map a portion of the file around the part you want to display, and read that information. But without knowing the data structure of your file(s), it's hard to tell how to sync data with display. E.g. if you have a text file, you might want to manage some line or character count to calculate which portion of the file to map. If it's structured binary, you need something similar. Please give some info on what you have, and how you want it displayed.

home

sarfaraznawaz

thanks its is \\C: drive i am want to display in hexadecimal

Alexander M

as you usually read full sectors when accessing the hdd, you should use normal file i/o for it. allocate a buffer that is big enough to keep the maximum amount of data the hex editor can display and read it from disk just the way you to with a normal file. memory mapping 64gb can only work with 64-bit os, as it requires a very big virtual address space. but i haven't tried to map that much data so far.

Don't try it, just do it! ;-)

Niklas L

Here is a sample of how to map a small portion of a large file:

const TCHAR* path = _T("C:\\file.dat");
DWORD maxBytesToMap = 64*1024;

// Open the file
HANDLE hFile = ::CreateFile(path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, 0);

// Get file size for verification
DWORD fileSizeHigh;
DWORD fileSizeLow = ::GetFileSize(hFile, &fileSizeHigh);

// Create the file mapping for the entire file
HANDLE hShMem = ::CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, NULL);

// Create a smaller view at a specific region in the file
// fileOffsetHigh and fileOffsetLow is where the view should begin
unsigned char* pBytes = (unsigned char*)::MapViewOfFile(hShMem, FILE_MAP_READ,
fileOffsetHigh, fileOffsetLow, maxBytesToMap);

// Read data from pBytes
for (size_t i = 0; i < byteCount; ++i)
x = pBytes[i];

Error checking and clean-up omitted.

home

sarfaraznawaz

thanks 1)i tried but not able to understand what actually memory mapped file will do .... 2)i also studied in MSDN about this but i not understood the virtual address space ... 3)how it will access the data ...... please clear the above points......... now see how i had written the code for this.... HANDLE hfile,hmapfile; hfile = CreateFile(_T("C:\\saad.txt"),GENERIC_READ,FILE_SHARE_WRITE|FILE_SHARE_READ,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL); if(hfile == INVALID_HANDLE_VALUE) { MessageBox(_T("Fail to open the drive"),_T(" "),MB_OK | MB_ICONERROR); return -1; } DWORD filesize = GetFileSize(hfile,NULL); hmapfile = CreateFileMapping(hfile,NULL,PAGE_READWRITE |PAGE_WRITECOPY |SEC_COMMIT |SEC_RESERVE,filesize,filesize,NULL); if(hfile == INVALID_HANDLE_VALUE) { MessageBox(_T("Fail to open the drive"),_T(" "),MB_OK | MB_ICONERROR); return -1; } DWORD pbuf = (DWORD) MapViewOfFile(hmapfile,FILE_MAP_COPY |FILE_MAP_ALL_ACCESS,0,filesize,filesize); one more thing i want ask should i have read the file with Readfile ???

Niklas L

sarfaraznawaz wrote:

1)i tried but not able to understand what actually memory mapped file will do ....

A memory mapped file is faster than than reading a file the conventional way. On the other hand, if you are reading/writing text, you will not benefit from having end line characters translated correctly for you, and have to do that manually.

sarfaraznawaz wrote:

2)i also studied in MSDN about this but i not understood the virtual address space ...

The virtual address space is nothing you have to worry about yet. It is the memory your process is able to read. The operating system will help you take care of business. However, a 32 bit application can never address more space than can be stored in a 32 bit unsigned type variable. This is why mapping 64Gb data all at once will be a problem.

sarfaraznawaz wrote:

3)how it will access the data ......

You access the data just as any other C/C++ array, by offsetting the returned pointer from MapViewOfFile().

sarfaraznawaz wrote:

one more thing i want ask should i have read the file with Readfile ???

It looks like using ReadFile() would be a dead end if you want to display the entire file. My guess is that you will suffer from real performance problems.

home

sarfaraznawaz

thanks i mapped the file but not able to display whole file in the window

Niklas L

You should only display a small portion of the file at any one time. You don't need to show more than fits (+ maybe a little extra for performance) The user won't notice since he/she cannot seethe whole file anyway. Manage scrollbars manually.

home

sarfaraznawaz

thanks again ... but i not understand clearly the parameter DWORD dwMaximumSizeHigh, DWORD dwMaximumSizeLow in CreateFileMapping(......) even in PVOID MapViewOfFile(DWORD dwFileOffsetHigh,DWORD dwFileOffsetLow,)not understood .. please clear it ....

Niklas L

Your DWORD is a 32 bit unsigned integer. That means it can store values up to 4 * 1024 * 1024 * 1024. You cannot point to anything higher than that in your file by just using one DWORD. This equals 4GByte. To be able to read data from a file larger than 4Gbyte, you will have to make use of the dwFileOffsetHigh, which simply speaking is the index of the 4Gbyte block in the file you want to read. For the first 4Gbyte, dwFileOffsetHigh should be zero, for the next 4Gbyte data, ranging from 4Gbyte to 8Gbyte, it should be 1, and so on. You have to do some simple math there to calculate the correct offset for your desired view.

home

sarfaraznawaz

thanks .... i written the code which is to read the c:drive its showing the bad pointer here my code hfile = CreateFile(_T("\\\\.\\C:"),GENERIC_READ,FILE_SHARE_WRITE|FILE_SHARE_READ,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL); DWORD filesizeLow = GetFileSize(hfile,&fileSizeHigh); HANDLE hmapfile = CreateFileMapping(hfile,NULL,PAGE_READONLY ,0,0,NULL); if(hfile == INVALID_HANDLE_VALUE) { MessageBox(_T("Fail to open the drive"),_T(" "),MB_OK | MB_ICONERROR); return -1; } CloseHandle(hfile); pbuf = (BYTE*) MapViewOfFile(hmapfile,FILE_MAP_READ ,0,0,filesizeLow); but its ok with the simple text file ...... one more thing i want to ask that is there any way to hide the folder or file........

Niklas L

I can spot two problems with your code. 1. You cannot read a drive. You have to read a file. What exactly is it you want to do? If you want to read raw data from the hard drive sector by sector, you'll need a different API. 2. You cannot close the handle before accessing the file mapping. Read what you have to, and close the file once you're done.

sarfaraznawaz wrote:

one more thing i want to ask that is there any way to hide the folder or file.....

I saw you started a new thread for this, which is good. Keep that discussion there, and you will more likely attract others who might know.

home

sarfaraznawaz

thanks ........... ya i want to read raw data from the hard drive sector by sector which is are the API are there

Niklas L

I found the following article on MSDN[^]. It's for reading sectors off a CD_ROM drive. Maybe it's possible for hard drives as well. If not, I suggest you start a new thread with this more specific requirement. (Reading sector by sector.)

home