News:

MASM32 SDK Description, downloads and other helpful links
MASM32.com New Forum Link
masmforum WebSite

Writing data from MapViewOfFile to another file.

Started by white scorpion, July 02, 2006, 09:06:17 PM

Previous topic - Next topic

white scorpion

I presented myself with a new nice little problem...
The program i have written works like this:

1. copy inputfile to outputfile
2. map view of output file
3. do operations on outputfile
4. write data back to outputfile and close handles.

What i want to achieve is to do everything in memory from the inputfile, and then write the complete file back as outputfile without changing the inputfile. So that the outputfile will only be created if everything went as expected.

I've tried doing this by mapping view of inputfile and then call WriteFile to write the contents to outputfile.
Even though this is working as expected, an unexpected side-effect is that it also writes the same data back to inputfile.
This is without flushing by calling FlushViewOfFile() or even without actually calling UnmapViewOfFile().
As soon as the handle is closed the inputfile contains the new data.
As long as UnmapViewOfFile() has not been called the file handle stays open as well.

Maybe i'm thinking too difficult, but i can't find a proper solution for this.
Reading inputfile with ReadFile into a buffer is no solution since i can never create a buffer of a few 100mb's, plus the performance is extremely degraded when doing this.

What would be the best solution for this problem?

asmfan

WriteFile takes a buffer address as a parameter so why don't to pass a address of MMF in memory... I think it will succeed
Russia is a weird place

white scorpion

That's not the problem, i've done that.
But then to close the MMF without putting the data back to the original file is.
If i map it as READ, then i am unable to change the file in memory, if i map it as R/W, then it automatically gets written back to the original file as well.

asmfan

If you don't call FlushViewOfFile and just call UnmapViewOfFile isn't it solution?
Russia is a weird place

white scorpion

Nope, that was the most logical thing to try, but without calling UnMapViewOfFile you can't even close the handle to the file.
even if you kill the process before calling those API's, it's still written to the file somehow...

arafel

I think you can't avoid a file to be modified when using FILE_MAP_WRITE flag. The changes are written directly to a file from the pending operations buffers, so it doesn't really matter if you don't use UnMapViewOfFile or FlushViewOfFile (which is used only to assure that all pending changes have been flushed down) the changes applied to a file as you make them, not after closing the handle.(although I might be wrong)

If you need to process the file sequentially, that is without accessing random far located offsets, perhaps then it would be better to copy parts of a mapped file chunk by chunk to other smaller buffer, process it there and write to the output. If at some point operation fails, simply delete the output. I know this what you wanted to avoid, but I see no other method without allocating memory block of a file`s size.

TNick

But how about this:
- you create the output file and store the handle
- you copy your input file to a temporary file, and you map this file
- use some kind of flag, to know at the end if you want to keep the output file
- at the end, you unmap the file
- you close the handle to output file
- if your flag is set, you copy the temporary file to output file

white scorpion

QuoteI think you can't avoid a file to be modified when using FILE_MAP_WRITE flag. The changes are written directly to a file from the pending operations buffers, so it doesn't really matter if you don't use UnMapViewOfFile or FlushViewOfFile (which is used only to assure that all pending changes have been flushed down) the changes applied to a file as you make them, not after closing the handle.(although I might be wrong)
I'm afraid your right :(
Quote
If you need to process the file sequentially, that is without accessing random far located offsets, perhaps then it would be better to copy parts of a mapped file chunk by chunk to other smaller buffer, process it there and write to the output. If at some point operation fails, simply delete the output. I know this what you wanted to avoid, but I see no other method without allocating memory block of a file`s size.
This the the whole issue.
Bytes are swapped from side to side in the file. This can be over the full 0xFFFFFFFF filesize depending on the password entered and tablefile used.

@TNick,
Then it would be just as easy to already copy the file and delete it if something goes wrong.
My biggest concern is the data it can contain. If i use a temp location, then the file stored there can be retrieved easily.
If the computer crashes or the programs process gets killed before it is actually encrypting the file, then you have a double copy of the plaintext. Since the user selected the outputfile it will know it is originated from the plaintext. If i would use a tempfile, then the user would have no clue a tempfile still existed.

As long as i have no other solution i will keep it like it is  ::)

asmfan

Hey, i remembered that there is option called *COPY* (don't remember full definition) that prevents changes of basic file making a copy if there are changes... See MSDN on flags of MMF functions...
Russia is a weird place