WriteFile with Progress

[Jackal]

Is there a way i can use createfile, writefile and have a progress bar that shows the progress as the file is bieing written. I would assume that i would have to muti thread it but i was unable to get it working when i tested it.

[Tedd]

You can do it with two threads: your main one taking care of the app window, and the extra one writing the file in chunks and updating some progress variable (that the main thread reads and adjusts the progress indicator appropriately.)
The other way is to use "overlapped IO" which basically means you set off your read/write and the function returns immediately, and then calls a function (supplied by you) when it's finished. So you write your chunks in overlapped mode (WriteFile can do this, but look at WriteFileEx) and then update the progress indicator when each chunk is finished. The main window should then remain responsive while this is in progress.

[Jackal]

ok ill give it a try again. Thanks for the reply..

[Ian_B]

Overlapped I/O will work, and it can be faster in some cases (because it's basically the kernel handling the multithreading you were thinking of doing) IF (and this is a big if) you are doing a lot of heavy processing on your file before writing, and you can interleave processing a fileblock with writing the block you just processed, but generally it's a lot more trouble that you may not need with a lot more programming overhead.

The simplest thing to do first would be to decide what size of memory buffer you want to allocate. That will dictate what "granularity" of detail you will get in the progressbar. It depends on the size of the file you are writing. If you are going to need a progressbar, then presumably it's a large file.  :bg  It would be a waste of time programming one for a very small file. You can allocate a buffer half the size of the file if it's not too large, and you'll only get two progress steps, or use a small buffer relative to the filesize and get many steps. A good size of buffer to use is 64Kb or a multiple thereof, as this is matched to the internal file I/O buffers and will process efficiently.

Set up a loop to read in then write out your file in chunks using the buffer. You can divide the filesize by the buffersize to know what percent to advance the progressbar every time you have completed one buffer read or write, but there's absolutely no need to use overlapped I/O for that and a separate thread will be overkill. Just update the progressbar with the new value when you've written the buffer and validated the writefile return, then loop and process another bufferlength. Then you just need to initialise the progressbar when you open or start writing the file, and close it (presumably hiding it) when there are no more bufferlengths to process and you close the file.

To get the fastest speeds using plain Readfile and Writefile, open the file using both the FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_NO_BUFFERING flags. That'll reduce your need to use Overlapped I/O if you don't want to. SEQUENTIAL_SCAN matches what you are doing with the write and the NO_BUFFERING will prevent the OS from rebuffering internally, which just wastes time if you've set your buffer size correctly. Check the restrictions on using that second flag - buffers have to be an exact sector multiple (hence my suggestion for the buffer size). Also remember to use VirtualAlloc to create the buffer, to guarantee that the buffer is aligned correctly on a sector boundary in memory.

EDIT: Using FILE_FLAG_NO_BUFFERING works well for Readfiile using a fixed-size buffer that's a sector multiple, as the requirement is that only a full sector multiple may be read at all times, and reading beyond the end of the file won't cause an error, it just leaves the buffer undefined past the final valid byte. The call returns the number of valid bytes read to you. This can't work quite as simply for Writefile, however, as your file may not be exactly a sector-multiple in length and generally there's a "remainder" less than a bufferlength left to write at the end. The simplest way round this is to not use that flag with Writefile, else if it's a large file you may want to keep the value of using the flag to gain any speedup possible. To do that, close the file after writing the last full buffer, then reopen it without the flag to add the last odd section once you've set the filepointer to the end correctly:

Code Select Expand

invoke  SetFilePointer, filehandle, 0, 0, FILE_END
Or write a whole buffer past the file end then move the filepointer back to truncate the filelength short correctly. Either will work but involves a little more API calling overhead at the end of your loop, so design the loop end to handle this.

IanB