CreateFile flags

[bomz]

   invoke CreateFile, addr TimeHeader, GENERIC_READ or GENERIC_WRITE, FILE_SHARE_READ,\
   NULL,CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,NULL
................................................................................................
   mov hFailLog, eax

   invoke CreateFile, addr TimeHeader1, GENERIC_READ or GENERIC_WRITE, FILE_SHARE_READ,\
   NULL,CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,NULL
................................................................................................
   mov hOKLog, eax
................................................................................................
.................................................................................................
   invoke WriteFile, hFailLog, addr HTTP, edi, addr bytesWrite,NULL
.................................................................................................

My application create and write two logs. But if I run simultaneously 6 or more copies of it - this slow down system read-write access to disk (each write ~100 byte to log each 1-3 sec). I try different flags - any result.  What may be a reason of this?
Also I try make LOG from the begining max possible size to prevent fragmentation - any result

[baltoro]

Hi, BOMZ,
I've done similar types of operations, writing application log files,...in both C++, and assembly language. I don't think it's the Create Flags (you are using the normal flags),...and, I'm assuming that you are not executing this code remotely, and that access security is not a problem. I have found that these things operate quicker and more reliably if you close the file handle after each separate write operation. The only inconvenience is that you must call CreatreFile again for the next write operation, using the OPEN_EXISTING flag, and, maybe a current fille pointer, if necessary.

You, no doubt, have read the MSDN documentation: CreateFile, MSDN, especially, the section about: Caching Behavior
Also, interesting is this explanation of:Windows File Buffering, and, File Caching

By default, Windows caches file data that is read from disks and written to disks. This implies that read operations read file data from an area in system memory known as the system file cache, rather than from the physical disk. Correspondingly, write operations write file data to the system file cache rather than to the disk, and this type of cache is referred to as a write-back cache. Caching is managed per file object.
The amount of I/O performance improvement that file data caching offers depends on the size of the file data block being read or written. When large blocks of file data are read and written, it is more likely that disk reads and writes will be necessary to finish the I/O operation. I/O performance will be increasingly impaired as more of this kind of I/O operation occurs.

When an application is finished using the object handle returned by CreateFile, use the CloseHandle function to close the handle. This not only frees up system resources, but can have wider influence on things like sharing the file or device and committing data to disk. Specifics are noted within this topic as appropriate.

[bomz]

Thanks. I think about this, but I want do correct. strange. in dos was FILESHIGH=10 as you may be remmember

I do program for fast copy files using RAM buffer. I need with command promt. There are two program for this TeraCopy and FastCopy. I try all flags in all combination, and do that my program copy 5% faster than TeraCopy, but 5% slowly than Japan FastCopy and I don't understand how he do this.
And what about download manager which download files in 10 Threads? they close files each 64 kb? or if this 10 handels for 1 files or may be they use 1 handle???

[qWord]

you will get best results using filemapping for copying. Also the size of the buffer is important: to small or to huge buffers slow down the mapping process - a test on your system should make clear what the optimal mapping size is.

[bomz]

what is this - filemapping?

the best buffer on all computers 32 mb, and there is no sence to change it. because some good flags work only with 32 mb.
My "secret" how I do 5% faster than TeraCopy - I don't create file from the very begining in full size. This gives good result in dos as for windows

[baltoro]

File Mapping, MSDN

[mineiro]

invoke CreateFile,  esi,GENERIC_READ ,0,0,OPEN_ALWAYS,FILE_FLAG_SEQUENTIAL_SCAN,0

[bomz]

[qWord]

the best buffer on all computers 32 mb, and there is no sence to change it. because some good flags work only with 32 mb.

Ah! - good to know.

[bomz]

invoke CreateFile,  esi,GENERIC_READ ,0,0,OPEN_ALWAYS,FILE_FLAG_SEQUENTIAL_SCAN,0

Program work quickly

[jj2007]

Gamasutra says it's all useless...

[bomz]

I run now 8 copies of program all is OK.

[jj2007]

Mysteries... :green2

Code Select Expand

Normal SeqRead
5844    6063 ms, 800000000 bytes read
5515    5735 ms, 800000000 bytes read
5406    4859 ms, 800000000 bytes read
4422    4422 ms, 800000000 bytes read
5047    5578 ms, 800000000 bytes read
5031    4516 ms, 800000000 bytes read
4391    4375 ms, 800000000 bytes read
4375    4766 ms, 800000000 bytes read
5859    5469 ms, 800000000 bytes read
5672    5797 ms, 800000000 bytes read
5718    5500 ms, 800000000 bytes read
5500    5500 ms, 800000000 bytes read
5313    4484 ms, 800000000 bytes read
4375    4375 ms, 800000000 bytes read
4359    4391 ms, 800000000 bytes read
4375    4391 ms, 800000000 bytes read
4390    4375 ms, 800000000 bytes read
4375    4375 ms, 800000000 bytes read
4375    4375 ms, 800000000 bytes read
5859    5610 ms, 800000000 bytes read
5562    5766 ms, 800000000 bytes read
That is a big improvement, almost 1%.

P.S.: Attachment requires MB and a file with exactly 80,000,000 bytes

[bomz]

we are talking not about that .

even if you use FILE_FLAG_NO_BUFFERING file leave in pagefile. for your competition you must use each time another file with same size. speed of copying from pagefile two time faster. fragmentation influence. if you need copy 1000 G, 1% = 10G. if you have sata2 - 300 mbit there is no difference, but if you have IDE 33 mbit....

FastCopy. code hear C++
http://ipmsg.org/tools/fastcopy.html.en

[jj2007]

bomz,

My testbed uses the same file and reads it alternating with either method. That is the only sound approach. Result is that the improvement is almost zero - no surprise, actually, otherwise you would find many more Google hits for this "magic" flag.

Read the Gamasutra page above. They have tested it extensively and come to exactly the same conclusion: It doesn't matter. Even File Memory Mapping, hobby horse of many coders, does not perform significantly better. Get used to it: the normal routines are already optimised. 640 MB/second is not that bad, actually...