File Compare

[P1]

P4 2GHz, W2K-SP4
Reading test file...
Creating comparison files...

filecmp_heapbuffers: 897ms

filecmp_filemapping: 870ms - 97.0%

filecmp_nobuffering: 875ms - 97.5%

filecmp_nobuffering_async: 876ms - 97.7%

Press ENTER to exit...

Regards,  P1  :8)

[mnemonic]

Turion 64 / XP Home (32bit) SP2

Code Select Expand

Reading test file...
Creating comparison files...

filecmp_heapbuffers: 1364ms

filecmp_filemapping: 841ms - 61.7%

filecmp_nobuffering: 840ms - 61.6%

filecmp_nobuffering_async: 834ms - 61.1%


Press ENTER to exit...

[VlasRad]

You forget important.

First, avoid fragmentation - use on defragmented disks, SetFilePointer to allocate entire output size before start WriteFile.

Also, you only compare absolute time? You also should check CPU usage - filemapping might surprise you!

When test buffered approach (heapbuffer, filemap) make sure to test new file or flush fs cache. Note that filemap has no chance be uncached, always will cahce.

[Ian_B]

You forget important.

First, avoid fragmentation - use on defragmented disks, SetFilePointer to allocate entire output size before start WriteFile.

VlasRad, there are many variables. The point of Michael's original test and my followup was to compare the different times of loading a file that a small, normal program might use. Most normal programs have to cope with fragmented disks, they don't defragment before they start reading/writing files! What we have now is something approaching a real-world test with real-world settings, I don't believe for a moment it's perfect, though, and I hope people will develop it for better results.

When test buffered approach (heapbuffer, filemap) make sure to test new file or flush fs cache. Note that filemap has no chance be uncached, always will cahce.

That's why in my test I make sure the copy files are written unbuffered. All 80 test files are new on every run and should be equally uncached for the four tests.

Ian_B

[VlasRad]

When test buffered approach (heapbuffer, filemap) make sure to test new file or flush fs cache. Note that filemap has no chance be uncached, always will cahce.

That's why in my test I make sure the copy files are written unbuffered. All 80 test files are new on every run and should be equally uncached for the four tests.

Good, sorry I not look completely into your test methods :(
Still fragmentation can have big effect punish one method needlessy. If use on defragmented drive and reserve filesize before test (simple SetFilePointer should sufficient and fast on NTFS) raw test be more fair.

VlasRad, there are many variables. The point of Michael's original test and my followup was to compare the different times of loading a file that a small, normal program might use.

Yes raw speed of routine is not only important factor, you need consider environment and how is typically used. Unbuffered+async good for massive file hashing/checksum, but not necisarily best for "normal" use.

Most normal programs have to cope with fragmented disks, they don't defragment before they start reading/writing files!

Yes but if you want measure speed of actual routines and not of fragmented filesystem, you need defragment file. Of course it might interesting to test routines on fragmented files but you need exactly same fragmented file then, and to not filesystem cache.

What we have now is something approaching a real-world test with real-world settings, I don't believe for a moment it's perfect, though, and I hope people will develop it for better results.

Please do the test of CPU usage while reading file, it is other important test factor. In the end you should really get similar task completion speeds because task I/O bound by slow hard drive, CPU usage is not bounded like that. Page faults expensive, and you get many on using mapped file.

[dsouza123]

Will these various I/O routines work with a RAM Disk ? (such as RAMDisk from Cenatek)
If so how will the routines perform and what would affect performance ?

I don't have one so I couldn't test it.

There are also hardware memory based drives, such as the Rocket Drive also from Cenatek,
and the i-RAM from Gigabyte, that are PCI cards with sticks of RAM to hold the data.

[VlasRad]

I see no reason they notwork with ram disk or i-ram, but performance difference hard to measure then, likely.

But cpu usage might be more interesting with that kind drives since not I/O bound.

[sixleafclover]

Errm if you cared...

correct return vals: 1 0 0 0 -1 -1 1
filecmp_heapbuffers: 1 0 0 0 -1 -1 -1
filecmp_filemapping: 1 0 0 0 -1 -1 -1

filecmp_heapbuffers: 1ms
filecmp_filemapping: 1ms

100.0%

Press any key to exit...

seems to be no difference on my system.

2.8ghz opteron 144, 250gb maxline3 HD, xp sp2

[paranoidx]

filecmp4, P4 2.8 HT/160 HDD seagate/512 RAM

Code Select Expand


                            1st run            2nd run            3rd run
correct return vals:        1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1

filecmp_heapbuffers:        1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1
filecmp_filemapping:        1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1    1 0 0 0 -1 -1 1

filecmp_nobuffering:        x x x x  x  x 1    x x x x  x  x 1    x x x x  x  x 1

filecmp_nobuffering_async:  x x x x  x  x 1    x x x x  x  x 1    x x x x  x  x 1

filecmp_heapbuffers:        623ms              662ms              632ms
filecmp_filemapping:        307ms              306ms              279ms

                            49.3%              46.2%              44.1%

filecmp_nobuffering:        6524ms             8434ms             5928ms

filecmp_nobuffering_async:  12202ms            13022ms            9108ms


After the 1st/2nd time I ran Filecmp4, my comp struggles to revive and lags terribly for about 10secs.

3rd Run: After I switch Avast AV/Tiny personal FW OFF