Slow Writes.

[KeepingRealBusy]

Ok, I'm at a loss. I have a program that reads, writes, and compares files. The files are 256 MB files. The reads take about 250 msec, the compares about the same (using XMM regs), the writes open the files as OPEN_EXISTING with GENERIC_READ|GENERIC_WRITE (the files are all pre-created, copies of a 256 MB file, to avoid creation time), but it is taking 4 to 6 seconds to write the files, not 250 msec. The files are on a non-compressed NTFS partition, XP Pro SP2 system.

Is there any way to write a file quickly? Is this just the "nature of the beast", or is the system trying to verify? Is there any way to stop such a verify? If I could write it out in 250 msec, read it back in 250 msec, and compare it in 250 msec, then the total would be just 750 msec and not 4 to 6 seconds.

Dave.

[Tedd]

Writing to disk will inevitably take longer than reading from it - writing requires a physical alteration of the disk surface, whereas reading just detects what's already there.
I'm not sure whether the differences are that great though, so there may be some other things happening (like verification, etc.) Also, large amounts of the disk could be already cached, so reading from those will be considerably faster.
It's worth playing with the flags to CreateFile as they allow you to do things like write without buffering - which will result in the data getting to the disk more quickly, but might be slower overall if you're then reading the file again; so you need to consider how these things affect each other. Another issue - you'll get faster results if the data is aligned on a sector-size boundary, and is of a sector-size multiple length.

[Mark Jones]

Also consider that "reading" a 256MB file in 250mS is 1GB/s throughput -- this is an impossible sustainable data rate except for specialized RAID arrays, etc. So either this data is cached in RAM, or the I/O routines are working asynchronously (and the delay is noticed in the write routine because all pending I/O operations must complete first...)

Overall, a 256MB file written in 6 seconds sounds reasonable on a standard IDE or SATA drive today. (That's 42.67MB/s, much closer to the actual hardware limitations.)

[Rockoon]

Most RAID0's cannot even do 1gb/sec regardless of how many drives are in it.

From here:

http://www.nextlevelhardware.com/storage/battleship/

You see that this guy had a real problem finding a raid controller that can handle such massive throughput. He perchased a very high end raid controller and was still topped out at 800MB/sec.

Unless you have one of the new high end SSD's, your single drive read rates will most certainly be under 100MB/sec

[donkey]

Overall, a 256MB file written in 6 seconds sounds reasonable on a standard IDE or SATA drive today. (That's 42.67MB/s, much closer to the actual hardware limitations.)

Actually that's quite good, my drive tops out at about 40 MB/s for sequential writes much less when I am performing random writes, the average throughput is only around 35 MB/s...

Benchmark Results

Drive Index : 35MB/s

Performance Test Status

Run ID : ************** on 07 January 2009 at 20:30:27

SMP Test : No
Total Test Threads : 1
SMT Test : No
Dynamic MP/MT Load Balance : No
Processor Affinity : No
Operating System Disk Cache Used : No
Use Overlapped I/O : Yes
IO Queue Depth : 4 request(s)
Test File Size : 1023MB
File Server Optimised : No

Benchmark Breakdown

Buffered Read : 93 MB/s
Sequential Read : 40 MB/s
Random Read : 30 MB/s
Buffered Write : 41 MB/s
Sequential Write : 40 MB/s
Random Write : 24 MB/s
Average Access Time : 9 ms (estimated)

Drive

Drive Type : Hard Disk
Total Size : 140GB
Free Space : 76GB, 100%

[Rockoon]

Using HD-Tune

my 200MB drive:  60MB/sec peak, 42.5MB/sec average, 14.9ms seek
my 40GB drive: 40MB/sec peak, 33.8MB/sec average, 12.7ms seek

I think its time for me to upgrade to an SSD, some of the lower cost ones ($2/GB or less) now pump up to 155MB/sec

[KeepingRealBusy]

I think that the quick read may be due to caching, and maybe the slow write may partially be due to indexing. Buffering may also be partly to blame. The file is exactly 256 MB, and on page boundaries (VirtualAlloc for the buffer).  I'll try to play around with the CreateFile options and see what happens.

Thanks to all who replied. I come back with any results that I find.

[MichaelW]

It seems to me that the system should cache reads and writes. At least normally and for a local HDD, and assuming that the system has a sufficient amount of physical memory, once the data is written to the cache, from the POV of the application the operation should be complete. What is the size of your system cache and how do the read and write times compare with smaller files?

[vanjast]

..Actually that's quite good..

My thoughts eggsactly...

Wow!! you are a demanding person...

:bg :green2

[Mark Jones]

Assembly is a demanding career/profession/hobby. :bg

(Seriously, if the hardware were capable of 100MB/s continuous but the provided I/O routines limited this to 40MB/s, then it makes perfect sense that one should question every bottleneck.)

Michael is right on the money also, by the way - caching is an important factor to consider. From my experience (Windows), read caches are commonplace, however write caches are more restricted/limited/non-existant. So it sounds entirely plausable that the read functions return immediately because they are entirely cached, where the write functions lag because they are not cached or use a very small cache. In terms of the disk itself, most older drives have 4 or 8MB onboard cache, while even the newest ones typically have less than 32MB.

Of course a definitive simple test would be to use a random number generator to write a gig or so of data to a contiguous block of the disk, reboot, then read it all back. At HIGH_PRIORITY_CLASS, this should paint a pretty accurate picture of the maximum sustainable system throughput. Of course there are many other factors to consider, but if you just want a solid number, this would work for that particular set-up.

[Jackal]

Using HD-Tune 2.55

I run 2 750GB drives in raid0

Max was 207.6MB/sec
Min was 87.8MB/sec
Average was 153.7MB/sec
11.1MS access rate
134.3MB/sec Burst rate

I have a 500GB not in any raid array also which benchmarked

Max was 76.9MB/sec
Min was 38MB/sec
Average was 63.3MB/sec
13.5MS access rate
163.5MB/sec Burst rate

I too wish i could write faster at times but oh well.. Maybe i should align it as mentioned and see how that goes.

[KeepingRealBusy]

Jackal,

Thank you for the program name HD Tune 2.55 (thank you Google). I downloaded this and tried it on my laptop. Disgusting. I will move it over to my desktop and report what I find later. On the desktop I can see that I can write 256 megabyte files in 4.6 sec, 4.3 sec , 4.7 sec. If I boot the system, the first read is about the same, thereafter it is .203 or .204 sec, I guess because of read caching.

Dave.