WD external drive

[Queue]

That's because you shouldn't format flash memory to NTFS. =P

Ideally, if you have a small enough flash drive, use FAT16; it's faster and more compatible with arbitrary devices that you can plug flash drives into.

Anything bigger than like 2 GB, use FAT32.

ExFAT is basically there to resolve max file size issues looking forwards, while still being an appropriate file system for flash memory, but compatibility is poor.

Don't use a journaling file system on flash memory; it shortens the lifespan of the memory and is far slower than a simple FAT-based file system.

---

U3 devices are another example where the default software on the emulated CD drive is dumb, but if you can hijack it to put your own software there, the added functionality is great. Now if I could only find a U3 device with a read-only switch.

Queue

[GregL]

That's because you shouldn't format flash memory to NTFS. =P

I have heard arguments both ways.

Anything bigger than like 2 GB, use FAT32

I would say use NTFS.

Generally speaking the performance of NTFS on large volumes is higher than performance of FAT32. NTFS performance on small volumes is lower than performance of FAT/FAT32.  I use large thumb drives with large files. NTFS supports the large (>4GB) files. 

Windows will not allow formatting 32GB or larger drives with FAT32. I have a 32GB and a 64GB drive.

U3 devices are another example where the default software on the emulated CD drive is dumb, but if you can hijack it to put your own software there, the added functionality is great.

I say get rid of that damn U3 crap. I can't stand it.

Dave,

To enable NTFS on your drive, right click My Computer and select Manage. Open the Device Manager and find your USB drive under the Disk Drives heading. Right click the drive and select Properties, then go to the Policies tab and select the "Optimize for performance" option and Click OK.
Once you do that, open up My Computer and right click on the flash drive and select Format. There you will see that you now have the option to format to NTFS in the File System dropdown box.

[Queue]

-_- You're giving seriously bad advice, and what you know about NTFS is only relevant to traditional hard drives, and not storage based on flash memory.

Let me lay out the issues:
- Whenever you write to flash memory, a block of flash memory is erased first, then it can be written to. Erasure is slow; a journaling file system makes more separate writes than a simple file system. It's even worse when it saves last accessed times, as simply reading a file causes a write to occur.
- Optimizing for performance turns on file caching for that drive, which means it's extra unsafe to simply unplug the drive without properly ejecting it as writes don't necessarily occur immediately
- Flash memory has a limited number of writes that any given block of flash memory can withstand before it can no longer be written to. Journaling file systems write a lot more, and particularly are hard on the flash memory due to the journal that's kept. Wasteful writes due to last access times don't help.

Queue

[dedndave]

ya know, Greg, I saw that setting - and dismissed it - lol
it works   :U
tomorrow, i may play with a little comparison
as slow as thumb drives are, i doubt there will be much difference - assuming i can measure it without the affects of caching

[GregL]

Queue,

You still haven't changed my mind. What's the worst case scenario, my drive doesn't last quite as long.

What about SSD drives, are you saying we should format those FAT32 too?  Greater than 32GB you can't.

You do it your way, I'll do it mine.


Dave,

As far as speed, the FAT32 will probably be faster. I like (and need) the other advantages of NTFS.

[Queue]

exFAT was designed precisely for the emerging flash-based hard drives. Microsoft would've loved if everyone threw out FAT and moved entirely to NTFS, but when flash-based hard drives suddenly became more and more common, they realized they needed a non-journaling file system for them and FAT32 was inadequate.

We're not talking about lasting for a little less time, we're talking about orders of magnitude. In the case of running an OS off of a flash-based hard drive, it was performance as much as lifespan.

Queue

[jj2007]

See MSDN for a comparison of file systems.

[GregL]

Well, maybe exFAT is the way to go. I'll give it a try. I had never really looked into exFAT before.

[Queue]

as slow as thumb drives are, i doubt there will be much difference - assuming i can measure it without the affects of caching

After formatting as NTFS, I think you can turn off ''optimize for performance'' to turn off file caching to let you do a direct speed comparison.

With how flash memory works, the entire drive will need to be filled up entirely once before you can get accurate write speed data.
http://en.wikipedia.org/wiki/Write_amplification

Here's a comparison of FAT32, NTFS and exFAT used on flash drives that some nut-job did. It's pretty thorough:
http://www.testfreaks.com/blog/information/usb-flash-drive-comparison-part-2-fat32-vs-ntfs-vs-exfat/

Queue

[dedndave]

i see in his comparison that he times 8 Gb file transfers on FAT32   :P
for these large files, FAT32 is slow
for smaller files (less than the 4 Gb that FAT32 actually supports - lol), FAT32 is a winner (FAT32 and exFAT32 are close)

he also mentions that exFAT is a vista+ thing, but can be made to work for XP
for those who are interested, KB955704 is the hotfix
i don't see support for win 2000, as it is beyond the support life-cycle for this kind of update
it's possible that the XP hotfix would work, i suppose
if not, it doesn't seem like it would be too difficult to extract the files from the hotfix and force it to work - lol

http://support.microsoft.com/kb/955704

[Queue]

He used a 350 MB AVI file for general copy speed testing.

The OPS/MINUTE comparison is specifically tailored to flash storage performance and tests 512 B, 32 KB, 256 KB, 2 MB and 64 MB file sizes.

Queue

[dedndave]

gotcha - i interpreted the graph incorrectly
refering to the 13th graph, it mentions 8Gb on the last 2 tests - that is the volume size - not the file size
that particular brand is just slow with FAT32

[GregL]

Queue,

exFAT is working very well for me on my thumb drives. I'm sold.

[dedndave]

me too   :bg

Code Select Expand

Free Space on an 8 Gb thumb drive:

NTFS  7.42 Gb
FAT32 7.44 Gb
exFAT 7.46 Gb

[Queue]

Just keep in mind the compatibility limitations of exFAT. Would suck to have a demonstration to give and your files are on an exFAT flash drive, but the junker computer you have access to doesn't support it (WinXP without exFAT support, for example). Eventually it'll become ubiquitous though.

NTFS isn't totally worthless for flash drives obviously, if you're purposefully going to make use of some NTFS-specfic features; just for general use it's simply the worst choice. That was really all I wanted to point out.

Anecdotal evidence has all 3 file systems catastrophically failing if a drive is removed while a write is in progress; I've seen no evidence that any one is ''safer'' than another in that regard. You'd think that due to the journaling, NTFS would be safe, but there are cases of NTFS-formatted drives being corrupted to the point where Windows won't recognize the drive as being formatted / having any files. Data is erased in large blocks (256 KB) before being rewritten (including data present in that block that wasn't related to the new data being written); 256 KB is a lot of data if it's file system structure so an erase/write that occurs in the same block as critical file system structure could trash the file system if the rewrite fails (due to the drive being yanked out, for example). Wear leveling can cause the movement of blocks containing critical data (even if you aren't explicitly changing the data in said block) which puts the file system at risk in the same way. My point here is: always tell Windows to eject a flash drive before physically unplugging it.

Queue