Fast File Search

[ecube]

Would multiple threads help file searching go faster? i'm using findfirstfile etc... also is that the fastest way to search for files?

[hutch--]

Cube,

Probably not as the file system is hardware and an increase in thread count will not give you any more acess. Most do file searches by a recursive algo but this technique is limited by the speed of the hardware used in the file system.

[redskull]

Since a thread executes on a single CPU, you would certainly need a MP system to have any effect; if you memory-mapped the files, got the cache manager to keep a large section of the file resident in memory, and had each thread working a seperate part of the file, then I could foresee a speedup.  Creating that environment (and coordinating the efforts of the threads) would be the tricky part.

-r

[joemc]

when using findfirstfile it seems window does create a cache of this (may be related to if indexing service is on)  because when go through the whole HD after a reboot and write all of its contents to a character array it takes about 5 minutes for my setup.  but if i run it again it takes about 2 seconds to run the same loop and get the same results.

The reason i mention it is i am not sure what your final goal is, but if you run through the whole filesystem one time at program (or system) startup, from then on it is very quick.

This is of course just my own experience, i have no idea why or how it works.

[greco558]

I noticed the same thing in one of my programs. I use findfile and findnext and recursively search.  It searches the hard drive for  files I am looking for lists them and the folder they are in. The first time I run the program after a reboot it takes  several seconds to search entire drive. Any search after that takes less then a second.  I have been looking at my program trying to figure out why that happens.   

[Astro]

Hi,

Unless you are performing direct disk I/O in your code and bypassing Windows entirely, you will find Windows will be caching the data when you access it.

Best regards,
Robin.