Memory management in drivers

[TNick]

Hello, friends!

At this moment I am researching about drivers, because I need to convert an app into such a virtual driver. My first problem is that I have no idea which method should I use for a specific purpose. For example, I have a 2*(memory page size) buffer that should be allocated (?) when my driver is being started (and freed when it is "deleted"/closed). On the other hand, I need buffers like 8*(memory page size) to be created when a particular event arrives (and closed in same manner). In my app, I did this by using VirtualAlloc to reserve the range that I need and, then, I commit memory as needed. Is there something similar for drivers, too?

I will study what google provide me, along with DDK documentation, but 'till now I couldn't find any good answer to my problem.

Some pages that i've read:
http://www.microsoft.com/whdc/driver/kernel/mem-mgmt.mspx
http://www.osronline.com/ddkx/appendix/enhancements5_3oc3.htm - Support of Giant Drivers Due to Increased Number of System PTEs (this may help me?)

Thanks,
Nick

[ramguru]

From my little, almost no experience in DDDevelopment I can say that you're probably supposed to use one of these for memory allocation:
ExAllocatePoolWithTagPriority
ExAllocatePoolWithTag
ExAllocatePoolWithQuotaTag
ExAllocatePoolWithQuota
ExAllocatePool
ExAllocateFromPagedLookasideList

from ntoskrnl.lib

[TNick]

First of all, thanks for your reply, ramguru!  :clap:
Second, I must admit I wasn't clear enough :red, so let me undo this mistake.
I did read a lot before posting. Only straight forward documentation that I've found was DDK. In it's help file, in

General Driver Information -> Reference -> Driver Support Routines -> Summary of Kernel Mode Support Routines -> Memory

we are informed about "Allocate long-term internal driver buffers". Following the link, we told that we should use:

MmAllocateContiguousMemory
MmAllocateNonCachedMemory
MmAllocateMappingAddress
AllocateCommonBuffer

to allocate memory, and

MmFreeContiguousMemory
MmFreeNonCachedMemory
MmFreeMappingAddress 
FreeCommonBuffer

to free it.

Now:

MmAllocateContiguousMemory can be called to allocate a contiguous block of physical memory for a long-term internal buffer...
A device driver that must use contiguous memory should allocate only what it needs during driver initialization because nonpaged pool is likely to become fragmented as the system runs

MmAllocateNonCachedMemory can be called from a DriverEntry routine to allocate a noncached block of virtual memory for various device-specific buffers.
A device driver that must use noncached memory should allocate only what it needs during driver initialization because nonpaged pool is likely to become fragmented as the system runs

MmAllocateMappingAddress reserves the specified range of memory for use by the caller. No physical memory is allocated for the virtual address range and the virtual memory cannot be accessed until it is mapped by the MmMapLockedPagesWithReservedMapping routine

AllocateCommonBuffer Allocates and maps a logically contiguous region of memory that is simultaneously accessible both from the processor and from a device

Excepting
AllocateCommonBuffer, all other methods look interesting. But which should I use?

And, by the way, which memory is "cached" and which is not?

Waiting for your help, guys ::). Thanks!
Nick

[later]

noncached memory is a scarce system resource and allocation of such memory should be limited

so, now, we only have to decide between MmAllocateContiguousMemory and MmAllocateMappingAddress.
[/later]

[Tedd]

I think it would depends very much on what you're using the memory buffers for - that's why there are different options.
When working with devices, you need to make sure they have access to the memory they're reading/writing to, which usually means locked and contiguous, at least for the duration of the operation.
So it depends on what the requirements of the device are. Of course, you'll need some memory for your driver, and then probably a larger buffer to use for exchanging data with the device?

[P1]

The best driver designs don't do anything the program can do for itself.  Including memory management, if you can design it that way.

Regards,  P1   :8)

[TNick]

Hello and thanks for your replies!!

No, I can't do memory management in program. This is a .. database driver. I intend to avoid the buffering system that Windows provides for files, because my tests showed  increase in speed using my own algo, when talking about random access. I now want to implement this in a driver. So, when a file is opened, I need about 8 pages of virtual space allocated, and pages according to settings.

I will try to play with these two methods that I was talking about and will see what I get (not too many BSOD, I hope :)

Nick

[Mark Jones]

Also I'm not too sure how well this would work or not, but consider installing and running your development driver on a BOCHS or Virtual Machine? :bg

[SystemError]

Hi!
I see the last post was some time ago; the topic seems interesting, what was the conclusion, after all?

[TNick]

The conclusion was "Read for your self, understand, test, compare, and, then, share the results" :)

Nick