memory allocator project ...

[hutch--]

James,

It may help in the design of what you are after I you gave everyone some idea of what you need in a memory manager. You have classic problems like small repeatedly allocated and de-allocated blocks of memory that can leave holes all over the place, various theories of garbage collection to rewrite the fragmented remains back into more contiguous blocks and a whole host of other ideas to deal with well known problems. Ultrano did a memory allocation scheme that was very effective on badly fragmented small allocations and it may be worth you having a look at it.

Just as an example of using system resources where you can, recently i wrote a hash table in another language that needed a very large count without massive fixed memory blocks to support it. I used OLE string memory because of its characteristic of being able to allocate an empty handle then later reallocate the handle to a variable length of string. The design is very flexible in that it can handle variable length strings for every table member and it only uses 4 bytes for the handle if nothing gets pointed at that particular hash bucket. I could regularly allocate 1 million hash buckets which is 4 meg of memory and only use the additional memory for the data I wrote to the hash table.

[James Ladd]

Hutch,

I'll look into the word done by Ultrano.

I think I have been specific in that I want a framework that can do alloc/free and
handle allocations with a minimum of overhead and handle re-use of fragmented
free memory in an efficient way, in terms of both speed and size.

The most important thing is to be able to measure/see the allocation overhead
and fragmentation. So that different allocators can be tried. Different allocators
in terms of a high level not a low level like VirtualAlloc/HeapAlloc.

However, the framework should be easily changable to see the effects of
using different OS level allocators and their effect on the allocation schemes
which really should be 'none' given the manager should be handling the
blocks given to it from the OS. Although different policies could circumvent this.

Its the measuring that is important to me.

[stanhebben]

I'm trying to write my own memory allocator. When it's ready, I'll publish it under the MIT license.

The allocator will be suitable in cases where you perform a lot of allocations/deallocations. It uses VirtualAlloc to allocate large chunks, and splits them in smaller ones as requested. It is relatively complex, but should be fast with minimal fragmentation.

Downside: it currently uses about 2MB of memory to manage the allocated/freed chunks. (though I'll try to improve that)

Stan

[James Ladd]

Stan,

I look forward to seeing your work.
I guess 2MB is a large overhead for some situations.

How do you plan on being able to show people the effectiveness of the allocator?

Rgs, James.

[stanhebben]

How do you plan on being able to show people the effectiveness of the allocator?

Some benchmarks and some practical situations, probably. Care for later - I'll finish it first.

Stan

[stanhebben]

Just wanted to let you guys know that I created something which is much faster than HeapAlloc/LocalAlloc in most cases. But I'm now rewriting it for even higher speeds.