Ram Defragmenter

[cman]

Thanks for the code PBrennick !  :U

[hutch--]

To address a couple of issues about the actual memory range available the output of the toy I posted does this.

Code Select Expand


Block allocated at 4259872 memory address
Block allocated at 71434272 memory address
Block allocated at 138608672 memory address
Block allocated at 205783072 memory address
Block allocated at 272957472 memory address
Block allocated at 340131872 memory address
Block allocated at 407306272 memory address
Block allocated at 474480672 memory address
Block allocated at 541655072 memory address
Block allocated at 608829472 memory address
Block allocated at 676003872 memory address
Block allocated at 743178272 memory address
Block allocated at 810352672 memory address
Block allocated at 877527072 memory address
Block allocated at 944701472 memory address
Block allocated at 1011875872 memory address
Block allocated at 1079050272 memory address
Block allocated at 1146224672 memory address
Block allocated at 1213399072 memory address
Block allocated at 1280573472 memory address
Block allocated at 1347747872 memory address
Block allocated at 1414922272 memory address
Block allocated at 1482096672 memory address
Block allocated at 1549271072 memory address
Block allocated at 1616445472 memory address
Block allocated at 1683619872 memory address
Block allocated at 1750794272 memory address
Block allocated at 1817968672 memory address
Block allocated at 1885143072 memory address
Block allocated at 2009530400 memory address
Press any key to continue ...


The addresses make the point, memory is allocated in most instances at the lowest address available and the next allocation is the next lowest address available.

For application that are messy in how they allocate and free memory, Microsoft introduced the Low Fragmentation Heap memory strategy to address applications/programmers that don't accurately control how memory is used. From memory the OS also provides a strategy that allocates memory at the highest address available, apparently to try and avoid a fragmented mess at lower addresses. I can't find the article in a hurry but recent design of Microsoft servers addressed the memory fragmentation problem by controlling their own memory pool and tracking when a connection was closed so the SAME MEMORY ADDRESS was reused to avoid ever more fragmentation. The technique was something like a circular buffer and each new connection was placed in the memory hole left by the last connection that had closed.

There is a mountain of reference material on Google that addresses the problems of memory fragmentation and it is an old problem that has been around for years, the older techniques were called memory compaction.

[jj2007]

the SAME MEMORY ADDRESS was reused to avoid ever more fragmentation. The technique was something like a circular buffer and each new connection was placed in the memory hole left by the last connection that had closed.

That's exactly the technique I use in MasmBasic for the Print macro and a number of other purposes. Much faster than HeapAlloc'ating every time a new string, and fragmentation-proof, but you are limited by the size of the buffer and need to maintain a "rotating" pointer.

[PBrennick]

JJ, I am curious. Why do you feel there is a limit on the size of the buffer? For example, my editor is using a 4mb buffer.

Paul

[jj2007]

JJ, I am curious. Why do you feel there is a limit on the size of the buffer? For example, my editor is using a 4mb buffer.

Paul,
You are right, the limit is more theoretical. Currently I use 640000 bytes allocated in the .data? section with this well-known technique:

Code Select Expand

MbBufferStart LABEL DWORD
ORG $+MbBufSize-1+16 ; MbBufSize plus one para of 'airbag'
db ?

That is more than generous for printing, but it has of course a theoretical limit. If a user ever decides to print more than 640k to the console (or to a pipe), a runtime error message will pop up. Anyway, I don't see a compelling argument to allocate more for this "common buffer". Ordinary strings and arrays created with Dim use HeapAlloc and have no limit other than available memory.

Regards,
Jochen

[PBrennick]

I agree that you are using a more than sufficient size. My buffer is larger only because it is a general use buffer that serves multiple purposes. Printing in the normal sense is done by use of a que (spool) set up by the system so you are actually creating a bottleneck that generates an error on overflow. You are actually restricting what is actually available instead of making use of available space. In other words, your reserved space is smaller than the que space. Are you aware of this? You could probably increase the capability of your Print function by spooling your own buffer from within your editor if the need is detected by continually dumping to the same buffer when you receive a signal that you can do so. RichEdit does this automatically for you which is why I only need 4mb whereas I can print up to 4gb. Don't think that will ever happen, though, because both of us are only dealing with text files. You need to be aware, though, that printing in RTF style carries a much larger penalty than just the text.

Paul

[jj2007]

Printing in the normal sense is done by use of a que (spool) set up by the system

Paul,
There is a misunderstanding: I mean printing as used here in the forum, i.e. send a string to the console:

Print Str$("\nConverting %i hex equates in Windows.inc to decimals", MyCounter), Str$(" took %i ms\n", 0-MyTimer)

The same algo is used for assigning a string:

Let L$(n)=Left$(L$(n), pos-1)+Str$("EQU %i", esi)+Mid$(L$(n), posAfter)

In both cases, the problem is that the argument macros usually return eax before the Print or Let macros start acting, i.e. inside the Let macro you get Let eax=eax+eax+eax - very funny ::)
Now in the case of the Print macro, I let the argument macros write to a circular buffer, where I can pick them inside the Print macro by counting backwards. The technique is tricky but fast.

[PBrennick]

I understand. Programming can be fun or funny. ::)

Paul

[ThexDarksider]

I understand. Programming can be fun or funny. ::)

Paul

LoL that's so true, you all have very interesting points here, I'd like to see more posts about this, this is getting really interesting! ^^

~

On another note, I see that people say that OS > guest apps, that's not always true though, you can always hack up ring0. :P

[hutch--]

Hmmmm,

> On another note, I see that people say that OS > guest apps, that's not always true though, you can always hack up ring0.

Tread carefully here, we would hate to see you disappear forever.  :green

[ThexDarksider]

Hmmmm,

> On another note, I see that people say that OS > guest apps, that's not always true though, you can always hack up ring0.

Tread carefully here, we would hate to see you disappear forever.  :green

Lol I don't even know how to do it. :toothy But I've been reading lots about it from a VM security article, it's amazing what assembly can do...