Ram Defragmenter

[slovach]

They are a scam.

RAM access time is likely in nanoseconds, whereas a harddrive is in milliseconds. Random being the keyword that it can retrieve from anywhere with no penalty, as 'fragmented' as the data becomes, it does not matter. Access time remains constant.

Older versions of windows will probably move idle / minimized programs out to save memory, Vista and W7 do the smarter thing and make use of it though. Free ram is unused ram. RAM is blazingly fast compared to your HD and the OS will free space when something else requests it.

[hutch--]

slovach,

What you have said here is nonsense. Allocate a gigabyte of memory and reference sequential addresses more than the processor cache size apart and watch how slow it is. Its called page thrashing and it is based on how the hardware works, not gizmos in the operating system.

Here is a rough toy that was similar to what I used to use in win9x. For what its worth it does not seem to matter on the XP I am working on which is an NT based system.

Code Select Expand


; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
    include \masm32\include\masm32rt.inc
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

comment * -----------------------------------------------------
                        Build this  template with
                       "CONSOLE ASSEMBLE AND LINK"
        ----------------------------------------------------- *

    .code

start:
   
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    call main
    inkey
    exit

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

main proc

    LOCAL pfree     :DWORD
    LOCAL mst       :MEMORYSTATUS
    LOCAL msiz      :DWORD
    LOCAL hArr[128] :DWORD
    LOCAL hcnt      :DWORD

    push ebx
    push esi
    mov hcnt, 0

    lea esi, hArr

  @@:
    mov [esi], alloc(1024*1024*64)              ; allocate 64 meg block
    test eax, eax                               ; exit loop when it fails
    jz @F
    mov ebx, [esi]
    print "Block allocated at "
    print ustr$(ebx)," memory address",13,10    ; display the address
    add esi, 4
    add hcnt, 1
    jmp @B

  @@:
    invoke SleepEx,1000,0                       ; wait a second


  @@:
    free [esi]                                  ; loop down the handle array
    sub esi, 4                                  ; freeing each memory block
    sub hcnt, 1
    jnz @B

    pop esi
    pop ebx

    ret

main endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

end start


[slovach]

I tried just that and my system yawned at me in return.

My understanding is:
Obviously if you end up off the multiples, you'll suffer unaligned horrors. In theory, say you were to access 0-7 then 64-71, the access would be fast and aligned.

[hutch--]

You have missed the obvious, page load time, if you have ever had to write code that addresses location spread over a large address range you will know what PAGE THRASHING is about.

> I tried just that and my system yawned at me in return.

So does mine with 4 gig on 32 bit XP but someone may find a use for it.

[slovach]

And if you have more memory available than the working set requires?

I can only see this becomming less and less problematic. The last time I bought memory, I got a set of 4gb set of Corsair for $15 after rebate. At the ridiculous rate it's currently going, I'm pretty sure I'll end up with a previously seen as ludicrous quantity the next time I upgrade. I could always just make it into a RAM disk.

[sinsi]

You can't defrag RAM - windows allocates virtual memory from its pool of physical ram. Your memory at virtual address 00400000h could be any physical address, and the
page after it (virtually) might not be physically the next page.

[hutch--]

This fascinates me, big servers suffer the problem all the time and more recent server software has had to develop strategies to solve the problem. While each app has memory mapped to 400000h the operating system handles ALL of the physical memory and one of the problems of OS level memory management is block of memory that are deallocated inbetween other blocks. Think of an allocation scheme where you have a total of a gigabyte of memory allocate in 64k blocks, then delete every alternate block and you have a gigabyte of address space with half of it as 64k holes. This is MEMORY FRAGMENTATION. Then as extra programs allocate memory the OS has to start using those holes so you do not get contiguous reads but stepped reads from one fragment to another.

Web servers do this type of memory allocation regularly, a connection is made, memory is allocated for it, another connection is closed so the memory is freed. Multiply this by a year running and you will understand why servers must have a memory strategy that restricts memory fragmentation.

[sinsi]

From what I've read, every memory access is checked by the processor, so it doesn't matter if the physical pages are contiguous or not. The problem comes from
the steps the processor goes through to check, the first step is the TLB. If the page is cached here, there isn't a penalty but if it isn't the next steps are expensive
in terms of the time taken.

[redskull]

Think of an allocation scheme where you have a total of a gigabyte of memory allocate in 64k blocks, then delete every alternate block and you have a gigabyte of address space with half of it as 64k holes.

Why can't each of the 64K holes be reused and remapped as another half-gig of continuous space for another process (at least, under x86)?  Every address is translated via a PTE, every time, no exceptions, so why wouldn't Windows just remap a continuous *virtual* address space to the discontinuous physical pages?  The page thrashing meltdown comes from reading and writing pages back and forth *from disk*, not from jumping around between pages in RAM.  That's a function of the NUMBER of pages you need in memory, not the location in RAM where they're put.

Also, @jj, i would assume reserved memory has no effect on anything.  IIRC memory that's not committed has no pages (or PTE's) setup, so there would be nothing to page out at all.

[hutch--]

 :bg

> Why can't each of the 64K holes be reused and remapped ....

It can be and it is but the price is memory fragmentation, that is why server software that runs continuously with endless allocations and deallocations have special purpose memory managers to control the reuse of memory.

[redskull]

What makes you able to assume that the originally allocated 1GB of memory would be contingious physical pages in the first place?  The only way you control that is to actually be the one who makes the PTE's (and/or VDA's), which I doubt Windows would allow willingly.  Considering you can't avoid having to translate an address whether the pages are sequential or not, I still just don't see where any speed gain would be.

-R

[PBrennick]

The memory that is reserved IS technically available as just said so to be specific, all that is probably being done is to force windows to do a clean up and release of reserved memory. It should be noted, reserved memory is only done when a running program (or process) reserves and then releases memory while it is still running. Once the program exits, windows releases it. This means you can probably just close all programs and get a gain. The onlly potential gain you will not see would be from active processes not involved with programs started by the user that manipulate the memory pool. Older versions of windows were not very good at doing this type of clean up, though, so I still favor Hutch's solution. Also, I do not much trust even the most recent versions of windows.



Paul

[PBrennick]

Oh, and thanks for the little program Hutch. I plan to make use of it.

Paul

[cman]

I'll second that, thanks for the code Hutch!  :U Is there an api for reporting the amount of free RAM of your system? I'd like to see the effects of running the above code. Thanks for any information.

[PBrennick]

cman,

About your question, I am attaching a program written by John Lyons that shows memory usage. I have not the time to look at it right now but I am sure it will give you the answer you need.

Paul