Memory manager (allocator)

[z941998]

Hi All,

I created a procedure to perform memory management for my applications.  It works great for allocating up to 64K.  Getting above 64K is my issue as I have a few items that need about 256K.

I have read several posts and have gone thru the SDK stuff on allocation.  Nothing is referenced on the limits of allocation, even with masm32 the three types of allocation do not warn us about 64K limits.

Any suggestions, below is the code I am using:

.data
;                         Addr,  Eq,  Recs, Elem,   Size, dflag
    nMemQty            dd   55
    Page1_System     dd    0,   0, 60000,    1,  60000, 0
    Environment        dd    0,   1,  8192,    1,   8192, 0
    Tbl_Schedules      dd    0,   2,  2500,   10,  25000, 0

; ------------------------------
    Struct_GA STRUCT
        M_Addr   DWORD ?
        GA_EQ    DWORD ?
        GA_Recs  DWORD ?
        GA_Elem  DWORD ?
        GA_Size  DWORD ?
        GA_Dflag DWORD ?
    Struct_GA ENDS
; ------------------------------

Process_Memory_Setup proc
    LOCAL gSize :DWORD

    push esi
    push edi
    push eax
    push ebx
    push ecx
    push edx
    mov ecx, nMemQty
    lea esi, Page1_System
again:
    push ecx
    mov eax, (Struct_GA PTR [esi]).M_Addr    ; Memory location address
    cmp eax, 0                               ; See if already allocated, 0 means no allocation done
    jne @F
    mov eax, (Struct_GA PTR [esi]).GA_Size   ; Needed memory
    mov gSize, eax                           ; Temp save
    push esi
    invoke SysAllocStringByteLen, 0, eax     ; Allocate Memory
    pop esi
    mov (Struct_GA PTR [esi]).M_Addr, eax    ; Save memory location
    push esi
    invoke memfill, eax, gSize, 0            ; initialize memory to zero's
    pop esi
    @@:
    pop ecx
    add esi, 24
    dec ecx
    jnz again
    lea esi, Page1_System
    mov eax, (Struct_GA PTR [esi]).M_Addr    ; Memory location
    mov page1_ptr, eax
    pop edx
    pop ecx
    pop ebx
    pop eax
    pop edi
    pop esi
    ret
Process_Memory_Setup endp

[BlackVortex]

What's wrong with VirtualAlloc ?

[hutch--]

z,

I don't know what is going on with your code, the OLE string memory API you have used easily allocates 250 MEG, GlobalAlloc() can routinely allocate over a gigabyte of memory, HeapAlloc() needs to be set up properly and VirtualAlloc() tends to be used for system based large blocks. I would ask a couple of normal questions, how much physical memory does the machine have and are you using it in a profile that restricts memory on a per application level. Also tell us which OS version you are running.

LATER : here is a quick demo that shows 128 meg of allocated memory.

Code Select Expand


; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
    include \masm32\include\masm32rt.inc
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

comment * -----------------------------------------------------
                        Build this  template with
                       "CONSOLE ASSEMBLE AND LINK"
        ----------------------------------------------------- *

    .data?
      value dd ?

    .data
      item dd 0

    .code

start:
   
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    call main
    inkey
    exit

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    msize equ <1024 * 1024 * 128>   ; allocate 128 meg

main proc

    LOCAL hMem  :DWORD
    LOCAL bLen  :DWORD

    print "Allocating 128 meg",13,10

    invoke GlobalAlloc, GMEM_FIXED or GMEM_ZEROINIT,msize
    mov hMem, eax

    print "Filling it with '1'",13,10

    mov eax, "1111"     ; fill buffer with "1"
    mov edx, hMem
    mov ecx, msize
    shr ecx, 2

  @@:
    mov [edx], eax
    add edx, 4
    sub ecx, 1
    jnz @B

    print "Getting buffer size in bytes",13,10
    print str$(len(hMem))," bytes length",13,10

    inkey "Press any key to free memory"
    invoke GlobalFree,hMem

    ret

main endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

end start


this is the screen output.

Code Select Expand


Allocating 128 meg
Filling it with '1'
Getting buffer size in bytes
134217728 bytes length
Press any key to free memory
Press any key to continue ...


[z941998]

Hutch,

I will do some more testing, but I have 2 Gig of memory with 1.5 Gig Free when I run and test applications.  I was also PM'd on this and they suggested that I change the model to a higher level (ie .386 to .486 etc before compiling and running).  I will look into this also.

When I ask for anything that is over 64K the procedure works and returns an addres. but when I go to use the addressed space anything after 64K I get one of those microsoft error messages ("Send Report") and the program then immediately crashes.  I assume this to be an allocation probrem, but now I am not sure.

I will do some more testing and get back to you.

Thanks
Steve

[BlackVortex]

If you want to allocate a lot of memory, then why not use VirtualAlloc ?

I found a nice comparison of the different methods here :
http://msdn.microsoft.com/en-us/library/aa366533%28VS.85%29.aspx

Granularity isn't a problem when allocating a big chunk of memory   :toothy
Also, MSDN seems to consider GlobalAlloc a legacy function and not recommended.

P.S.: You guys always have to do things the hard way.

P.S.2: I thought the model directive is only for the assembler to know which instructions to use etc. Would it make any difference to the bahaviour of the program ? ::)

[hutch--]

Steve,

You must use .386 or higher for 32 bit code, even on the oldest machine that will run 32 bit Windows. If you are not writing code for legacy hardware that is over 10 years old, use the highest specification, .686p and add the .MMX and .XMM as well so you can use any of the modern opcodes. try this at the start of you code.

Code Select Expand


      .686                      ; create 32 bit code
      .MMX
      .XMM
      .model flat, stdcall      ; 32 bit memory model
      option casemap :none      ; case sensitive


BlackVortex,

> If you want to allocate a lot of memory, then why not use VirtualAlloc ?

>> The VirtualAlloc function allows you to specify additional options for memory allocation. However, its allocations use a page granularity, so using VirtualAlloc can result in higher memory usage.

VirtualAlloc() has its place but there are other memory strategies that work better in other situations. GlobalAlloc() with fixed memory flag GMEM_FIXED is in fact very fast for large blocks, easily handles over 1 gig and is simpler to use. If you look i the guts of KERNEL you will see that it references the same function but with different flags. Great fun can be had poking around in the guts of NTOSKRNL.EXE and HAL.DLL.  :bg

[z941998]

Hutch I will test your latest recommendation in a bit, but here's what I found out so far.

I loaded an ran Hutch's procedure on my P4, it passed.
So I ran my procedure and it failed.
When I went to my other computers/systems both failed.

After debuging thru the steps I found that my definitions in the data area were at fault.
Here's the reason, I have four operating plaforms (CPM, DOS, Win 3.1 and XP) and I have 15 different
computers (8080 to P4's) so I normally try to establish certain rules for each level.
This is where I found that defining dw value may work on some systems and not others, this also goes for db amd dd definitions.

After making appropriate changes (even with Hutch's model), I recompiled and ran on the 16 bit systems and it passed.
I would then copy the exe to the 32 bit systems and they would proceed to failure.
For the most part, I use data driven systems to avoid hard coding issues, and by default dw only goes to 64K.
Once I saw this, I change the source code to match the required systems, recompiled for each configuration and they all ran find.

This problem eventually related to a configuration problem that I was over looking by coping files between systems verus generating code for the different systems.

I want to thank all of you for putting me back on track, sometimes we all need a little pushing or whatever.

Sincerly, humiliated in defeat and humbled in success.

P.S. I use the Page1_System definition to streamfile information to the RichEd feature in my apps and not knowingly this also happens to have a 64K limit.  I will work this issue latter along with the .686 .MMX  etc.