Slow building problem

[akalenuk]

I've found that making huge buffers in .data segment slows down a build process a lot. is it somehow possible to avoid this?
e.g.: "curString db  1048576 dup(?)" it takes very long time to build this. How come?

[hutch--]

Yes,

Use dynamically allocated memory, then you file is not as large and the build time is normal. This is a well known quirk in MASM and it is generally not recommended in any assembler as it makes the data section very large.

[Mark Jones]

Hello Alcalenuk, welcome to the group.

Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:

Code Select Expand


.data?
    hMem        dd ?
.code
    invoke GlobalAlloc,GMEM_PTR,1024*1024   ; allocate 1MB of ram
    mov hMem,eax                            ; hMem = address
; use the allocated memory here...
    invoke GlobalFree,hMem                 ; must free allocated mem


Edit: Use GlobalFree to release memory (and not CloseHandle).

[TNick]

Wellcome!

Or you can do something like this:

.data?
        hHeap           DWORD               ?
        pMem1          DWORD               ?
.code

        INVOKE             HeapCreate,   0,5000h,0 ;create a growable heap, with an initial size of 5000h
                                                                    ;do this once at the begining of your executable
        mov                  hHeap,          eax
        .
        .
        .
        INVOKE             HeapAlloc,,HEAP_ZERO_MEMORY,100h    ; when you need memory, just allocate from your heap
        mov                  pMem1,         eax
        .
        .
        .

        INVOKE   HeapFree,   hHeap,0,pMem1      ;free memory as needed
        .
        .
        .
        INVOKE             HeapAlloc,,HEAP_ZERO_MEMORY,100h    ; when you need memory, just allocate from your heap
        mov                  pMem1,         eax
        .
        .
        .

        INVOKE   HeapFree,   hHeap,0,pMem1        ;free memory as needed


        INVOKE   HeapDestroy, hHeap    ;never forget to destroy the heap when you no longer need it.



Maybe the masters around here can tell us when it is better to use GlobalAlloc, and when you should use HeapCreate... From what I know, it is better to use HeapCreate - HeapAlloc when you need memory in small parts, and GlobalAlloc when you need large blocs of memory, but this isn't shure.

[Ossa]

There are several "Alloc" sets of functions:

• Use Global when doing simple medium sized allocations (or if you need to write something quickly and can't be bothered with the others)
• I never use Local, but it is recommended for certain uses by the MSDN spec (for edit controls with memory allocated by the program for instance)
• Heap is useful for nearly everything - I would use this when you can, especially if you are doing many small allocations (see docs for details)
• Virtual is the best for large allocations, but cannot always be used and can lead to complications in certain cases

I think that is accurate and fairly comprehensive, but I'm sure people will correct me if I'm wrong.

Ossa

[KSS]

Ossa,
About what "lead to complications" you said?  I use MemMan based on VirtualAlloc() many years and have not any trouble with this. (used in Delphi7)

[zooba]

• Use Global when doing simple medium sized allocations (or if you need to write something quickly and can't be bothered with the others)

Global is slow compared to the others, especially the Heap functions. It actually allocates from a heap which has global scope, so it can be used by any process. MSDN recommends using the Heap functions and I tend to agree. Global only remains because it is necessary for some DDE and OLE functions.

The easiest way is to throw together a set of allocation macros (or download them - there are some (StringAlloc/ReAlloc/Free) in ASM Runtime, I might put them up separately too) and including the file whenever you want them.

Cheers,

Zooba :U

[akalenuk]

Use dynamically allocated memory, then you file is not as large and the build time is normal. This is a well known quirk in MASM and it is generally not recommended in any assembler as it makes the data section very large.

But the final size of .exe is about 12Kb anyway and it takes about 2Mb of memory under XP, so i am not concerned about the size of data, just the building time. But thanks, at least i won't try to fix this anymore  :bg

[akalenuk]

Hello Alcalenuk, welcome to the group.

I'm glad i've found this forum. The last place i tried to ask question about masm coding was some german newsgroup. Ii am not even that good in english, and writing deutsch was a real pain in the ass  :lol

Code Select Expand


    invoke GlobalAlloc,GMEM_PTR,1024*1024   ; allocate 1MB of ram


Thanks, but what is this GMEM_PTR exactly? Can i use GMEM_FIXED | GMEM_ZEROINIT instead?

[akalenuk]

        INVOKE             HeapCreate,   0,5000h,0 ;create a growable heap, with an initial size of 5000h
                                                                    ;do this once at the begining of your executable
        mov                  hHeap,          eax

Thanks, i'll try this someday.

[akalenuk]

Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:

And how to access that memory? Assuming it meant to be some text buffer. Is this allright?

Code Select Expand


lea ebx,hMem
add ebx,Some_Index
mov Some_Char_From_Buffer,[ebx]

Or maybe something more elegant exists?

[ToutEnMasm]

invoke globalalloc,,,,,
return the handle Hmem then:

Invoke globallock,Hmen,...
return a pointer to Hmem

                           ToutEnMasm

[akalenuk]

invoke globalalloc,,,,,
return the handle Hmem then:

Invoke globallock,Hmen,...
return a pointer to Hmem

Can't say i've got the point. Do you mean, that GlobalAlloc does not return a pointer itself?
{a minute later}
Right, i've checked this in a manual.
Thanks.

[Ossa]

About what "lead to complications" you said?  I use MemMan based on VirtualAlloc() many years and have not any trouble with this. (used in Delphi7)

I was asked a question privately a little while ago through a PM by someone who was having trouble with VirtualAlloc. Up until this point I had never had any issues with VirtualAlloc.

The test program I was presented with was initially quite complex, but after finding where the bug was, we found that the problem could be simplified a lot:

1) Create a window
2) Use VirtualAlloc to get some memory
3) Create an edit control
4) Set the VirtualAlloc'd memory as the edit control's memory.
5) Now try to type into the edit control.

The result: nothing happened. We both verified that the VirtualMemory was read/writable and that the edit control wa set up right.

To verify the control was created OK, I used GlobalAlloc (and once agian with LocalAlloc) to replace the VirtualAlloc call (both worked fine). To verify the VirtualAlloc, a buffer created using LocalAlloc/GlobalAlloc was used for the edit control and some VirtualAlloc'd memory created seperately. Then a string was copied to the VirtualAlloc'd memory and then copied from there to the Local/GlobalAlloc'd memory. The string was displayed fine even after modifying it in the VirtualAlloc'd memory.

Now, I don't use edit controls that often (all my apps in the last 2 years have been console apps), so I was not really the right person to bring this to, but it seems to me that it can lead to "complications". Whether this is because I am unaware of some of the limitations of edit controls (completely possible) or because I missed something remains to be seen. However, from what I have seen, VirtualAlloc can cause "complications".

If anyone has any input on this, then feel free to comment (I seem to have lost the code that displayed the problem, otherwise I would attach it).

Global is slow compared to the others, especially the Heap functions. It actually allocates from a heap which has global scope, so it can be used by any process. MSDN recommends using the Heap functions and I tend to agree. Global only remains because it is necessary for some DDE and OLE functions.

I say this only because it is by far the easiest one to use and really performs fine under most circumstances. As I said later, the Heap functions are better, but can be more complex to use. If performance is an issues, use Heap things by all means, but it really doesn't matter most of the time.

Ossa

[Mark Jones]

Code Select Expand


    invoke GlobalAlloc,GMEM_PTR,1024*1024   ; allocate 1MB of ram


Thanks, but what is this GMEM_PTR exactly? Can i use GMEM_FIXED | GMEM_ZEROINIT instead?

Oh goodness I'm sorry, I quoted the wrong text. It is GPTR not GMEM_PTR. :red

Yes, you can use either:
GMEM_FIXED + GMEM_ZEROINIT
GMEM_FIXED or GMEM_ZEROINIT
GPTR

I tried "GMEM_FIXED | GMEM_ZEROINIT" but ML complained about "illegal character in file."

Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:

And how to access that memory? Assuming it meant to be some text buffer. Is this allright?

Code Select Expand


lea ebx,hMem
add ebx,Some_Index
mov Some_Char_From_Buffer,[ebx]

Or maybe something more elegant exists?

That should work. Yes, the memory works like any other text buffer, you get the address and read/write bytes/words/dwords of data to it. :U

When the call to GlobalAlloc returns, the handle to the requested memory block is in EAX. So, you save that handle in a variable for later use. That handle is the memory offset where the requested space begins - i.e., say you requested 1MB of memory- after the call, EAX will contain the memory offset to where 1MB of RAM is reserved for your use. Windows allocates memory seperately for each process, so a GlobalAlloc in one application is not accessible from another application. The returned handle is also used in the GlobalFree function, to let windows know that it can once again re-allocate that memory for other programs.

In regards to addressing the data directly, here are some discussions on the "base + index * scale + displacement" method. It is very useful to know when working with large memory blocks:
http://www.masm32.com/board/index.php?topic=2766.msg21943#msg21943

Have fun! :bg