I've found that making huge buffers in .data segment slows down a build process a lot. is it somehow possible to avoid this?
e.g.: "curString db 1048576 dup(?)" it takes very long time to build this. How come?
Yes,
Use dynamically allocated memory, then you file is not as large and the build time is normal. This is a well known quirk in MASM and it is generally not recommended in any assembler as it makes the data section very large.
Hello Alcalenuk, welcome to the group.
Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:
.data?
hMem dd ?
.code
invoke GlobalAlloc,GMEM_PTR,1024*1024 ; allocate 1MB of ram
mov hMem,eax ; hMem = address
; use the allocated memory here...
invoke GlobalFree,hMem ; must free allocated mem
Edit: Use GlobalFree to release memory (and not CloseHandle).
Wellcome!
Or you can do something like this:
.data?
hHeap DWORD ?
pMem1 DWORD ?
.code
INVOKE HeapCreate, 0,5000h,0 ;create a growable heap, with an initial size of 5000h
;do this once at the begining of your executable
mov hHeap, eax
.
.
.
INVOKE HeapAlloc,,HEAP_ZERO_MEMORY,100h ; when you need memory, just allocate from your heap
mov pMem1, eax
.
.
.
INVOKE HeapFree, hHeap,0,pMem1 ;free memory as needed
.
.
.
INVOKE HeapAlloc,,HEAP_ZERO_MEMORY,100h ; when you need memory, just allocate from your heap
mov pMem1, eax
.
.
.
INVOKE HeapFree, hHeap,0,pMem1 ;free memory as needed
INVOKE HeapDestroy, hHeap ;never forget to destroy the heap when you no longer need it.
Maybe the masters around here can tell us when it is better to use GlobalAlloc, and when you should use HeapCreate... From what I know, it is better to use HeapCreate - HeapAlloc when you need memory in small parts, and GlobalAlloc when you need large blocs of memory, but this isn't shure.
There are several "Alloc" sets of functions:
- Use Global when doing simple medium sized allocations (or if you need to write something quickly and can't be bothered with the others)
- I never use Local, but it is recommended for certain uses by the MSDN spec (for edit controls with memory allocated by the program for instance)
- Heap is useful for nearly everything - I would use this when you can, especially if you are doing many small allocations (see docs for details)
- Virtual is the best for large allocations, but cannot always be used and can lead to complications in certain cases
I think that is accurate and fairly comprehensive, but I'm sure people will correct me if I'm wrong.
Ossa
Ossa,
About what "lead to complications" you said? I use MemMan based on VirtualAlloc() many years and have not any trouble with this. (used in Delphi7)
Quote from: Ossa on July 05, 2006, 12:19:40 AM
- Use Global when doing simple medium sized allocations (or if you need to write something quickly and can't be bothered with the others)
Global is slow compared to the others, especially the Heap functions. It actually allocates from a heap which has global scope, so it can be used by any process. MSDN recommends using the Heap functions and I tend to agree. Global only remains because it is necessary for some DDE and OLE functions.
The easiest way is to throw together a set of allocation macros (or download them - there are some (StringAlloc/ReAlloc/Free) in ASM Runtime (http://web.aanet.com.au/zooba/ASMRT.zip), I might put them up separately too) and including the file whenever you want them.
Cheers,
Zooba :U
Quote from: hutch-- on July 04, 2006, 02:49:47 PM
Use dynamically allocated memory, then you file is not as large and the build time is normal. This is a well known quirk in MASM and it is generally not recommended in any assembler as it makes the data section very large.
But the final size of .exe is about 12Kb anyway and it takes about 2Mb of memory under XP, so i am not concerned about the size of data, just the building time. But thanks, at least i won't try to fix this anymore :bg
Quote from: Mark Jones on July 04, 2006, 03:38:32 PM
Hello Alcalenuk, welcome to the group.
I'm glad i've found this forum. The last place i tried to ask question about masm coding was some german newsgroup. Ii am not even that good in english, and writing deutsch was a real pain in the ass :lol
Quote from: Mark Jones on July 04, 2006, 03:38:32 PM
invoke GlobalAlloc,GMEM_PTR,1024*1024 ; allocate 1MB of ram
Thanks, but what is this GMEM_PTR exactly? Can i use GMEM_FIXED | GMEM_ZEROINIT instead?
Quote from: TNick on July 04, 2006, 03:55:15 PM
INVOKE HeapCreate, 0,5000h,0 ;create a growable heap, with an initial size of 5000h
;do this once at the begining of your executable
mov hHeap, eax
Thanks, i'll try this someday.
Quote from: Mark Jones on July 04, 2006, 03:38:32 PM
Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:
And how to access that memory? Assuming it meant to be some text buffer. Is this allright?
lea ebx,hMem
add ebx,Some_Index
mov Some_Char_From_Buffer,[ebx]
Or maybe something more elegant exists?
invoke globalalloc,,,,,
return the handle Hmem then:
Invoke globallock,Hmen,...
return a pointer to Hmem
ToutEnMasm
Quote from: ToutEnMasm on July 05, 2006, 01:45:59 PM
invoke globalalloc,,,,,
return the handle Hmem then:
Invoke globallock,Hmen,...
return a pointer to Hmem
Can't say i've got the point. Do you mean, that GlobalAlloc does not return a pointer itself?
{a minute later}
Right, i've checked this in a manual.
Thanks.
Quote from: KSS on July 05, 2006, 03:30:59 AM
About what "lead to complications" you said? I use MemMan based on VirtualAlloc() many years and have not any trouble with this. (used in Delphi7)
I was asked a question privately a little while ago through a PM by someone who was having trouble with VirtualAlloc. Up until this point I had never had any issues with VirtualAlloc.
The test program I was presented with was initially quite complex, but after finding where the bug was, we found that the problem could be simplified a lot:
1) Create a window
2) Use VirtualAlloc to get some memory
3) Create an edit control
4) Set the VirtualAlloc'd memory as the edit control's memory.
5) Now try to type into the edit control.
The result: nothing happened. We both verified that the VirtualMemory was read/writable and that the edit control wa set up right.
To verify the control was created OK, I used GlobalAlloc (and once agian with LocalAlloc) to replace the VirtualAlloc call (both worked fine). To verify the VirtualAlloc, a buffer created using LocalAlloc/GlobalAlloc was used for the edit control and some VirtualAlloc'd memory created seperately. Then a string was copied to the VirtualAlloc'd memory and then copied from there to the Local/GlobalAlloc'd memory. The string was displayed fine even after modifying it in the VirtualAlloc'd memory.
Now, I don't use edit controls that often (all my apps in the last 2 years have been console apps), so I was not really the right person to bring this to, but it seems to me that it can lead to "complications". Whether this is because I am unaware of some of the limitations of edit controls (completely possible) or because I missed something remains to be seen. However, from what I have seen, VirtualAlloc can cause "complications".
If anyone has any input on this, then feel free to comment (I seem to have lost the code that displayed the problem, otherwise I would attach it).
Quote from: zooba on July 05, 2006, 05:09:15 AM
Global is slow compared to the others, especially the Heap functions. It actually allocates from a heap which has global scope, so it can be used by any process. MSDN recommends using the Heap functions and I tend to agree. Global only remains because it is necessary for some DDE and OLE functions.
I say this only because it is by far the easiest one to use and really performs fine under most circumstances. As I said later, the Heap functions are better, but can be more complex to use. If performance is an issues, use Heap things by all means, but it really doesn't matter most of the time.
Ossa
Quote from: akalenuk on July 05, 2006, 12:55:18 PM
Quote from: Mark Jones on July 04, 2006, 03:38:32 PM
invoke GlobalAlloc,GMEM_PTR,1024*1024 ; allocate 1MB of ram
Thanks, but what is this GMEM_PTR exactly? Can i use GMEM_FIXED | GMEM_ZEROINIT instead?
Oh goodness I'm sorry, I quoted the wrong text. It is GPTR not GMEM_PTR. :red
Yes, you can use either:
GMEM_FIXED + GMEM_ZEROINIT
GMEM_FIXED or GMEM_ZEROINIT
GPTR
I tried "GMEM_FIXED | GMEM_ZEROINIT" but ML complained about "illegal character in file."
Quote from: akalenuk on July 05, 2006, 01:22:41 PM
Quote from: Mark Jones on July 04, 2006, 03:38:32 PM
Here's some quick code to allocate a block of memory at runtime and provide a handle (address) to it:
And how to access that memory? Assuming it meant to be some text buffer. Is this allright?
lea ebx,hMem
add ebx,Some_Index
mov Some_Char_From_Buffer,[ebx]
Or maybe something more elegant exists?
That should work. Yes, the memory works like any other text buffer, you get the address and read/write bytes/words/dwords of data to it. :U
When the call to GlobalAlloc returns, the handle to the requested memory block is in EAX. So, you save that handle in a variable for later use. That handle is the memory offset where the requested space begins - i.e., say you requested 1MB of memory- after the call, EAX will contain the memory offset to where 1MB of RAM is reserved for your use. Windows allocates memory seperately for each process, so a GlobalAlloc in one application is not accessible from another application. The returned handle is also used in the GlobalFree function, to let windows know that it can once again re-allocate that memory for other programs.
In regards to addressing the data directly, here are some discussions on the "base + index * scale + displacement" method. It is very useful to know when working with large memory blocks:
http://www.masm32.com/board/index.php?topic=2766.msg21943#msg21943
Have fun! :bg
Quote from: Mark Jones on July 06, 2006, 12:05:26 AM
In regards to addressing the data directly, here are some discussions on the "base + index * scale + displacement" method. It is very useful to know when working with large memory blocks:
http://www.masm32.com/board/index.php?topic=2766.msg21943#msg21943
Thanks, i'll look it through.
Ossa, I think I'm the one who was trying to work out the VirtualAlloc / editbox issue, don't think we ever figured that out.
Here are my left over comparisons for edit boxes though.
I have two programs here, one is a simple program with an edit box using HeapAlloc memory, and one is using VirtualAlloc.
The virtualalloc is odd. It doesn't usually work, which I can accept. The odd thing is that on some machines it DOES work. Or, it will work occasionally. I'm thinking it's happening from a coincidental alignment of memory blocks, since I don't see any other explanation for it to "sometimes" work.
Attached the two programs in one zip file. They are safe and won't crash or anything, so feel free to download them and do anything you want with them.
[attachment deleted by admin]
about the editbox topic...
psdk clearly states that you should use local-mem functions
on the edit handle (atleast on NT, 9x is surely different).
hence, when you set the VirtualAlloc-handle to the edit,
OS calls LocalSize() on it and fails (you can verifyy it by setting bp in OllyDbg).
VirtualAlloc-handle might work only on 9x os, i don't know.
about big initialized buffers...
you can create big binary file and use bin2obj or fda to convert it to obj,
then you can link that to exe...OR switch to poasm :)
Quote from: Mark JonesThe returned handle is also used in the CloseHandle function
this is wrong, you must use the appropriate xxxxFree counterparts. Memory handle is not a Kernel Object.
Quote from: drizz on July 26, 2006, 04:51:53 PM
OS calls LocalSize() on it and fails (you can verifyy it by setting bp in OllyDbg).
@drizz
Nice to know the reason, thanks
@General thread topic
This illustrates why the Virtual* functions are not always the best choice and why I said that they can "lead to complications". The most appropriate set of memory functions should be used every time.
Ossa
Quote from: drizz on July 26, 2006, 05:31:36 PM
Quote from: Mark JonesThe returned handle is also used in the CloseHandle function
this is wrong, you must use the appropriate xxxxFree counterparts. Memory handle is not a Kernel Object.
This is correct, thanks for catching that brain fart. Use GlobalFree instead of CloseHandle to free that memory.
Re: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/globalalloc.asp