HeapAlloc, registers and access violation exception

[shlomok]

Hi,

I am using Four-F's (four-f@mail.ru) great KMD kit and one of his commonly used methods is his HeapAlloc wrapper which uses his own stack frame and temporarily disable PROLOGUE and EPILOGUE.
His code follows:

Code Select Expand


malloc proc dwBytes:DWORD

option PROLOGUE:NONE
option EPILOGUE:NONE

invoke GetProcessHeap
invoke HeapAlloc, eax, 0, [esp+4]
ret (sizeof DWORD)

option PROLOGUE:PROLOGUEDEF
option EPILOGUE:EPILOGUEDEF

malloc endp



I am trying to write the same method however without using my own stack frame. My method is as follows:
(please forgive me for the heavy documentation ... 15 years of Java does that to you)

Code Select Expand


; -----------------------------------------------------------------------------
;       
;NAME:           allocateUsingHeapAlloc
;
;DESCRIPTION:    Memory allocation

;allocateUsingHeapAlloc returns the address of block of memory from an existing heap,
;identified by a heap handle. The allocated memory cannot be moved.
;If the memory cannot be allocated, the function returns NULL (0).
; -----------------------------------------------------------------------------
allocateUsingHeapAlloc PROC NEAR STDCALL PUBLIC uses eax ebx edx hHeap:DWORD, dwBytes:DWORD
;LPVOID WINAPI HeapAlloc(
; __in  HANDLE hHeap,
  ;__in  DWORD dwFlags,
  ;__in  SIZE_T dwBytes
;);
LOCAL dwBytesLocal:DWORD, hHeapLocal:DWORD                   
                   
invoke  HeapAlloc,hHeap,HEAP_NO_SERIALIZE + HEAP_ZERO_MEMORY,dwBytes
.if eax==NULL
invoke  StdOut,addr generalException
mov eax,FALSE
ret
.endif
;HEAP_ZERO_MEMORY indicates that the allocated memory will be initialized to zero
;HEAP_NO_SERIALIZE eliminates mutual exclusion on the heap which means
;Serialized access will not be used for this allocation.

;The return value is a pointer to the allocated memory block
;All return values of 4 bytes or less (except for floating-point values)
;are returned in the EAX register.
ret
allocateUsingHeapAlloc endp


Here we go with my questions. My program crashes, see later, but first:
1-I prefer being explicit in my code, however in the following snippet:

Code Select Expand


allocateUsingHeapAlloc PROC NEAR STDCALL PUBLIC uses eax ebx edx hHeap:DWORD, dwBytes:DWORD

Do I really need to preserve  eax ebx edx? or does the invoke take care of that? under which circumstances would it be mandatory to actually write "uses  eax ebx edx" etc?

2-I was calling the original method as follows:

Code Select Expand


;invoke  HeapAlloc,hMainHeap,HEAP_NO_SERIALIZE + HEAP_ZERO_MEMORY,eax

and then replaced it with my own method:

Code Select Expand


invoke  allocateUsingHeapAlloc,hMainHeap, eax

Now, inside my method i am using eax directly, but when declaring "uses eax" does that "corrupt" the value which is already inside eax when it is passed as a parameter to the method?

4-I saw many examples in which inside a PROC, local variables are first used to copy the method parameters to them and only they are copied to the registers.
why is that so? why cant the register used directly?

4- I am attaching the OllyDBG debug session and the exact location where the violation occurs, any help would be appreciated.

The whole source code is here also:

Code Select Expand


; --------------------------------------------------------------------------------------------------------------------


; -------- Include Files and Libraries --------------------------------------------------------------------------------
include     simple.inc


; -------- Initialised Data -------------------------------------------------------------------------------------------
.data
fileNameToOpen    db 'test.txt',0     ; file to read
fileNotFoundException db 'file not found',0
generalException db 'general exception',0

; -------- UnInitialised Data -----------------------------------------------------------------------------------------
.data?
hFile       dd ?
fileSize    dd ?
hMem        dd ?
bytesRead   dd ?


hInstance   dd ?
hWinGraphic dd  ?
hMainHeap DWORD ?
; -------- Program Code -----------------------------------------------------------------------------------------------
.code

main PROTO
allocateUsingHeapAlloc PROto hHeap:DWORD, dwBytes:DWORD

start:
;We store the handle to the heap and hence it is only invoked once
invoke GetProcessHeap
mov hMainHeap, eax
;INVOKE pushes arguments onto the stack and converts argument types to types expected when possible.
;These arguments can be referenced by their parameter label, rather than as offsets of the stack pointer.
invoke  main
    invoke  ExitProcess,0

;PROC sets up the appropriate stack frame according to the processor mode.
main proc
xor eax,eax
invoke GetModuleHandle, NULL
    mov    hInstance,eax        

    invoke  CreateFile,ADDR fileNameToOpen,GENERIC_READ,0,0,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,0
       
    .IF eax == INVALID_HANDLE_VALUE
invoke  StdOut,addr fileNotFoundException
invoke  ExitProcess,0
.ENDIF           
    mov     hFile,eax   
   
    invoke  GetFileSize,eax,0
    mov     fileSize,eax
    inc     eax               
   
    ;invoke  HeapAlloc,hMainHeap,HEAP_NO_SERIALIZE + HEAP_ZERO_MEMORY,eax   
    invoke  allocateUsingHeapAlloc,hMainHeap, eax
    mov     hMem,eax
               
    mov     BYTE PTR [eax],0
    invoke  ReadFile,hFile,hMem,fileSize,ADDR bytesRead,0
    invoke  CloseHandle,hFile
    invoke  StdOut,hMem
    invoke  HeapFree, hMainHeap, HEAP_NO_SERIALIZE, hMem       
main endp


; -----------------------------------------------------------------------------
;       
;NAME:           allocateUsingHeapAlloc
;
;DESCRIPTION:    Memory allocation

;allocateUsingHeapAlloc returns the address of block of memory from an existing heap,
;identified by a heap handle. The allocated memory cannot be moved.
;If the memory cannot be allocated, the function returns NULL (0).
; -----------------------------------------------------------------------------
allocateUsingHeapAlloc PROC NEAR STDCALL PUBLIC uses eax ebx edx hHeap:DWORD, dwBytes:DWORD
;LPVOID WINAPI HeapAlloc(
; __in  HANDLE hHeap,
  ;__in  DWORD dwFlags,
  ;__in  SIZE_T dwBytes
;);
LOCAL dwBytesLocal:DWORD, hHeapLocal:DWORD                   
                   
invoke  HeapAlloc,hHeap,HEAP_NO_SERIALIZE + HEAP_ZERO_MEMORY,dwBytes
.if eax==NULL
invoke  StdOut,addr generalException
mov eax,FALSE
ret
.endif
;HEAP_ZERO_MEMORY indicates that the allocated memory will be initialized to zero
;HEAP_NO_SERIALIZE eliminates mutual exclusion on the heap which means
;Serialized access will not be used for this allocation.

;The return value is a pointer to the allocated memory block
;All return values of 4 bytes or less (except for floating-point values)
;are returned in the EAX register.
ret
allocateUsingHeapAlloc endp

 

5-Last question, is there a code beautifier/formater under RadASM?

Thanks,

[qWord]

To be compatible with the WinABI, you must only preserve EDI,ESI and EBX. Preserving EAX is senseless, because this cause the return value to be overwritten (-> crash). Also: flags should be combined using the OR-operator:

HEAP_NO_SERIALIZE or HEAP_ZERO_MEMORY

[dedndave]

preserving EAX is not only senseless - it destroys the value you want returned

HEAP_NO_SERIALIZE should not be used with the default process heap

[shlomok]

To be compatible with the WinABI, you must only preserve EDI,ESI and EBX. Preserving EAX is senseless, because this cause the return value to be overwritten (-> crash). Also: flags should be combined using the OR-operator:

HEAP_NO_SERIALIZE or HEAP_ZERO_MEMORY

Hi,
Thanks for the input, I removed eax and however with respect to the OR-operator, I inspected the assembly and it does look that it is OK (e.g it shows: HEAP_NO_SERIALIZE | HEAP_ZERO_MEMORY as the FLAGS value).
Can you elaborate on that? 

In any case, the program keeps crashing, even when I am NOT invoking the HeapAlloc method:
(the following code crashes, if I remove the allocateUsingHeapAlloc declaration, all is well)

Code Select Expand


; -------- Include Files and Libraries --------------------------------------------------------------------------------
.386
.model flat,stdcall
option casemap:none

include     \masm32\include\windows.inc
include     \masm32\include\kernel32.inc
include     \masm32\include\user32.inc
include     \masm32\include\masm32.inc

;include \masm32\macros\macros.asm

includelib  \masm32\lib\kernel32.lib
includelib  \masm32\lib\user32.lib
includelib  \masm32\lib\masm32.lib

INCLUDE \masm32\include\Gdi32.inc
INCLUDELIB \masm32\lib\Gdi32.lib


; -------- Initialised Data -------------------------------------------------------------------------------------------
.data

; -------- UnInitialised Data -----------------------------------------------------------------------------------------
.data?

; -------- Program Code -----------------------------------------------------------------------------------------------
.code
main PROTO

start:
invoke  main
    invoke  ExitProcess,0

main proc

main endp

; -----------------------------------------------------------------------------
;       
;NAME:           allocateUsingHeapAlloc
;
;DESCRIPTION:    Memory allocation

;allocateUsingHeapAlloc returns the address of block of memory from an existing heap,
;identified by a heap handle. The allocated memory cannot be moved.
;If the memory cannot be allocated, the function returns NULL (0).
; -----------------------------------------------------------------------------
allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD                    
invoke  HeapAlloc,hHeap, HEAP_ZERO_MEMORY,dwBytes
ret
allocateUsingHeapAlloc endp

END start



[dedndave]

Code Select Expand

start:
invoke  main
    invoke  ExitProcess,0

main proc

main endp

oops - main has no code in it   :P
put a RET in there   :U

as for OR'ing flags...
it is safer to OR them
here is an example

Code Select Expand

WS_OVERLAPPEDWINDOW + WS_MINIMIZEBOX = 0D10000h
WS_OVERLAPPEDWINDOW or WS_MINIMIZEBOX = 0CF0000h

these flags set a pre-defined pattern of bits
when you OR them together, the other bits should not be affected
when you ADD them together, other bits may be affected due to carry from one bit to the next (ripple-carry)

[shlomok]

oops - main has no code in it   :P
put a RET in there  ....

Once again thanks for all the input, after fixing and debugging again I can see that the line that crashes now is:

Code Select Expand


.IF eax == NULL
;invoke  StdOut,addr generalException
;mov eax,FALSE
ret ; this line causes an access violation
.endif


I fail to understand, I am reaching the "ret" inside the "if" so if eax is NULL after invoking HeapAlloc, how come the String is allocated properly?

Full code follows:

Code Select Expand


; -------- Include Files and Libraries --------------------------------------------------------------------------------
include     simple.inc


; -------- Initialised Data -------------------------------------------------------------------------------------------
.data
fileNameToOpen    db 'test.txt',0     ; file to read
fileNotFoundException db 'file not found',0
generalException db 'general exception',0

; -------- UnInitialised Data -----------------------------------------------------------------------------------------
.data?
hFile       dd ?
fileSize    dd ?
hMem        dd ?
bytesRead   dd ?


hInstance   dd ?
hWinGraphic dd  ?
hMainHeap DWORD ?
; -------- Program Code -----------------------------------------------------------------------------------------------
.code

main PROTO
allocateUsingHeapAlloc PROTO hHeap:DWORD, dwBytes:DWORD


start:
invoke GetProcessHeap
mov hMainHeap, eax
invoke  main
    invoke  ExitProcess,0

main proc
xor eax,eax
invoke GetModuleHandle, NULL
    mov    hInstance,eax        
    invoke  CreateFile,ADDR fileNameToOpen,GENERIC_READ,0,0,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,0
       
    .IF eax == INVALID_HANDLE_VALUE
invoke  StdOut,addr fileNotFoundException
invoke  ExitProcess,0
.ENDIF           
    mov     hFile,eax   
   
    invoke  GetFileSize,eax,0
    mov     fileSize,eax
    inc     eax               
           
    invoke  allocateUsingHeapAlloc,hMainHeap, eax
    mov     hMem,eax
               
    mov     BYTE PTR [eax],0
    invoke  ReadFile,hFile,hMem,fileSize,ADDR bytesRead,0
    invoke  CloseHandle,hFile
    invoke  StdOut,hMem
    invoke  HeapFree, hMainHeap, 0 , hMem
ret       
main endp

; -----------------------------------------------------------------------------
;NAME:           allocateUsingHeapAlloc
;DESCRIPTION:    Memory allocation

;allocateUsingHeapAlloc returns the address of block of memory from an existing heap,
;identified by a heap handle. The allocated memory cannot be moved.
;If the memory cannot be allocated, the function returns NULL (0).
; -----------------------------------------------------------------------------
allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD
                   
invoke  HeapAlloc,hHeap, HEAP_ZERO_MEMORY,dwBytes
.IF eax == NULL
;invoke  StdOut,addr generalException
;mov eax,FALSE
ret
.endif
ret
allocateUsingHeapAlloc endp

END start




[dedndave]

Code Select Expand

allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD
                   
    invoke  HeapAlloc,hHeap, HEAP_ZERO_MEMORY,dwBytes
    .if eax == NULL
        invoke  StdOut,addr generalException
        mov     eax,FALSE
    .endif
    ret

allocateUsingHeapAlloc endp

you only need the one RET

seems to work ok, here
how are you building it ?
this must be linked as a console application

[dedndave]

Code Select Expand

allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD

this is overkill   :P

the .MODEL directive at the beginning of the file says stdcall - so that is not needed
PROC's are public by default - so that's not needed
the PROC does not use EDI, ESI, or EBX - so....

Code Select Expand

allocateUsingHeapAlloc PROC  hHeap:DWORD, dwBytes:DWORD

:bg

what we are not seeing is the simple.inc file
maybe show us that ???

[shlomok]

Code Select Expand

allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD
                   
    invoke  HeapAlloc,hHeap, HEAP_ZERO_MEMORY,dwBytes
    .if eax == NULL
        invoke  StdOut,addr generalException
        mov     eax,FALSE
    .endif
    ret

allocateUsingHeapAlloc endp

you only need the one RET

seems to work ok, here
how are you building it ?
this must be linked as a console application

Can you please explain why the code branches into the "ret" inside the ".IF" even if the statement " .if eax == NULL" evaluates to false?

Thanks,

[shlomok]

Code Select Expand

allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD

this is overkill   :P

the .MODEL directive at the beginning of the file says stdcall - so that is not needed
PROC's are public by default - so that's not needed
the PROC does not use EDI, ESI, or EBX - so....

Code Select Expand

allocateUsingHeapAlloc PROC  hHeap:DWORD, dwBytes:DWORD

:bg

what we are not seeing is the simple.inc file
maybe show us that ???

I changed the method signature, according to what you wrote it still crashes on the "ret" inside the .IF.
I know that I dont need the "ret" but I have to understand why its existence causes a crash. 

here is my include statement:

Code Select Expand


.386
.model flat,stdcall
option casemap:none

include     \masm32\include\windows.inc
include     \masm32\include\kernel32.inc
include     \masm32\include\user32.inc
include     \masm32\include\masm32.inc

;include \masm32\macros\macros.asm

includelib  \masm32\lib\kernel32.lib
includelib  \masm32\lib\user32.lib
includelib  \masm32\lib\masm32.lib

INCLUDE \masm32\include\Gdi32.inc
INCLUDELIB \masm32\lib\Gdi32.lib



[jj2007]

No problems here on Win7, see full code attached. What does Olly say about the exception?

[shlomok]

No problems here on Win7, see full code attached. What does Olly say about the exception?

Thanks jj2007 so nice of you to create your own exe,

Your executable runs flawlessly on my computer (64 bit Windows 7 ultimate). Just to make sure, I used the source code you attached and built the project, but it crashes again.
It is amazing that the size of the executable on my computer using poasm.exe is exactly the size that you attached e.g. 1536K, however when using RadASM, the size is 2560K (sure a different linker but still a big difference).

I configured OLLY as an just in time debugger to catch the crash, see the screenshot.

And these are my build options on RadASM:
C:\Masm32\Bin\ML.EXE /c /coff /Cp /nologo /I"C:\Masm32\Include" "heapalloc.asm"
Assembling: heapalloc.asm
Make finished.

C:\Masm32\Bin\ML.EXE /c /coff /Cp /nologo /I"C:\Masm32\Include" "heapalloc.asm"
Assembling: heapalloc.asm
C:\Masm32\Bin\LINK.EXE /SUBSYSTEM:CONSOLE /RELEASE /VERSION:4.0 /LIBPATH:"C:\Masm32\Lib" /OUT:"heapalloc.exe" "heapalloc.obj"
Microsoft (R) Incremental Linker Version 5.12.8078
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

I am also attaching my own executable (run it on a VM if you are worried about memory access)

These are my build options (i tried both ml.exe and poasm.exe) using Pelles C IDE:

Code Select Expand


> poasm.exe -AIA32 -Gd  -Fo"d:\dambo\forensics\prj\output\HeapAllocShlomok.obj" "d:\dambo\forensics\prj\HeapAllocShlomok.asm"
Building jj2007.tag.
Done.


Code Select Expand


> polink.exe -subsystem:console -machine:x86 -verbose  -out:"d:\dambo\forensics\prj\jj2007.exe" d:\dambo\forensics\prj\output\HeapAllocShlomok.obj
Loading d:\dambo\forensics\prj\output\HeapAllocShlomok.obj
Processed /Defaultlib:\masm32\lib\kernel32.lib
Processed /Defaultlib:\masm32\lib\user32.lib
Processed /Defaultlib:\masm32\lib\masm32.lib
Processed /Defaultlib:\masm32\lib\Gdi32.lib
Searching \masm32\lib\kernel32.lib
    Found _GetProcessHeap@0
        kernel32.lib(kernel32.dll)
    Found _ExitProcess@4
        kernel32.lib(kernel32.dll)
    Found _GetModuleHandleA@4
        kernel32.lib(kernel32.dll)
    Found _CreateFileA@28
        kernel32.lib(kernel32.dll)
    Found _GetFileSize@8
        kernel32.lib(kernel32.dll)
    Found _ReadFile@20
        kernel32.lib(kernel32.dll)
    Found _CloseHandle@4
        kernel32.lib(kernel32.dll)
    Found _HeapFree@12
        kernel32.lib(kernel32.dll)
    Found _HeapAlloc@12
        kernel32.lib(kernel32.dll)
    Found __IMPORT_DESCRIPTOR_kernel32
        kernel32.lib(kernel32.dll)
    Found __NULL_IMPORT_DESCRIPTOR
        kernel32.lib(kernel32.dll)
    Found kernel32_NULL_THUNK_DATA
        kernel32.lib(kernel32.dll)
Searching \masm32\lib\user32.lib
Searching \masm32\lib\masm32.lib
    Found _StdOut@4
        masm32.lib(stdout.obj)
    Found _StrLen@4
        masm32.lib(strlen.obj)
Searching \masm32\lib\gdi32.lib
Searching \masm32\lib\kernel32.lib
    Found _GetStdHandle@4
        kernel32.lib(kernel32.dll)
    Found _WriteFile@20
        kernel32.lib(kernel32.dll)
Searching \masm32\lib\user32.lib
Searching \masm32\lib\masm32.lib
Searching \masm32\lib\gdi32.lib
Searching \masm32\lib\kernel32.lib
Searching \masm32\lib\user32.lib
Searching \masm32\lib\masm32.lib
Searching \masm32\lib\gdi32.lib

Discarded .data from masm32.lib(stdout.obj)
Discarded .data from masm32.lib(strlen.obj)
Discarded size: 0 bytes
Creating executable: d:\dambo\forensics\prj\jj2007.exe
Done.


And additionally, the default project options under Pelles C use "_cdcel" as the calling convention for the compiler and the linker, this might be related but on RadASM, same crash different compiler and linking options.

 

[qWord]

what ever the reason, in the function allocateUsingHeapAlloc() the last RET is replaced by an INT3, which is followed by the import table. Maybe a bug of poasm?

[dedndave]

ahhhh
it assembles and runs fine using masm - with the extra RET

[jj2007]

allocateUsingHeapAlloc PROC STDCALL PUBLIC uses EDI ESI EBX hHeap:DWORD, dwBytes:DWORD   
   invoke  HeapAlloc,hHeap, HEAP_ZERO_MEMORY,dwBytes
   .IF eax == NULL
      ; invoke  StdOut,addr generalException       
      ; mov eax,FALSE
      ret
   .endif   
   ret
allocateUsingHeapAlloc endp


*** My executable: ***

Code Select Expand

Address                    Hex dump               Command
004010D3                   ³. E8 44000000         call <jmp.&kernel32.HeapAlloc>
004010D8                   ³. 83F8 00             cmp eax, 0
004010DB                   ³. 75 07               jne short 004010E4
004010DD                   ³. 5B                  pop ebx
004010DE                   ³. 5E                  pop esi
004010DF                   ³. 5F                  pop edi
004010E0                   ³. C9                  leave
004010E1                   ³. C2 0800             retn 8
004010E4                   ³> 5B                  pop ebx
004010E5                   ³. 5E                  pop esi
004010E6                   ³. 5F                  pop edi
004010E7                   ³. C9                  leave
004010E8                   À. C2 0800             retn 8



*** Your executable: ***

Code Select Expand

Address                    Hex dump               Command
004010D0                   ³. E8 39000000         call <jmp.&kernel32.HeapAlloc>
004010D5                   ³. 85C0                test eax, eax
004010D7                   ³. 75 04               jne short 004010DD
004010D9                   ³. C9                  leave
004010DA                   ³. C2 0800             retn 8
004010DD                   ³> CC                  int3
004010DE                   À$ FF25 84204000       jmp near [<&kernel32.GetProcessHeap>]


So it's not the linker, it's the assembler that creates havoc - see jne short 004010DD