Local Variables - Size Limit?

[raleeper]

I was trying to declare in my windows procedure a local variable consisting of 2E40h dwords, using

   LOCAL   smp[50*25*2]:DWORD

I found that the largest number of dwords I could use was 76Dh (= 1DB4h bytes).
Other locals use 4Ch bytes, so it looks as if the total limit is 1E00h, which is at least a fairly round number.

But can this be right?  I find no mention of any limit on local variable size in the documentation I've found so far, which makes me suspect that I have some kind of alignment problem.

Actually, all I need is [25*2] dwords = 128h bytes, well within the limit, and if it is an alignment problem it will probably be corrected as I rewrite and document my code.

So this question falls more under the category of idle curiosity.

Thanks.

[hutch--]

The absolute limit is determined by the amount of stack memory that is available with the exe file. It is normally 1 meg but this can be changed with linker settings.

[zooba]

I was under the impression that the stack will automatically grow as needed to some extent. Chep posted a prologue macro here which will cause Windows to commit each page of the stack and allow large amounts of local variables. Chep quotes 4kB as the limit, but I believe the limit to actually be (just under) 8kb, since the crash will only occur when you try and access the page beyond the guard page.

In any case, try Chep's code and see if it fixes the (non-)issue.

If you are allocating DWORDs it won't be an alignment problem unless you also have MMX or SSE variables on the stack (in which case you need better than 4-byte alignment).

Cheers,

Zooba :U

[sinsi]

I found that the largest number of dwords I could use was 76Dh (= 1DB4h bytes).

How did you find out the limit?

[hutch--]

It is the two linker option Stack Reserve - Stack Commit where you do the adjustments. If you are going to run recursive code like a Quick sort or similar, you set the stack size to handle your expected largest recursion depth.

[raleeper]

I found that the largest number of dwords I could use was 76Dh (= 1DB4h bytes).

How did you find out the limit?

Trial and error.

2e40  no
1000  no
800   no
400   yes
600   yes
...
76d   yes
76e   no

Yes = program works; no = program doesn't work.

[hutch--]

Here is the right way to do it, as a linker option.

Code Select Expand


\MASM32\BIN\Link.exe /SUBSYSTEM:WINDOWS /STACK:16777216 template.obj rsrc.obj


Very large stack allocation (16 meg).

[zooba]

hutch--,

I don't believe that will solve the specific problem we are seeing here. Allocating 7604 bytes won't approach the stack limit at all, but it could cross two page boundaries.

The stack reserves the size specified by the linker but it doesn't commit it until you try and access it. It detects attempted accesses using a guard page, so the first time you attempt to access a page that hasn't been committed, Windows will catch an exception and commit it. However, it only guards one page, which is the next uncommitted page. Attempting to access the page beyond the guard page will cause an access violation exception.

The local variables on the stack are 'allocated' by simply changing the stack pointer (ESP). Until you attempt to access the next page there's no extra memory allocated. By shifting the stack pointer by more than one page (probably 4096 bytes) you risk missing the guard page and attempting to access an uncommitted page which causes an access violation.

raleeper,

How do you know the program is not working? Does it simply disappear? Or do you get an error dialog (Send Error Report or similar)?

If the program disappears (or never appears and the process is not visible in the Task Manager), it's a stack overflow, in which case hutch's solution may help, but it's more likely that there's an uncontrolled recursive function.

If you're getting an error dialog, try using Chep's prologue macro I linked to earlier, or alternatively add some reads/writes along the large array starting from the end up to the start in gaps of 4096 bytes (or zero the array from end to start if that's what you want). (Note that you must start from the end of the array, since those addresses will be on either a committed page or the guard page while the beginning of the array will be in uncommitted space).

I know that it's not an issue now, but for the benefit of people searching the forum I think it's worthwhile having a complete answer.

Cheers,

Zooba :U

Update - I just found Chep's original explanation of this issue. He's got diagrams  :dazzled: :bg

[hutch--]

Diagrams are no substitute for PE specifications.

The batch file to build the test piece. Allocates 12 meg at link time. Runtime file allocate 8 of that 12 meg and writes to it as a test. Build exe without stack mod and it does not run.

Code Select Expand


@echo off

if exist "stack.obj" del "stack.obj"
if exist "stack.exe" del "stack.exe"

\masm32\bin\ml /c /coff "stack.asm"
if errorlevel 1 goto errasm

:nores
\masm32\bin\Link /SUBSYSTEM:WINDOWS /STACK:12582912,12582912 "stack.obj"
if errorlevel 1 goto errlink
dir "stack.*"
goto TheEnd

:errlink
echo _
echo Link error
goto TheEnd

:errasm
echo _
echo Assembly Error
goto TheEnd

:TheEnd

pause


The test piece.

Code Select Expand


; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    include \masm32\include\masm32rt.inc
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

    stack_test PROTO

    .code

start:
   
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

    call main

    exit

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

main proc

    invoke stack_test

    fn MessageBox,0,"Stack test","Done",MB_OK

    ret

main endp

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

stack_test proc

    LOCAL pbuff :DWORD
    LOCAL big_buffer[1024*1024*8]:BYTE  ; allocate 8 meg of memory

    mov pbuff, ptr$(big_buffer)

    invoke memfill,pbuff,8388608,0      ; write to 8 meg of it

    ret

stack_test endp

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

end start


[raleeper]

raleeper,

How do you know the program is not working? Does it simply disappear? Or do you get an error dialog (Send Error Report or similar)?

If the program disappears (or never appears and the process is not visible in the Task Manager), it's a stack overflow, in which case hutch's solution may help, but it's more likely that there's an uncontrolled recursive function.

It either never appears or appears then disappears in a millisecond or so.  So I guess it is stack overflow, but I thought the default stack was 1 meg, (F4240h) which leaves plenty of room for a measly 1E01 bytes.  Anyway, the problen is doubly academic now, since I have decided to abandon the idea of using a local variable - too hard to debug.

I don't have any recursives, at least not intentionally, but wouldn't an uncontrolled recursive - one that never terminates - also cause stack overflow? Maybe just taking a little longer and possibly doing something visible before it does?

Thanks.

[drizz]

the default values for stack are 0x00100000 for Stack Reserve and 0x00001000 for Stack Commit.
if you increase the size of Commited Stack ("/STACK:reserve[,commit]" linker command) memory you will not get a crash.

[MichaelW]

Or just access the stack space from higher addresses to lower addresses, in steps not greater than 4096 bytes, and Windows will increase the committed stack for you (but only up to the point that the committed stack equals the stack reserve).

[tenkey]

I don't have any recursives, at least not intentionally, but wouldn't an uncontrolled recursive - one that never terminates - also cause stack overflow? Maybe just taking a little longer and possibly doing something visible before it does?

You can have unintentional recursion when handling messages. You process window messages in a window procedure. The window procedure is a "callback" function because it is intended to be called directly from the OS, as opposed to your own code.

This is the scenario:

You get a message, and your window procedure is called. You handle a message by calling an API. If the API needs to notify your window of status changes or is asking for a change in the window display, it will call your window procedure again before you are done (i.e., recursively).