Stack Probing PROLOGUE macro

[Petroizki]

You guys are reinventing what we have already figured out...  :eek

- Yes, every page only needs to be probed on one DWORD.
- Use 'mov dword ptr [ebp], eax' instead of 'mov dword ptr [ebp], 0', to make probing smaller annd faster.
- Just reserve the local stack at once (sub/add only once), and then probe the pages (or vice versa), it makes less code this way.

But in our case I guess we don't mind if the stack is overwritten by a debugger before esp is adjusted, as we are writing dummy values just to make sure each page is probed.
On the contrary it's more likely a problem could arise if we adjust esp before probing the stack, as a debugger could then hit a non probed, unguarded page, thus leading to a GPF.

Am I wrong?

I don't know. What if we overwrite some important value the debugger is currently using? It might be possible that both ways would crash on some debuggers. But i guess your way might be better.

[chep]

Mirno,

All the tests I have done show that only USES has an effect on the generated code. NEAR/FAR/calling convention etc do not affect the generation of the stack frame.

Also, you don't need the alignedLocals thing as MASM automatically rounds up the value for you:

Code Select Expand

TestProc PROC
  LOCAL odd[3]:BYTE
  ...
TestProc ENDP

generates the following stack frame:

Code Select Expand

push    ebp
mov     ebp, esp
add     esp, 0FFFFFFFCh ; -4
(even when using a custom prologue, the localbytes argument is already rounded)

What if we overwrite some important value the debugger is currently using?

My understanding here is that debuggers don't (or at least shouldn't) leave important values on the stack: as soon as the control is returned to the debugged program, the debugger should assume that the program will mess up the stack (after all, it's the program's stack, not the debugger's).

Well, anyway I guess we'll have hard time really sorting this out... unless a Visual Studio team member shows up to clarify everything! :P

[chep]

I finally added an option for looped probing, using a macro argument (ROLLED) :

Code Select Expand

OPTION PROLOGUE:STACKPROBE
TestProc PROC <ROLLED> USES esi edi prm:DWORD
  ; ...
TestProc ENDP

It generates the following code:

Code Select Expand

    push ebp
    mov  ebp, esp
      add  ebp, (-max_probe) ; [1]
    @rolled:
      mov  DWORD PTR [ebp], eax
      add  ebp, page_size
      cmp  ebp, esp
      jne  @rolled ; [2]
      add  esp, (-localbytes)

The loop body itself (from [1] to [2] included) takes 19 bytes, while the unrolled version takes 6 bytes *per page*. So it becomes space-efficient to use the rolled version starting at 4 probed pages, ie. strictly more than 12Kb of LOCALs.

Q: maybe it could be useful to have FORCEUNROLLED / FORCEROLLED arguments, and by default let the macro decide of the most efficient version?