AddVectoredExceptionHandler single stepping

[donkey]

So I have to single step a program, easy enough in Win64 using vectored exception handling however there is a massive problem. First you set up the exception handler using AddVectoredExceptionHandler and then single step your program using the X64 CONTEXT. When the EXCEPTION_SINGLE_STEP is encountered it faithfully executes the handler code and processes the data I need. So the code looks like this:

Code Select Expand

// Set the single step bit
SetSingleStep FRAME
uses rdi,rsi,rbx
LOCAL Alignment:Q
LOCAL context:CONTEXT

invoke GetCurrentThread
mov rcx,rax
mov rbx,rax

mov D[context.ContextFlags],CONTEXT_CONTROL
invoke GetThreadContext,rcx,offset context
or D[context.EFlags],0x100
invoke SetThreadContext,rbx,offset context
ret
endf

Code Select Expand

invoke AddVectoredExceptionHandler,1,offset VectoredHandler
mov [hHandler],rax
invoke SetSingleStep

Code Select Expand

VectoredHandler FRAME  pExcptPointers
uses rbx,rdi,rsi
LOCAL ExceptLine[256]:%TCHAR
LOCAL hThread:%HANDLE

invoke GetCurrentThread
mov [hThread],rax

mov rdi,[pExcptPointers]
mov rsi,[rdi+EXCEPTION_POINTERS.ContextRecord] // Pointer to context structure
mov rdi,[rdi+EXCEPTION_POINTERS.ExceptionRecord] // Pointer to exception record

xor rbx,rbx
mov ebx,[rdi+EXCEPTION_RECORD64.ExceptionCode]
mov rax,[rdi+EXCEPTION_RECORD64.ExceptionAddress]

// Displays some info here
// Also handle other error codes etc...
// This was a rather lengthy bit of stuff that works fine
// and didn't have any bearing on the problem so I left it out

or D[rsi+CONTEXT.EFlags],0x100
invoke SetThreadContext,[hThread],rsi
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret
endf

That should be it right ? No, because there is no way to tell easily how long the instruction was since there is nothing in the CONTEXT (that I can find) that indicates that. And the execution continues at the beginning of the instruction that threw the error so it just sits on the same instruction instead of single stepping. Enough to drive you nuts, do I have to develop a complete instruction length decoder in order to single step a program ? There just isn't enough documentation on vectored exception handling even though its been around since XP, does anyone know of any good way to find the next instruction address using the given CONTEXT ?

Edgar

[baltoro]

...I had to laugh when I read this,... :bg

...do I have to develop a complete instruction length decoder in order to single step a program ?

You undoubtedly have read this (although it's 32-bit): New Vectored Exception Handling in Windows XP, Matt Pietrek, Sept 2001

[donkey]

Yes, I have read that thanks. Vectored exception handling has enough advantages over the old SEH model that I am interested in it. However the problem of turning single step back on exists in both as far as I know, you have to advance RIP yourself. The examples (in C) at MSDN just do a RIP++ and increment it. Obviously this will not work as moving it ahead a single byte is not useful or isn't as far as I know. For now I'm trying a couple of things until I finally break down and do an instruction length thingy, though that adds *A LOT* of work to what in essence is a very minor piece of the overall project. The alternative is to use one of the prebuilt libs like BeaEngine though that would add several hundred KB to a program that now stands at 5 KB.

Experimenting a little bit I have found that I should actually set TF on an INT3 instead of calling a procedure during set up. The problem with doing it outside of the exception handler is that I was getting lots of OS related garbage to walk through. By using a breakpoint I can jump directly to the code I need single stepped. Still no idea how to step it though, all the docs say that Rip should hold the address of the NEXT instruction but it doesn't, I must be missing something here...

[Tedd]

If you're single-stepping, you should get the exception (trap) after successfully executing each instruction, so there shouldn't be any issues with instruction length, or following branches, or other complexity.
If your instruction keeps restarting, that implies it didn't complete the first time - there was an exception (not a trap) to indicate something went wrong. The point is to allow you to fix the situation, and then re-execute the instruction in the hope it works this time. You shouldn't be skipping over instructions.

INT3 will restart each time, since it's a breakpoint to be used in-place (overwriting the original byte.) You're meant to put the original byte back, execute the full instruction, and then patch the breakpoint back in if desired (or leave it as a one-shot..)

I did implement single-stepping some time ago, though it was for low-level code without any OS interference, but I don't remember any problems (except for having to set TF each time you continue.)

[donkey]

Hi Ted,

Yes, I do know what it "should" be doing in theory however in practice I am having issues with it using x86-64 and WIN7 with vectored handling. My modified code waits for an INT3 to set TF then increments RIP in order to skip over the breakpoint and should execute a series of XOR EAX,EAX instructions (for test purposes). Any attempt to modify the code section in order to insert a NOP in place of the INT3 is met with an access violation so there is no apparent option to "repair" the situation. However since it is only a single byte instruction that is easily handled. My issue is that it sits at the first XOR EAX,EAX afterwards without incrementing RIP. The code is (only the relevant handlers):

Code Select Expand

invoke AddVectoredExceptionHandler,1,offset VectoredHandler
mov [%VEHANDLER],rax
int3

TESTLABEL:
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax

invoke RemoveVectoredExceptionHandler,[%VEHANDLER]
mov Q[%VEHANDLER],-1

Code Select Expand

VectoredHandler FRAME  pExcptPointers
uses rbx,rdi,rsi
LOCAL ExceptLine[256]:%TCHAR
LOCAL hThread:%HANDLE

invoke GetCurrentThread
mov [hThread],rax

mov rdi,[pExcptPointers]
mov rsi,[rdi+EXCEPTION_POINTERS.ContextRecord] // Pointer to context structure
mov rdi,[rdi+EXCEPTION_POINTERS.ExceptionRecord] // Pointer to exception record

xor rbx,rbx
mov ebx,[rdi+EXCEPTION_RECORD64.ExceptionCode]
mov rax,[rdi+EXCEPTION_RECORD64.ExceptionAddress]

// Make sure this is a single step
cmp ebx,EXCEPTION_SINGLE_STEP
jne >>.STARTSINGLESTEP
invoke wsprintf,offset ExceptLine,"RIP = %X",[rsi+CONTEXT.Rip]
invoke Print2Output,offset ExceptLine

.EXIT
// Reset single step
or D[rsi+CONTEXT.EFlags],0x100
invoke SetThreadContext,[hThread],rsi
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret

.STARTSINGLESTEP
cmp ebx,EXCEPTION_BREAKPOINT
jne >>.UNHANDLED
invoke Print2Output,"Breakpoint reached"
// A break point indicates that we should start single stepping
or D[rsi+CONTEXT.EFlags],0x100
// Step over the INT3 by incrementing RIP
inc Q[rsi+CONTEXT.Rip]
// Write the context
invoke SetThreadContext,[hThread],rsi
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret

.UNHANDLED
// Check for custom exception code here.
mov rax,EXCEPTION_CONTINUE_SEARCH
ret
endf

The address of the first xor eax,eax is printed before any of the exception handling is executed in order to know what it is, then the handler is installed. The output of the above code is:

Code Select Expand

Line 50: offset TESTLABEL = 401106
Breakpoint reached
RIP = 401106
RIP = 401106
RIP = 401106
RIP = 401106
RIP = 401106
... ad infinitum

As you can see RIP is not moving at all even though there is no exception other than the single step.

Edgar

[donkey]

OK I got it, DO NOT USE SetThreadContext to write the context record back, it completely screws up the execution. The API will reload the context record with any changes when it exits the handler, you just have to set the appropriate members and return. Should have known this, duh ! but I have only ever written external debugging algos so I missed it.

Code Select Expand

VectoredHandler FRAME  pExcptPointers
uses rbx,rdi,rsi
LOCAL ExceptLine[256]:%TCHAR

mov rdi,[pExcptPointers]
mov rsi,[rdi+EXCEPTION_POINTERS.ContextRecord] // Pointer to context structure
mov rdi,[rdi+EXCEPTION_POINTERS.ExceptionRecord] // Pointer to exception record

xor rbx,rbx
mov ebx,[rdi+EXCEPTION_RECORD64.ExceptionCode]
mov rax,[rdi+EXCEPTION_RECORD64.ExceptionAddress]

// Check for single step
cmp ebx,EXCEPTION_SINGLE_STEP
jne >>.STARTSINGLESTEP

invoke wsprintf,offset ExceptLine,"RIP = %X",[rsi+CONTEXT.Rip]
invoke Print2Output,offset ExceptLine
jmp >>.EXIT
.EXIT
// Reset single step
or D[rsi+CONTEXT.EFlags],0x100
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret

.STARTSINGLESTEP
cmp ebx,EXCEPTION_BREAKPOINT
jne >>.UNHANDLED
cmp Q[%SPYENABLED],0
jne >>
mov Q[%SPYENABLED], 1
// A break point indicates that we should start single stepping
invoke Print2Output,"Insert Breakpoint reached"
or D[rsi+CONTEXT.EFlags],0x100
inc Q[rsi+CONTEXT.Rip]
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret

:
mov Q[%SPYENABLED], 0
invoke Print2Output,"Remove Breakpoint reached"
and D[rsi+CONTEXT.EFlags],0xFFFFFEFF
inc Q[rsi+CONTEXT.Rip]
mov rax,EXCEPTION_CONTINUE_EXECUTION
ret

.UNHANDLED
// Pass other exceptions to the next handler, we're now using INT3 to exit single step
mov rax,EXCEPTION_CONTINUE_SEARCH
ret
endf

I found that it was better to use another INT3 to exit the single step so a flag was set to indicate whether I was entering or exiting it and checked when the breakpoint exception was processed. The calling code is:

Code Select Expand

invoke AddVectoredExceptionHandler,1,offset VectoredHandler
mov [%VEHANDLER],rax
int3

TESTLABEL:
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax
xor eax,eax

int3
invoke RemoveVectoredExceptionHandler,[%VEHANDLER]
mov Q[%VEHANDLER],-1

Output is as expected:

Code Select Expand

Line 55: offset TESTLABEL = 401105
Insert Breakpoint reached
RIP = 401107
RIP = 401109
RIP = 40110B
RIP = 40110D
RIP = 40110F
RIP = 401111
RIP = 401113
RIP = 401115
Remove Breakpoint reached
Line 73: Normal Exit

[drizz]

A call to SetThreadContext is not necessary and might be the source of your problem.

edit: As you figured out yourself :)

[donkey]

Thanks Drizz,

It was actually the example at MSDN that helped, go figure :) This is the bit that gave it away:

Code Select Expand

LONG WINAPI
VectoredHandlerSkip1(
    struct _EXCEPTION_POINTERS *ExceptionInfo
    )
{
    PCONTEXT Context;
   
    Sequence++;
    Context = ExceptionInfo->ContextRecord;
    Actual[0] = 0xcc;
#ifdef _AMD64_
    Context->Rip++;
#else
    Context->Eip++;
#endif   
    return EXCEPTION_CONTINUE_EXECUTION;
}

No call to SetThreadContext. So now Spy/StopSpy for the GoAsm x64 version of vKim's debug macros for RadAsm3 is done.

[baltoro]

Edgar,
:toothy In all seriousness, you should get an award for being a trailblazer. :toothy
We need more threads like this.

[donkey]

Well shucks, thanks baltoro. I would have combed through that MSDN code a little closer if it didn't have this at the top:

64-bit Windows:  This code is not suitable for 64-bit Windows.

But I saw the RIP line and thought maybe it actually did apply here. But really though, I should have known that SetThreadContext was the problem, its just been too long since I've done an in-process exception handler and I missed it.

[donkey]

As an addendum to this thread I have been chatting with the big debug guy at Microsoft and though the vectored handling interface is well documented enough it will not be expanded on at MSDN since it is too OS dependent. However, it will always react the same at the level I'm using here even though "under the hood" it might change. In our conversation I asked what the difference was between AddVectoredExceptionHandler and AddVectoredContinueHandler which in tests seemed to do exactly the same thing. The difference is not explained at MSDN in any docs I could find but essentially a vectored exception handler will be called before stack based SEH and a vectored continue handler will be called after SEH. So the continue handler becomes your last chance to fix a problem after all other error handling has failed. Thought this might be something useful to pass along.