Hi folks!
wrote a basic dll, including a function:
_test proc STDCALL [someargs]
here be 2 cascaded loops (using ecx)
calculating a basic matrix-vector product
with the fpu
ret
_test endp
loading the dll in C works, the function itself works correctly,
calling the function consecutively 4 times works,
but calling it a fifth time causes segfault...
seems to be an fpu related problem,
because removing the fpu-instructions works.
what am i missing?
regards,
robbje
Quote from: robbje on April 14, 2010, 04:00:32 AM
what am i missing?
Robbje,
You miss Olly (http://ollydbg.de/). Put an int 3 at the start of the routine, and check what happens. The FPU does have modes that throw exceptions, but without code nobody here can guess what the problem is...
First guess would have been that STDCALL is not the right way to call a proc from C - but you wrote that it works if you omit the fpu stuff... ::)
Best, jj
you can always guess ;)
Calculates pDst = pMat x pVec
_mulMatrixVector proc STDCALL pDst:DWORD, pMat:DWORD, pVec:DWORD, rowcount:DWORD, colcount:DWORD
mov edx, [pMat]
mov ebx, [pDst]
mov ecx, rowcount
rows:
fldz
push ecx
push ebx
mov ebx, [pVec]
mov ecx, colcount
cols:
fldz
fadd qword ptr [edx]
fmul qword ptr [ebx]
faddp st(1), st
add edx, 8
add ebx, 8
loop cols
pop ebx
fstp qword ptr [ebx]
add ebx, 8
pop ecx
loop rows
ret
_mulMatrixVector endp
Olly says the error is that trying to read [edx] at the faddition causes access violation.
What bugs me is that this happens after the routine worked for 4 times without error.
What is the value of ecx?
Does edx point to valid memory?
Does esp slowly creep after each call to the routine?
...?
ecx shows normal behaviour, then suddenly goes 33333333 ;O
edx does not point to valid memory anymore
seems like esp is creeping by 14 every call...
did not find out why yet...
olly shows that at the beginning of the routing it does:
push ebp
mov ebp, esp
and LEAVE at the end...
I can not see an inconsistency in my code. Probably I'm not aware
of calling functions and what it means to the stack. Anyone can
enlighten me?
Calling the procedure from assembly code the only way I can find to trigger an exception is by specifying values for the row and column counts that cause the procedure to access outside of my test data. Clearing the FPU interrupt masks showed no problems.
If you are calling the procedure from C your prototype needs to specify the __stdcall attribute.
my C prototype looks like this:
extern "C" void _mulMatrixVector(double*, double*, double*, int, int);
compiler is
gcc (GCC) 3.4.2 (mingw-special)
running on vista
edit: do i have to pop stuff manually from the stack?
No time to test this, but I would guess that it should be:
extern void __stdcall _mulMatrixVector(double*, double*, double*, int, int);
For the STDCALL calling convention the called procedure removes the parameters from the stack. For the C calling convention the calling code removes the parameters from the stack.
Or the other way round, proc C pDst:...
You have 5 args, and esp creeps by 14h=20 per call...
proc C did the trick,
extern void __stdcall _mulMatrixVector( ... );
gives undefined reference...