basic esp problem, was: basic(?) fpu problem

[robbje]

Hi folks!

wrote a basic dll, including a function:

_test proc STDCALL [someargs]

here be 2 cascaded loops (using ecx)
calculating a basic matrix-vector product
with the fpu

ret
_test endp

loading the dll in C works, the function itself works correctly,
calling the function consecutively 4 times works,

but calling it a fifth time causes segfault...

seems to be an fpu related problem,
because removing the fpu-instructions works.

what am i missing?

regards,
robbje

[jj2007]

what am i missing?

Robbje,

You miss Olly. Put an int 3 at the start of the routine, and check what happens. The FPU does have modes that throw exceptions, but without code nobody here can guess what the problem is...

First guess would have been that STDCALL is not the right way to call a proc from C - but you wrote that it works if you omit the fpu stuff... ::)

Best, jj

[robbje]

you can always guess ;)

Calculates pDst = pMat x pVec

Code Select Expand


_mulMatrixVector proc STDCALL pDst:DWORD, pMat:DWORD, pVec:DWORD, rowcount:DWORD, colcount:DWORD
mov edx, [pMat]
mov ebx, [pDst]
mov ecx, rowcount
rows:
    fldz
    push ecx
    push ebx
    mov ebx, [pVec]
    mov ecx, colcount
    cols:
        fldz
        fadd qword ptr [edx]
        fmul qword ptr [ebx]
        faddp st(1), st
        add edx, 8
        add ebx, 8
    loop cols
    pop ebx
    fstp qword ptr [ebx]
    add ebx, 8
    pop ecx
loop rows
ret
_mulMatrixVector endp


Olly says the error is that trying to read [edx] at the faddition causes access violation.
What bugs me is that this happens after the routine worked for 4 times without error.

[jj2007]

What is the value of ecx?
Does edx point to valid memory?
Does esp slowly creep after each call to the routine?
...?

[robbje]

ecx shows normal behaviour, then suddenly goes 33333333 ;O
edx does not point to valid memory anymore
seems like esp is creeping by 14 every call...

did not find out why yet...
olly shows that at the beginning of the routing it does:

Code Select Expand


push ebp
mov ebp, esp

and LEAVE at the end...

I can not see an inconsistency in my code. Probably I'm not aware
of calling functions and what it means to the stack. Anyone can
enlighten me?

[MichaelW]

Calling the procedure from assembly code the only way I can find to trigger an exception is by specifying values for the row and column counts that cause the procedure to access outside of my test data. Clearing the FPU interrupt masks showed no problems.

If you are calling the procedure from C your prototype needs to specify the __stdcall attribute.

[robbje]

my C prototype looks like this:

extern "C" void _mulMatrixVector(double*, double*, double*, int, int);

compiler is
gcc (GCC) 3.4.2 (mingw-special)
running on vista

edit: do i have to pop stuff manually from the stack?

[MichaelW]

No time to test this, but I would guess that it should be:

extern void __stdcall  _mulMatrixVector(double*, double*, double*, int, int);

For the STDCALL calling convention the called procedure removes the parameters from the stack. For the C calling convention the calling code removes the parameters from the stack.

[jj2007]

Or the other way round, proc C pDst:...
You have 5 args, and esp creeps by 14h=20 per call...

[robbje]

proc C did the trick,

extern void __stdcall _mulMatrixVector( ... );
gives undefined reference...