News:

MASM32 SDK Description, downloads and other helpful links
MASM32.com New Forum Link
masmforum WebSite

Calling a dll with parameters

Started by PeterRod, November 14, 2008, 04:28:15 PM

Previous topic - Next topic

PeterRod

Hi,
I am trying to call an exported function of a MASM dll that has two parameters.
The call is being made from managed code in a vb.net app.
I can successfully call the exported function from another MASM app using LoadLibrary and GetProcAddress etc

I can also call the exported function from vb.net if I change it to have no parameters.

I can step thru the dll code in VS2005 after the managed call.

This is the exported function:

;----------------------------------------------------
Func1 proc  Param1:DWORD , Param2:DWORD
            mov edi, Param1      
            mov eax,6789
            mov edi, Param2
            mov dword ptr [edi],eax
            mov eax,99
            ret
Func1    endp
;-------------------------------------------------------
Func1 does not really do much as I am just testing

This is the vb.net code

Private Declare Ansi Function Func1 Lib "C:\Documents and Settings\PETER\My Documents\Visual Studio 2005\Projects\AsmPgms\debug\Dynamo.dll" (ByVal p1 As IntPtr, ByRef p2 As IntPtr) As Integer

        Dim st As String
        Dim sP, Rnk As IntPtr
        Dim i As Integer
        st = "Some Text"
        sP = Marshal.StringToHGlobalAnsi(st)
        Rnk = Marshal.AllocHGlobal(4)
        Marshal.WriteInt32(Rnk, 65)
        i = Func1(sP, Rnk)
        i = i
       
After the call to Func1(sP, Rnk) I can step thru the disassembly code of Func1.
MASM puts out the prolog code of 'push ebp'   and       'mov ebp,esp'
the parameter values in Func1 are correct and point to the correct memory locations.

MASM puts out the epilog code of    'leave'     and               'ret 8'
It is on the return that things get messed up.
Func1 does not return to the calling app. The vb.net app crashes.

I tried defining  Func1 proc as STDCALL ,  C ,  BASIC   but no success.

If I chage Func1 to a function without parameters the call and the return succeeds.

I dont know what else to try!!!

Can an exported function of a MASM dll with parameters be called from managed code??

Thanks for any help

PeterRod


Mirno

Firstly edi is a reserved register, and should be preserved (add a "uses edi" to your proc, or change it to ecx, or edx). This could in theory be throwing .net off.

Secondly check what got pushed on the stack for the call, and make sure it's what you expect (including the return address). Also check what happens immediately after the return - it'll give you a hint as to the calling convention.

p1 seems to not be used (you copy it's value to edi, then overwrite it almost immediately after)... Take it out of your app for the moment, it'll only be confusing the issue.

Mirno

PeterRod

Thank you Mirno. It was the edi problem.
I simply saved it and now both call & return are ok.
Is there any reading material on dll's and managed .Net code?

Also the '  mov edi, Param1 ' is only there so I could check in the debugger that the passed pointer Param1 was indeed pointing to the correct mem location.

Thanks again
PeterRod