NASM to MASM, What am I doing wrong?

[chrios]

I'm extremely new to ASM and I'm learning on my own (not a student). I'm trying to convert a Hash function I found to a MASM procedure.  So far it will assemble but crashes my app when it's invoked.  Can anyone please help me figure out what I'm doing wrong?
Any help is greatly appreciated.

The original NASM code can be found here, about half way down the page.
"Fast and Good Hash Function"
http://www.ntecs.de/old-hp/uu9r/lang/html/assembler.en.html

Code Select Expand

mix_line MACRO P1,P2,P3,P4,P5,P6
    sub P1, P2
    sub P1, P3
    mov P5, P3
ifidni <P6>,<left>
        shl P5, P4
else
        shr P5, P4
endif
    xor P1, P5
ENDM

mix MACRO P1,P2,P3,P4
    mix_line P1, P2, P3, 13, P4, <right>
    mix_line P2, P3, P1, 8, P4, <left>
    mix_line P3, P1, P2, 13, P4, <right>
    mix_line P1, P2, P3, 12, P4, <right>
    mix_line P2, P3, P1, 16, P4, <left>
    mix_line P3, P1, P2, 5, P4, <right>
    mix_line P1, P2, P3, 3, P4, <right>
    mix_line P2, P3, P1, 10, P4, <left>
    mix_line P3, P1, P2, 15, P4, <right>
ENDM

Code Select Expand

Hash proc key:DWORD, Len:DWORD, Level:DWORD

;temp textequ <edx>        ; temporary register: edx
;a textequ <ecx>           ; a
;b textequ <ebx>           ; b
;c textequ <eax>           ; c
;len textequ <edi>         ; len
;key textequ <esi>         ; key

push    ebp
mov     ebp, esp
push    esi
push    edi
push    ebx
push    ecx
push    edx

; --------------------------------------------------------

mov     esi, [ebp+8]            ; first argument (key)
mov     edi, [ebp+12]           ; second argument (length)
mov     eax, [ebp+16] ; Level
mov     ecx, 9e3779b9h
mov     ebx, ecx

.while edi >= 12
add ecx, [esi]
add ebx, [esi+4]
add eax, [esi+8]

mix ecx, ebx, eax, edx

add esi, 12
sub edi, 12
.endw

add eax, [ebp+12]

SWITCH edi
CASE 11
add ecx, [esi]
add ebx, [esi+4]
mov edx, [esi+8]
shl edx, 8
add eax, edx
CASE 10
add ecx, [esi]
add ebx, [esi+4]
mov edx, [esi+8]
shl edx, 8
and edx, 0FFFFFFh
add eax, edx
CASE 9
add ecx, [esi]
add ebx, [esi+4]
mov edx, [esi+8]
shl edx, 8
and edx, 0FFFFh
add eax, edx
CASE 8
add ecx, [esi]
add ebx, [esi+4]
CASE 7
add ecx, [esi]
mov edx, [esi+4]
and edx, 0FFFFFFh
add ebx, edx
CASE 6
add ecx, [esi]
mov edx, [esi+4]
and edx, 0FFFFh
add ebx, edx
CASE 5
add ecx, [esi]
mov edx, [esi+4]
and edx, 0FFh
add ebx, edx
CASE 4
add ecx, [esi]
CASE 3
mov edx, [esi]
and edx, 0FFFFFFh
add ecx, edx
CASE 2
mov edx, [esi]
and edx, 0FFFFh
add ecx, edx
CASE 1
mov edx, [esi]
and edx, 0FFh
add ecx, edx
ENDSW

mix ecx, ebx, eax, edx


; return value in eax
;ifnidni c,eax
; mov eax, eax
;endif

; --------------------------------------------------------
pop     edx
pop     ecx
pop     ebx
pop     edi
pop     esi
pop     ebp
ret
Hash endp

[dedndave]

the masm assembler generates a prologue and epilogue
you need to either let it do its' thing or turn it off

turning it off:

Code Select Expand

        OPTION  PROLOGUE:None
        OPTION  EPILOGUE:None

Hash    PROC    key:DWORD, Len:DWORD, Level:DWORD

        push    ebp
        mov     ebp,esp
        push    esi
        push    edi
        push    ebx
        push    ecx
        push    edx

        mov     esi,[ebp+8]            ; first argument (key)
        mov     edi,[ebp+12]           ; second argument (length)
        mov     eax,[ebp+16]           ; Level
;
;
;
        pop     edx
        pop     ecx
        pop     ebx
        pop     edi
        pop     esi
        pop     ebp
        ret     12

Hash    ENDP

        OPTION  PROLOGUE:PrologueDef
        OPTION  EPILOGUE:EpilogueDef

here, the assembler PUSH's and POP's the registers, sets up the stack frame, and determines the operand for RET N

Code Select Expand

Hash    PROC    uses esi edi ebx ecx edx key:DWORD, Len:DWORD, Level:DWORD

        mov     esi,key                ; first argument (key)
        mov     edi,Len                ; second argument (length)
        mov     eax,Level              ; Level
;
;
;
        add     eax,Len
;
;
;
        ret

Hash    ENDP

no matter which method you use, you will want a PROTOtype near the beginning of the source file:

Code Select Expand

Hash    PROTO   :DWORD,:DWORD,:DWORD

then, you may call the function:

Code Select Expand

        INVOKE  Hash,key,Len,Level

[dedndave]

corrected typo = EILOGUE   :P

[dedndave]

also, i missed this one

Code Select Expand

        add     eax,Len

for win32 functions, EAX, ECX, and EDX are not usually preserved
it won't hurt anything to preserve ECX and EDX, though

i would rename the "key" parameter to "lpKey" because it's a pointer

[chrios]

Thanks for your reply.

I have tried playing around with the info you gave me.  I don't really understand what  Option  Prologue/Epilogue are for but I did add them and still my app crashes. Maybe the problem is somewhere besides the hash routine I don't see the part where it says "add eax, len".
Some other things came up so I may not be able to work on this until this weekend.   I'll look up Option Prologue/Epilogue to learn about them as this is the second time they have come up in my learning about asm.

I'll spend some more time on it to see if I can figure it out before I post back, but thanks again for the info.

[dedndave]

i made an include file out of your code...

Code Select Expand

;http://www.ntecs.de/old-hp/uu9r/lang/html/assembler.en.html

;--------------------------------------------------------------------------------------------------------

Hash    PROTO   :DWORD,:DWORD,:DWORD

;--------------------------------------------------------------------------------------------------------

mix_line MACRO  P1,P2,P3,P4,P5,P6

        sub     P1,P2
        sub     P1,P3
        mov     P5,P3

    ifidni  <P6>,<left>
        shl     P5,P4
    else
        shr     P5,P4
    endif

        xor     P1,P5

        ENDM

;--------------------------------------------------------------------------------------------------------

mix     MACRO   P1,P2,P3,P4

        mix_line P1,P2,P3,13,P4,<right>
        mix_line P2,P3,P1, 8,P4,<left>
        mix_line P3,P1,P2,13,P4,<right>
        mix_line P1,P2,P3,12,P4,<right>
        mix_line P2,P3,P1,16,P4,<left>
        mix_line P3,P1,P2, 5,P4,<right>
        mix_line P1,P2,P3, 3,P4,<right>
        mix_line P2,P3,P1,10,P4,<left>
        mix_line P3,P1,P2,15,P4,<right>

        ENDM

;--------------------------------------------------------------------------------------------------------

Hash    PROC USES EBX ESI EDI lpKey:DWORD,Len:DWORD,Level:DWORD

;temp   textequ <edx>              ; temporary register: edx
;a      textequ <ecx>              ; a
;b      textequ <ebx>              ; b
;c      textequ <eax>              ; c
;len    textequ <edi>              ; len
;key    textequ <esi>              ; key

; --------------------------------------------------------

        mov     esi,lpKey          ; first argument (key)
        mov     edi,Len            ; second argument (length)
        mov     eax,Level          ; Level
        mov     ecx,9E3779B9h
        mov     ebx,ecx

    .while  edi >= 12
        add     ecx,[esi]
        add     ebx,[esi+4]
        add     eax,[esi+8]
        mix     ecx,ebx,eax,edx
        add     esi,12
        sub     edi,12
    .endw

        add     eax,Len

SWITCH  edi

    CASE    11
        add     ecx,[esi]
        add     ebx,[esi+4]
        mov     edx,[esi+8]
        shl     edx,8
        add     eax,edx

    CASE    10
        add     ecx,[esi]
        add     ebx,[esi+4]
        mov     edx,[esi+8]
        shl     edx,8
        and     edx,0FFFFFFh
        add     eax,edx

    CASE    9
        add     ecx,[esi]
        add     ebx,[esi+4]
        mov     edx,[esi+8]
        shl     edx,8
        and     edx,0FFFFh
        add     eax,edx

    CASE    8
        add     ecx,[esi]
        add     ebx,[esi+4]

    CASE    7
        add     ecx,[esi]
        mov     edx,[esi+4]
        and     edx,0FFFFFFh
        add     ebx,edx

    CASE    6
        add     ecx,[esi]
        mov     edx,[esi+4]
        and     edx,0FFFFh
        add     ebx,edx

    CASE    5
        add     ecx,[esi]
        mov     edx,[esi+4]
        and     edx,0FFh
        add     ebx,edx

    CASE    4
        add ecx,[esi]

    CASE    3
        mov     edx,[esi]
        and     edx,0FFFFFFh
        add     ecx,edx

    CASE    2
        mov     edx,[esi]
        and     edx,0FFFFh
        add     ecx,edx

    CASE   1
        mov     edx,[esi]
        and     edx,0FFh
        add     ecx,edx

ENDSW

        mix     ecx,ebx,eax,edx
        ret

Hash    ENDP

;--------------------------------------------------------------------------------------------------------

the form is correct - not sure about the code   :P
i did not preserve ECX or EDX

this stuff is kind of old
if you search the forum, you can find some SSE algorithms that will be much faster

[hutch--]

chrios,

If you can build it in NASM, try marking the file with a row of NOPS before and after it, make sure it builds then disassemble it to get the opcodes in MASM format. What I am not sure about is if the version of NASM dated 1996 is still the same as the current versions, that is in the macros and opcode generation.

Something that is probably useful to know as I only had a quick look at the source, what is the hash used for ? Crypto, hash table etc ....

[Tedd]

This appears to work (not crash)..

Code Select Expand

.586
.model flat, stdcall
option casemap:none
include windows.inc
include kernel32.inc
includelib kernel32.lib

;***************************************************************************************************

Hash proto pKey:DWORD,Len:DWORD,Level:DWORD

;***************************************************************************************************

.const
switch_table    dd OFFSET case_00,OFFSET case_01,OFFSET case_02,OFFSET case_03,OFFSET case_04
                dd OFFSET case_05,OFFSET case_06,OFFSET case_07,OFFSET case_08,OFFSET case_09
                dd OFFSET case_10,OFFSET case_11

testkey         db "abcdefgh"

.code
start:
    invoke Hash, ADDR testkey,SIZEOF testkey,3

    invoke ExitProcess, NULL

;***************************************************************************************************

mix_line MACRO P1,P2,P3,P4,P5,P6
    sub P1,P2
    sub P1,P3
    mov P5,P3
    IFIDNI <P6>,<left>
        shl P5,P4
    ELSE
        shr P5,P4
    ENDIF
    xor P1,P5
ENDM

mix MACRO P1,P2,P3,P4
    mix_line P1,P2,P3,13,P4,<right>
    mix_line P2,P3,P1,8,P4,<left>
    mix_line P3,P1,P2,13,P4,<right>
    mix_line P1,P2,P3,12,P4,<right>
    mix_line P2,P3,P1,16,P4,<left>
    mix_line P3,P1,P2,5,P4,<right>
    mix_line P1,P2,P3,3,P4,<right>
    mix_line P2,P3,P1,10,P4,<left>
    mix_line P3,P1,P2,15,P4,<right>
ENDM

;***************************************************************************************************

Hash proc pKey:DWORD,Len:DWORD,Level:DWORD
    push esi
    push edi
    push ebx

    mov esi,pKey
    mov edi,Len
    mov eax,Level
    mov ecx,9e3779b9h
    mov ebx,ecx

    .WHILE (edi >= 12)
        add ecx,[esi]
        add ebx,[esi+4]
        add eax,[esi+8]

        mix ecx,ebx,eax,edx

        add esi,12
        sub edi,12
    .ENDW

    add eax,Len

    ;fall-through is necessary (i.e. no 'break' in each case)
    jmp DWORD PTR [switch_table + 4*edi]

    case_11 LABEL DWORD
        add ecx,[esi]
        add ebx,[esi+4]
        mov edx,[esi+8]
        shl edx,8
        add eax,edx
    case_10 LABEL DWORD
        add ecx,[esi]
        add ebx,[esi+4]
        mov edx,[esi+8]
        shl edx,8
        and edx,0FFFFFFh
        add eax,edx
    case_09 LABEL DWORD
        add ecx,[esi]
        add ebx,[esi+4]
        mov edx,[esi+8]
        shl edx,8
        and edx,0FFFFh
        add eax,edx
    case_08 LABEL DWORD
        add ecx,[esi]
        add ebx,[esi+4]
    case_07 LABEL DWORD
        add ecx,[esi]
        mov edx,[esi+4]
        and edx,0FFFFFFh
        add ebx,edx
    case_06 LABEL DWORD
        add ecx,[esi]
        mov edx,[esi+4]
        and edx,0FFFFh
        add ebx,edx
    case_05 LABEL DWORD
        add ecx,[esi]
        mov edx,[esi+4]
        and edx,0FFh
        add ebx,edx
    case_04 LABEL DWORD
        add ecx, [esi]
    case_03 LABEL DWORD
        mov edx,[esi]
        and edx,0FFFFFFh
        add ecx,edx
    case_02 LABEL DWORD
        mov edx,[esi]
        and edx,0FFFFh
        add ecx,edx
    case_01 LABEL DWORD
        mov edx,[esi]
        and edx,0FFh
        add ecx,edx
    case_00 LABEL DWORD
        ;do nothing

    mix ecx,ebx,eax,edx

    pop     ebx
    pop     edi
    pop     esi
    ret
Hash endp

;***************************************************************************************************

end start

There's a confusion caused by mixing up 'len' and 'length' so it would need further checking.
Also, the nasm version is incorrect with reference to the original C version (http://burtleburtle.net/bob/c/lookupa.c)
So it might be a nice learning experience to convert it yourself; and of course we can help you with that :wink

[drizz]

None of the assembly here and there(NASM version) is correct as it all reads past the end of the key.

Code Select Expand

add ecx,[esi]
add ebx,[esi+4]
mov edx,[esi+8] ;; <-------- if key is 11 more bytes, reading 12 bytes is invalid !


Besides, is there a point to having an ASM version?  Just remove endianess handling from original c code and you'll have 1:1 correspondence of generated asm code vs translated asm code.

[dedndave]

Code Select Expand

    .WHILE (edi >= 12)  ;<--------- i think that covers it
        add ecx,[esi]
        add ebx,[esi+4]
        add eax,[esi+8]

        mix ecx,ebx,eax,edx

        add esi,12
        sub edi,12
    .ENDW

[drizz]

Code Select Expand

    .WHILE (edi >= 12)  ;<--------- i think that covers it

Dave, the remaining 1-11 bytes handling code (inside switch-case/jump table).

[Tedd]

I think the assumption is that the key will be aligned to 4 bytes, and so accessing it in dword chunks won't be a problem (the trailing bytes are still masked out and not used in calculating the result.)

But, I don't claim what I posted to be 'correct,' it was just a fixed-up version of what chrios posted.
Like I said, it would end up being less trouble converting directly from the C version.

[chrios]

Something that is probably useful to know as I only had a quick look at the source, what is the hash used for ? Crypto, hash table etc ....

I was planning to use this for copy protection.  I am getting short strings of info about the PC running my program with WMI such as HDD Serial Number, Processor ID and Windows Serial Number, etc.  and was going to use this hash function to convert them into dwords so they are easier to use by the rest of the copy protection code.  My program is only going to be used within the company and not widely distributed so I don't want to make it overly complicated.  If there a better hash function then I would like to hear any suggestions.  This one sounded good but I don't want to have to worry about the length of the strings I use, and wouldn't want to spend the time to convert the C version if there is a better hash function I can use.

I really appreciate all of the input you guys have given me so far.