Odd calling behavior from VB

[Merrick]

This is really a VB question, but I thought I'd drop it here to see if anyone had any ideas...

I have large image data sets (typically 16 bits/pixel, 200x200 pixel images, 128 images in pairs (image and background data)
that are usually 20MB raw (or multiples of 20MB).

Sadly, this data is gathered and saved by (evil, vile, contemptible) LabView, and is therefore Big Endian. To display an image the following operations are performed:

retreive data into 20MB array if integers (2 bytes)
convert Big Endian integers into Little Endian longs (4 bytes, required since the 16 bit integers are unsigned)
calculate contrast image values (real), defined as:
    (image(x pixel, y pixel, image1) - image(x pixel, y pixel, background1)) / image(x pixel, y pixel, background1)
scale real values in each contrast image for display

So, I:
call the API to read the file into an array
wrote an assembly routine to convert the ints to longs
wrote an assembly routine to calculate the contrast images
wrote an assembly routine to find the max and min values in each contrast image for scaling

The first assembly routine is performed on the whole array in one call since the array can be treated as linear at this point.
I got a huge headache trying to work out the multidimensional arrays, so the following two steps are performed one image at a time and are looped for all images in VB.

All of this works perfectly... however...

When you get this all working, you'd naturally move it all into a subroutine, which I did. And here's the part I'm having a hard time with. It all works perfectly in the main routine. When I move it to a subroutine is crashes (literally). But upon further examination, it turns out that I can move everything EXCEPT the max/min routine to the subroutine and everything works perfectly.

Any ideas on why this behaves ths way?

Thanks,
Merrick

[Merrick]

Here's the routine for converting Big Endian ints to Little Endian longs:

Code Select Expand

IntToLongArray proc arrayCount:dword, intInput:dword, longOutput:dword
mov ecx, arrayCount
test ecx, ecx
je doneRotate
nextRotate:
mov edx, intInput
mov ax, word ptr[edx+2*ecx-2]
rol ax, 8
cwde
cmp eax, 0
jge noAdd
add eax, 65536
noAdd:
mov edx, longOutput
mov dword ptr[edx+4*ecx-4], eax
dec ecx
test ecx, ecx
je doneRotate
jmp nextRotate
doneRotate:
ret
IntToLongArray    ENDP

And the code for calculating the contrasts:

Code Select Expand

LongToContrast2 proc longArray:dword, doubleArray:dword
finit
mov ecx, 40000 ;40,000 pixels per image
nextPixel:
mov eax, ecx ;load current pixel offset
shl eax, 2 ;4 bytes/pixel
mov edx, longArray
fild dword ptr[edx+eax-4] ;load image pixel
add eax, 160000 ;background pixel is always 160,000 bytes (40,000 pixels)
;higher in the array than its corresponding image pixel
fidiv dword ptr[edx+eax-4] ;divide by background pixel
fld1 ;subtract 1 --> shift to contrast frame
fsub
mov eax, ecx ;load current pixel offset
shl eax, 3 ;8 bytes/pixel
mov edx, doubleArray
fstp qword ptr[edx+eax-8] ;load contrast into array
dec ecx ;decrement to next pixel
test ecx, ecx
je lastPixel ;if pixel 0 --> quit
jmp nextPixel ;if pixel n --> repeat
lastPixel:
ret
LongToContrast2 ENDP

And the code for calculating the max/min values:

Code Select Expand

MaxMin proc doubleArray:dword, minValue:dword, maxValue:dword, dummy:dword
finit
mov edx, doubleArray
fld qword ptr[edx] ;load first value into st[1] (min) and
fld qword ptr[edx] ;st[2] (max) to initialize state
mov ecx, 40000 ;40,000 (200x200) pixels per image
nextPixel:
mov eax, ecx ;load current pixel offset
shl eax, 3 ;8 bytes/pixel
fld qword ptr[edx+eax-8] ;load current pixel value into st[0]
fcom st[1] ;compare st[0] to st[1] (min location)
fstsw ax
fwait
sahf
ja greaterPixel ;if st[0] > st[1] -> do second comparison
mov eax, minValue
fst qword ptr[eax] ;if st[0]<=st[1] store st[0] in minValue
fxch st[1] ;place st[0] in st[1] (new minimum)
jmp doneCompare
greaterPixel:
fcom st[2] ;compare st[0] to st[2] (max location)
fstsw ax
fwait
sahf
jb doneCompare ;if st[0] < st[2] -> done
mov eax, maxValue
fst qword ptr[eax] ;if st[0]>=st[2] store st[0] in maxValue
fxch st[2] ;place st[0] in st[2] (new maximum)
doneCompare:
mov eax, dummy
fstp qword ptr[dummy] ;pop old value off stack
dec ecx ;decrement to next pixel
test ecx, ecx
je lastPixel ;if pixel 0 --> quit
jmp nextPixel ;if pixel n --> repeat
lastPixel:
ret
MaxMin  ENDP

Sorry, but I can't get the comments to line up right. The browser seems to understand and correctly align tab characters to the left of the code, but the tab characters between the code and the comments are ignored. Any helpful suggestions (about my problem, or the formatting)?

Thanks,
Merrick

[liquidsilver]

I wouldn't know how to help with the main problem, but to do with the formatting problem...

Instead of tabs, use spaces, as your editor and browser are treating the tab character as different numbers of spaces. I've had similiar problems. Most editors use tabs as 4 spaces and I think the browser uses 8. I don't know how to expain it better. The best would be to do a "test" on a few line of different lengths.

[dsouza123]

Have you tried testing MaxMin substituting 40000
with a lower upper bound ex 30000
then instead of decrementing down to 0 with ecx
drop down just a small amount ex 29990.

If it works it is likely an out of range access.

Why all the flag storing/reloading ?
fstsw ax
sahf

Maybe the direction flag or some other flag(s)
that windows expects a certain way gets messed up.

Can always try Ollydbg for debugging,
it is very good assembly level debugger.