Something to watch for in X64 builds

[donkey]

I was recently building an application in 64 bit that used a RECT structure and passed it to a procedure for processing. This is something that is done quite often but instead of passing it by reference I needed each member of the structure separated so it looked something like this:

invoke MyFunc, [rect.left], [rect.top], [rect.right], [rect.bottom]

A pretty straightforward function call however in X64 mode there is a catch, since parameters are now 64 bit the actual parameters passed are as follows:

Parameter 1: [rect.left] + ([rect.top]<<32)
Parameter 2: [rect.top] + ([rect.right]<<32)
Parameter 3: [rect.right] + ([rect.bottom]<<32)
Parameter 4: [rect.bottom] + (?????<<32)

For Parameter 4 the high order DWORD is anything that happens to follow your RECT structure in memory, if you're unlucky enough to cross a page boundary into an unallocated page you will get an access violation and your program could crash mercilessly. However, the problem also rears its ugly head when you mov one of the parameters into a 64 bit register and don't take into account that the parameter size is a DWORD and the high order 32 bits must be masked out. I admit scratching my head over why my program was crashing in 64 bit builds but worked perfectly in a 32 bit build., after all when looking at the source everything seemed fine. Since we're not allowed to use TYPE indicators in an invoke statement we can't "cast" the 32 bit parameter to 64 bit so we just have to be careful to pay attention to the TYPE of the parameter that is passed and adjust it accordingly before we attempt to use it.

Hope this saves someone a few headaches when debugging 64 bit code.

Edgar

[Yuri]

Fortunately I figured that out (or read somewhere) before anything crashed. I try to always use 32-bit registers for 32-bit params. Unfortunately GoAsm only warns you if RDX is used as the first param in invoke (the value in it would be overwritten by the second parameter put in it), but it doesn't do so if you use EDX. So I have to keep this in mind. This little bug once caused me a really big headache. :bdg

[donkey]

Hi Yuri,

I didn't run into the page fault problem but did have the problem with the high order dwords. Luckily the Windows API appears to truncate the data before using it so we only have to watch it in our own programs.