News:

MASM32 SDK Description, downloads and other helpful links
MASM32.com New Forum Link
masmforum WebSite

LNK2017 error with data tables in a dll

Started by HooKooDooKu, August 12, 2011, 01:58:48 PM

Previous topic - Next topic

HooKooDooKu

I've got some 32-bit C++ code I'm trying to upgrade to 64-bit.  The big problem is that the code includes some __asm blocks (some as large as 1200 lines).

I've succeded at moving all the code out of __asm blocks and relocating it in MASM PROCs that I can instead call.  I've even figured out how I've got to change some register names and deal with the fact that some parameters are passed in registers for x64.

But the one thing I can't seem to over-come is this linker error:
   'ADDR32' relocation to 'table' invalid without /LARGEADDRESSAWARE:NO

Actually I'm getting about 15 copies of this error as this code includes about 15 different lookup tables.  Now if this were a simple .exe, I could just turn LARGEADDRESSAWARE to NO, but the code is inside a dll.  The linker tells me the _DLL and /LARGEADDRESSAWARE:NO are mutually exclusive.

I have learned that I can make this problem go away by using fixed base addressing in the linker options.  But that's where I start to get in over my head.  The dll is just one of many dlls being used as a part of fairly complex system, and I'm clueless about what the ramifications would be to setting a fixed base address for this dll.

Surely I'm not the only one trying to include x64 assebly code in a dll with lookup tables (and based on internet searches, I'm not), but it seems like no one is finding any good answers.

I'm hoping someone in a MASM forum could provide some help (Experts Exchange was a dud).



HooKooDooKu

Well, a little more research has found AN answer (not as nice as I had hoped, but an answer).

It sounds like a part of the problem has to do with RIP addressing and how MASM automatically does it for you.  I had already seen this subject, but because it delt with making code jumps, not lookup tables, I didn't see how it applied to me.  But another look found that I've got to make a change to the asm code.

Basically, the existing code has several places where the code like this:
test    eax,table[4 * rcx] 

I've found that a solution to my problem is to replace this single line of code with something that looks like this:
lea      rbx,
text    eax, [rbx+4*rcx]