Issue with MASM 6.14.8444

[Randall Hyde]

Hi All,

I've been beating my brains out over this one this whole weekend. The end result is that I do believe I've found a pretty serious defect in MASM 6.14.8444. I've got a test program that checks out the object code generation for HLA and compares the results from compiling comparable programs under HLA, MASM6, MASM7, FASM, and Gas. If I ever break anything in HLA, I generally find the problem when comparing the object code output against the other assembler's output.

Comparing the output is quite a challenge, because each different assembler has its own ideas about how instructions ought to be encoded (encoding is ambiguous, if you didn't know that already). I solved this problem by modifying HLA to emit different object code based upon whatever assembler's object file I'm going to compare it against. For example, other than a few issues with FASM 1.66, my object code emission for the entire instruction set matches that produced by FASM v1.66 (in the FASM output compatible mode).

This weekend, I was working on the MASM6 compatible output mode in order to verify my code against MASM 6 (v6.14.8444). I started running into all kinds of problems. Most of the problems had to do with the "unusual" decisions Microsoft made with respect to instruction and operand encoding. Okay, no biggie, just duplicate what MASM did. However, as the testing proceeded, I came across some differences that were just flat-out bad encoding by Microsoft.

The current problem I encountered, which MASM 6 users should be aware of, is that the OMF->COFF conversion performed by LINK v6.00.8168 seems to be defective.  Specifically, I have the following HLA instructions

Code Select Expand


adc( sgb[ebx+edi+1], cl );
adc( sgb[ebx+edi-1], cl );
adc( sgb[ebx+edi*1+1], cl );
adc( sgb[ebx+edi*1-1], cl );
adc( sgb[ebx+edi*2+1], cl );
adc( sgb[ebx+edi*2-1], cl );
adc( sgb[ebx+edi*4+1], cl );
adc( sgb[ebx+edi*4-1], cl );
adc( sgb[ebx+edi*8+1], cl );
adc( sgb[ebx+edi*8-1], cl );

(as part of a *much* larger program) and the following, corresponding, MASM instructions:

Code Select Expand


adc cl, sgb[ebx+edi+1]
adc cl, sgb[ebx+edi-1]
adc cl, sgb[ebx+edi*1+1]
adc cl, sgb[ebx+edi*1-1]
adc cl, sgb[ebx+edi*2+1]
adc cl, sgb[ebx+edi*2-1]
adc cl, sgb[ebx+edi*4+1]
adc cl, sgb[ebx+edi*4-1]
adc cl, sgb[ebx+edi*8+1]
adc cl, sgb[ebx+edi*8-1]
adc cl, sgb[ebx+edi+1]
adc cl, sgb[ebx+edi-1]


After the OMF->COFF conversion performed by the linker, the relative address for sgb (a static byte object declared in the data section) is incorrect for the adc cl, sgb[ebx+edi*4-1] instruction.  Several other instructions in this test file break in similar ways.

In the past, I've recommeded to people who've complained about MASM v6's performance when assembling large static arrays to produce OMF files and let the linker translate those to COFF (which is much faster). Given my experiences of this past weekend, I'm not sure I can continue to make this recommendation.

Perhaps this problem has been addressed in a later version of the linker, I don't know.  Just be careful when producing OMF files that the linker translates to COFF.
hLater,
Randy Hyde