How Now? (GoLink, GoBug anomaly)

[Paul]

All,

I try to use Jeremy's Go Tools in my professional work. I am lately putting them through their paces. I encountered something quite anomalous and would like some feedback on the phenomena.

In fine, it seems that the position of a .obj file in the linker input file is causing some strange behavior.

When I GoLink with the object files in this sequence:

drddb.obj
ddb_format.obj
ddb_util.obj
ddb_delete.obj
ddb_insertion.obj
ddb_search.obj
ddb_CIUtil.obj
ddb_init.obj

everything works fine. But if I move file 'ddb_format.obj' to any other position, the C code throws an exception. I used GoBug to examine the failure case vs. the normal case. Some background info: the access exception occurs in the CreateFile called by this function snippet:

DDB_fhandle open_data_File(const char* datafileName){
   DDB_fhandle hf;
   hf = CreateFile(datafileName,GENERIC_READ | GENERIC_WRITE,0,0,CREATE_NEW,0,0);

This function is called by another function snippet:

int format_dtFile(int data_CI_size){
   int err = DDB_FAIL;
   DDB_fhandle dthfile;
   dthfile = open_data_File("data.btr");

I dumped the pane in the failure case and in the success case for function format_dtFile. Here is the failure case:

    ~~~format_dtFile:~~~>
    405A50: PUSH EBP
    405A51: MOV EBP,ESP
    405A53: SUB ESP,8
    405A56: MOV D[EBP-8],0
    405A5D: PUSH 14
    405A62: CALL open_data_File

Here is the success case:

   ~~~format_dtFile:~~~>
    4022E0: PUSH EBP
    4022E1: MOV EBP,ESP
    4022E3: SUB ESP,8
    4022E6: MOV D[EBP-8],0
    4022ED: PUSH 407014
    4022F2: CALL open_data_File

Please notice that in the failure case, the final PUSH before the CALL open_data_File pushes a constant 0x14. But in the success case it pushes a constant 0x407014. The This argument is supposed to point to the string that names the file being opened.

I am just now noticing the presence of the 0x14 in each constant and am wondering if, in the failure case, GoLink failed to relocate an address constant.

In any event, I can make this problem go away by placing ddb_format.obj near the very top of the list of .obj files (immediately after my GoAsm-based drddb.obj). But I am concerned that other address constants might not be getting relocated, and that it's only a matter of time before I hit another problem.

Any thoughts? Thanks.

Cordially,

Paul

[donkey]

It looks like a label that is unresolved because it is forward referenced, I am not sure of the mechanics of how GoLink handles these but that's my guess. At least it appears that way, by shuffling around the OBJ file order you take care of the forward referencing and the problem disappears. You would think GoLink would throw an error if this sort of thing happened though. The only other possibility I can think of is that somehow the label being pushed (probably hInstance in this case) is being redefined in one of the later OBJ files.

[jorgon]

Hi Paul

It is possible that this problem is related to the one identified by Mitchi and Beatrix in recent postings, that GoLink is not finding certain symbols emitted by a "C" compiler.
From the files they have posted, these are "common symbols" which are symbols which which are not actually defined in the source, but which need to be given an allocation in unitialised memory at link time by the linker.  As I understand it, all the data in such common symbols is merged [a symbol with the same name in different modules will occupy the same place in the resulting exe, although they may be different sizes and may not be defined (declared) at all].  In this respect they act like global symbols but the difference is that they are not actually declared anywhere.  It also means that data can be written over because of the merger, so it is possible this is happening as Donkey says.

As I understand it, you can intentionally make common symbols with the GNU assembler and with MASM using COMM (although I can't think of any reason to do so), and I now understand from recent postings that a "C" compiler will use these "behind your back" if you try to create a data reference using something like

Code Select Expand

int MyVar;

I do intend to adjust GoLink to support common symbols using the /mix switch since it will not link "C" object files properly without this, but I need to sleep on it to decide the best way to do this.

So that I can check whether your problem is related to this, I wonder if you would kindly to post or send me directly your files so that I can have a look at them, including the success and fail executables?  Many thanks.

[jorgon]

Paul

Thanks for sending your files.

This problem is related to your report that GoLink was complaining about duplicate symbols which were marked as "STATIC" in your object files.
See http://www.masm32.com/board/index.php?topic=10658.0.

I was wrong to allow such duplicates because your data was being written over using GoLink 0.26.10 beta.
So I have reinstated the error message when such duplicates are found, but reduced it for the time being to a warning.  See http://www.masm32.com/board/index.php?topic=10770.0

You can see that there are several duplicate symbols in your files in the form $SGxxxxx
These are all STATIC symbols, but are all causing problems.

Do you have any insight into how these should be handled by the linker, or maybe avoided altogether in your source?

[jorgon]

Actually Paul, you might like to try the attached.

This version of GoLink when doing the symbol fixups, simply ignores any labels marked IMAGE_SYM_CLASS_STATIC and IMAGE_SYM_CLASS_LABEL.
I think this might work ok because I have not seen any references to such symbols in any of the "C" compiler object files sent to me.  It seems to me therefore that there may well be no fixing up to do with these types of labels.

This only applies to the /mix switch.



[attachment deleted by admin]

[Paul]

Hi Jeremy,

I finally got to test GoLink 0.26.10 beta4.

My observations:

1. If I link without '-mix' GoLink complains about the duplicate symbols ("The following STATIC symbols were defined more than once") and creates the output executable despite displaying on console the message "Output file not made."
2. If I link with '-mix' GoLink does not complain about the duplicate symbols and creates the output executable (does not display the "Output file not made" message).
3. If I link with '-mix' the resulting executable enters an endless loop. I have no idea why. I have *never* seen this behavior and the code hasn't changed in at least one week.
4. If I link without '-mix' and leave ddb_format.obj 2nd in the linker's command file, the executable behaves as expected.
5. If I link without '-mix' and move ddb_format.obj further down in the linker's command file, the executable gets the same exception as noted earlier in this thread.

Thoughts:

Cases 4 and 5 conform to previously posted results obtained w/o using '-mix.' Case 3 is very troubling. I will try to see where the loop is occurring. A propos of this: can I use GoBug to "attach" to a running process?

Cordially,

Paul

[jorgon]

Paul

Please, you must use the /mix or -mix switch with these files.  They are "C" files and this is what the switch is for.

Concentrating therefore on your tests 2 and 3:-

2 is as expected.

As for 3, do you get an endless loop if you link with MS Link.exe?

On your question about GoBug, in order to break into an endless loop you need to start the exe under GoBug control and then press the "hot-key".

[Paul]

Jeremy,

I reassembled and linked using -mix. The first issue was the endless loop. I managed to find its cause in one of my wrapper functions. You might recall from previous on and offline conversations that I used these wrappers, e.g., "_malloc" to resolve references from the several C object modules. In these wrappers I simply jmp to the corresponding undecorated routine in msvcrt.dll, e.g.,

   _malloc:                 
   jmp malloc               ; call malloc in DLL

This worked fine until I added the -mix switch. But when I link with this switch the jmp instruction jumps to itself - as tight an endless a loop as one might want! I do recall your remark that -mix sees, say, '_malloc' and 'malloc' as identical.

So....I removed the wrapper functions and tried again. The results are almost identical to the phenomena I posted at the start of this thread. In fine. format_dtFile calls open_data_file with an 'unresolved' address constant whose value is 0x14. This argument should point to the string of the filename to open (see below).

I said 'almost' identical. The difference with -mix is that it does not matter where in the linker command file I place ddb_format.obj: I always get this error. At this point, I cannot use -mix. In fact, I can link and execute happily without it, so long as I place ddb_format.obj in a specific location within the command file.

Please let me know if you want the files.

Cordially,

Paul

  ~~~format_dtFile:~~~>
    4022E0: PUSH EBP
    4022E1: MOV EBP,ESP
    4022E3: SUB ESP,8
    4022E6: MOV D[EBP-8],0
    4022ED: PUSH 14
    4022F2: CALL open_data_File

[jorgon]

Please let me know if you want the files.

I don't think this would be any use since we know from your asm list that you have a PUSH 14 which is an error.

However, this must be an error in your source, rather than in GoLink.

Would you like to post your "C" source which is producing this asm list result, so that any "C" programmers amongst us can find the problem?

[Paul]

Sure thing. I'll do it when I get to work tomorrow. But I think it's pretty much what I posted in my initial post.

I'm no expert in C, but I am fairly certain that the problem lies elsewhere.

Cordially,

Paul

[jorgon]

Sorry Paul - I spoke in haste.

Having slept on this, I have now worked out what is going wrong, and I shall come back to you very soon with a fix.

[Paul]

Jeremy,

Any progress on this front?

Cordially,

Paul

[jorgon]

Paul

It's on its way ..

[jorgon]

Hi Paul

Sorry - I got waylaid in my day job.

The attached Version 26.20 beta5 should sort out your problems.

When you started this thread you said that you were putting the Go tools through their paces, but in fact GoLink was never designed as a linker for object files created by "C" compilers, so what you were asking for was an extension to the tools.  I have provided this facility through the "/mix" switch.  That is, if this latest fix works for what appears to be the one remaining issue - symbols of the IMAGE_SYM_CLASS_STATIC type and in fact also of the IMAGE_SYM_CLASS_LABEL type.  These are now treated by GoLink as labels limited to the module (object file) in which they appear if the /mix switch is used.  It appears that labels of these types are given names by the compiler, and it is possible for the names to be the same across modules.  In other words the compiler may use the same name in one module as it used in another module.  The linker must allocate a separate space in the exe for each of these declarations and not try to join them up as it would do for other symbols.






[attachment deleted by admin]

[Paul]

Jeremy,

It seems to work; thank you very much.

Cordially,

Paul