So, my first post looks like it's gonna be an avalanche of questions. I think some of these questions are a tad above the beginner level, but this forum seemed suitable as I'm learning the intricacies to working with MASM.
Now, the subject is a little misleading; I'm not having any trouble assembling or linking programs. What I want to know is some of the details about what goes on in each case.
1) Are module-definition files (.def) used by modern executables (as opposed to DLLs)? I want to know if I can check for the presence of a .def file before linking to heuristically determine if the object files should be linked as an EXE or a DLL. Basically, if .def files are more commonly used by EXEs than I've seen, I don't want the presence of a .def file to default to linknig as a DLL; otherwise, I do.
2) When you link multiple .obj files, does the order that they're sent to the linker matter? What about when you link files using a wildcard (*.obj for example)?
3) Why is rsrc.rc such a common name for a resource script file, rather than Project_Name.rc?
4) When comparing LINK.EXE and POLINK.EXE, it seems like LINK.EXE is giving the resource section 4 bytes more than it needs to (as specified in the PE header where the size of the .rsrc section is defined); is this the only major inefficiency in LINK.EXE, aside from the Rich header?
5) Why, among MASM32 projects, is it so common to specify includes with \masm32\etc. rather than using INCLUDE and LIB environment variables (set to the respective MASM32 folders), and then using path-less include and includelib entries in the ASM file?
I think that covers everything I can think of at the moment. I'm pretty new to MASM (well, assembly in general), but have had a pretty easy time working with it and have particularly enjoyed experimenting with building tiny executables and simple utilities. However, the more I've used it, the more I've wound up considering things like the questions above.
Queue
1) Perhaps, although it is possible to export functions/entry points from an EXE.
2) I don't think most application/tools accept wildcards unless linked against setargv.obj
The link order will define entry points, order of code/data within the file, and segment order and alignments. It's probably less problematic in 32-bit code, but things could get very weird with 16-bit code.
3) Hard to say, some applications have defaults, I've never used this naming.
4) What version, there are dozens on versions of LINK, and Microsoft has changed alignment and PE structure ordering over the years. LINK can make very compact files, however you might limit OS compatibility if you get stupid over a few 4K pages.
5) Probably because people don't use command lines like they used too, and the environment variables all seem to get cluttered. I host several compilers and assemblers, and use BAT files from the command line to set the appropriate environment for the specific tool chain. You should do what works for you, don't be bound to what others do.
Quote from: Queue on June 23, 2010, 09:24:35 PM
1) Are module-definition files (.def) used by modern executables (as opposed to DLLs)? I want to know if I can check for the presence of a .def file before linking to heuristically determine if the object files should be linked as an EXE or a DLL. Basically, if .def files are more commonly used by EXEs than I've seen, I don't want the presence of a .def file to default to linknig as a DLL; otherwise, I do.
Seems no difference at the time they are on export section... Also the same goes for imports. (correct me some one if i am wrong)
QuoteThe link order will define entry points, order of code/data within the file, and segment order and alignments. It's probably less problematic in 32-bit code, but things could get very weird with 16-bit code.
On C++ compilers ive saw the vector initialiser of each class are put next to other, no matter the order you pass the sources... (correct me again if i am wrong)
Quote from: Queue on June 23, 2010, 09:24:35 PM
3) Why is rsrc.rc such a common name for a resource script file, rather than Project_Name.rc?
..
5) Why, among MASM32 projects, is it so common to specify includes with \masm32\etc. rather than using INCLUDE and LIB environment variables (set to the respective MASM32 folders), and then using path-less include and includelib entries in the ASM file?
3) Organically Grown Habits
TM.
Pro rsrc.rc:
1. You can save your project as project_22June.asm, project_23June.asm etc, and always use the same resource file
2. Your batch file looks a little bit simpler.
Contra rsrc.rc: You need a new folder for each tiny "project"
In RichMasm (my RTF editor) I use Project_Name.rc as default, but with a simple
OPT_Res FileXy somewhere under end start you can define a "fixed" resource file.
5) It works flawlessly, especially if people respect that \Masm32 is more "compatible" than
C:\Masm32\
Setting environment variables is pretty clumsy, and has changed with every major Windows version.
#5 bugged me when I re-downloaded MASM32 to assemble some testbed code here. The source files for the testbed were on another drive. Even though I "fixed" the include path to an absolute drive specification in the testbed, that include file included another, and of course didnt have an absolute drive specification....
Thinking about it, there isnt really a good solution. Note that this fact also prevented the MASM32 IDE from assembling the file (with no error messages.. either... hrmf)
Quote from: Rockoon on June 23, 2010, 10:39:32 PM
#5 bugged me when I re-downloaded MASM32 to assemble some testbed code here. The source files for the testbed were on another drive.
Hey Rockoon, having too many drives is a typical nerd problem. But then, we probably all qualify for that title here :bg
Some of these questions are in regards to me setting up a system to automate assembling and linking (particularly other peoples example code).
For example, I think it's fair to assume that if a .def file is included, the ASM file is for a DLL. Now, I know an EXE CAN use a .def file, but even a bare-minimum one (only a NAME entry) will change the resulting EXE, and, according to Microsoft, ''If you are building an .exe file that has no exports, using a .def file will make your output file larger and slower loading.'' http://msdn.microsoft.com/en-us/library/28d6s79h(v=VS.80).aspx
But I asked just in case anyone had thoughts on such an assumption.
My solution to people using all sorts of screwy folder structures for includes was to make an ASM file preprocessor (well, and a resource script preprocessor) that reduces includes down to just their file name, and make sure I have INCLUDE and LIB environment variables set properly. But again, I wanted to ask why people don't just do it that way by default in case someone could share some insight.
jj2007, adding directives after the end statement is pretty darn clever. I've been toying with using echo's in the ASM file to issue build directives. My way is clunky but pretty capable since I run the output as a batch file.
clive, when I referred to LINK.EXE, I just meant the version from MASM32, so 5.12.8078.
Queue
Quote from: Queue on June 23, 2010, 11:11:23 PM
Some of these questions are in regards to me setting up a system to automate assembling and linking (particularly other peoples example code).
For example, I think it's fair to assume that if a .def file is included, the ASM file is for a DLL. Now, I know an EXE CAN use a .def file, but even a bare-minimum one (only a NAME entry) will change the resulting EXE, and, according to Microsoft, ''If you are building an .exe file that has no exports, using a .def file will make your output file larger and slower loading.'' http://msdn.microsoft.com/en-us/library/28d6s79h(v=VS.80).aspx
I guess they mean adding a .def file would write into your export section the export name and the ordinal number. Wich would make your output file larger (lol not that much) and when load will check for export section RVA.
Its a fair assumption but who uses a .def for EXE?
Also here is a good guide about PE format including at first the EXPORT section:
http://msdn.microsoft.com/en-us/magazine/cc301808.aspx
EDIT: Woops, theunknownguy basically beat me to it.
Quote from: Rockoon on June 23, 2010, 11:29:49 PM
EDIT: Woops, theunknownguy basically beat me to it.
Lol i hate this nickname... :(
Quote from: Queue on June 23, 2010, 11:11:23 PMBut again, I wanted to ask why people don't just do it that way by default in case someone could share some insight
Most newcomers know nothing of environment variables or path settings, and it's much easier to say 'install to the default directory and build with the default options", rather than walking them through painstaking setting (and resetting) the paths. However, you can do some pretty impressive stuff with NMAKE, if you're willing to invest the time. For example, you can get the current drive and directory via an mnake macro, and and pass a new INCLUDE path to ml via the /I switch when you build the file, or even set (or reset) the INCLUDE environment, etc, etc.
-r
I would love to see an example for dynamically setting the PATH, INCLUDE and LIB environment variables using NMAKE. I'm doing as much, but with a monster of a batch file (for example, setting the INCLUDE folder to be %0\..\INC, which means a subfolder named INC under the folder my batch file is in). My solution works, but I'd love to have more options to explore.
My system for assembling and linking is a Frankenstein's monster, that's for sure. The backbone is a heck of a batch file: it can handle an ASM file drag-and-dropped onto it, being called via a file association (I have ASM files set with an Edit command for notepad.exe and a Compile command for my batch file), being called via another batch file sending a commandline argument, or being called via another batch file with the project name in an environment variable. The ASM and RC pre-processor is a vbscript file that's called by a miniscule MASM-built EXE and it relies on Vortex's Scan.exe and inc2lib.exe (which relies on POLIB.EXE) to generate missing .inc and .lib files on demand. I just use Microsoft's ML, LINK, RC and CVTRES to do the actual assembling and linking.
Although I have decent error-checking to make sure everything is working, NMAKE would surely be a better fit for huge projects.
Regardless, I need to reconsider how I handle linking my object files; I'm currently doing it via "Project Name.*.obj" and my batch file generates "Project Name.res.obj" (if "Project Name.rc" or rsrc.rc exist) and "Project Name.asm.obj" but if I want to include additional object files, I don't have control over what order they're linked in.
theunknownguy, that's a great link; thanks.
Part of what got me to develop an automated system for assembling and linking for myself was MASM32's installation location being a folder on the root of my drive. -_- I love the resources that MASM32 provides, but there's no way I'm gonna let it tell me where it's going to be installed, especially if it's not under Program Files. Personally, my C drive has the following 4 folders and nothing else: Windows, Program Files, My Documents, Recycled. My D drive has Program Files, My Documents and Recycled. I'm the type that only puts one partition per physical drive, and keeps folders and files extremely organized, in particular I keep root folders of drives tidy and my Desktop, Quick Launch and Start Menu are near empty.
Queue
NMAKE can build on "psuedo targets", which are just file that don't exist (and are always 'built'). For example:
all: setenv myfile.exe
myfile.exe: myfile.asm
command_to_build file
setenv:
SET INCLUDE=SOMETHING_OR_OTHER
'all' is a pseudo target, which depends on 'setend' (another psuedo) plus myfile (the actual output). Both of them are evaulated as to the other blocks, and so on. Moreover, you can use the $(@D) Macro to expand to the current directory:
setenv:
SET INCLUDE=$(@D)\SOMETHING_OR_OTHER_IN_WHATEVER_DIRECTORY_I_WAS_RAN_IN
and so on. makefiles are WELL worth the investment into learning them, especially if you build from the commandline. The best guide is chapter 16 of the masm programmers guide, available all over the place.
-r
Quote from: jj2007 on June 23, 2010, 10:41:41 PM
Hey Rockoon, having too many drives is a typical nerd problem. But then, we probably all qualify for that title here :bg
After thinking about this more, I just dont see why any of the include files distributed with MASM32 have hard-coded paths in them. This *negates* those environment variables, boiling down to two obvious limitations:
* cant assemble across drives while referencing any of those includes containing hard-coded paths
* cant install masm32 to a different location (ex: D:/assemblers/masm32)
There could be other issues. For certain, QEditor breaks (without notice) when dealing with a source file on another drive that includes one of these offending include files. The one that caused me issues was \masm32\include\masm32rt.inc
For those that use the command line (like me), MASM32 can create a shortcut to CMD.EXE upon installation, essentially this would be the "MASM32 Console"
..this shortcut is basically:
Target: %windir%\system32\cmd.exe /K "environ.bat"
Start in: D:\masm32\
/K tells cmd.exe not to exit when done processing the command it was given, the D:\masm32\ is replaced with the install location at install time, and ENVIRON.BAT is:
@SET PATH=%CD%\bin;%PATH%
@SET LIB=%CD%\lib
@SET INCLUDE=%CD%\include
@ECHO MASM32 Console.
@ECHO ------------------------
@ECHO Run 'QEditor' to use the Integrated Development Environment.
Thing like this "MASM32 Console" (and NMAKE solutions) are useless as long as those hard-coded include paths remain.
Edited to add:
The current situation ISNT newbie friendly. I am much less likely to help someone if I cant get their code to assemble without a lot of work on my part, so I must choose between "use the default install location which I dont like" and "dont help people"
Thinking about this further, that environ.bat could just as easily be IDE.BAT:
@SET PATH=%CD%\bin;%PATH%
@SET LIB=%CD%\lib
@SET INCLUDE=%CD%\include
@start QEditor
@exit
That's what drove me to make an ASM file preprocessor Rockoon; it parses an ASM file reading each include and includelib line, checks to see if the specified file exists, and if it does, it changes it to JUST a filename with no path at all. Then it checks a couple different folders for the file (the current working directory and the path specified by the INCLUDE/LIB environment variable). If it's still not found, then it tries to generate the file automatically. So far I've been able to assemble and link, without modification, basically every bit of code I've found attached on these forums, and several different MASM tutorial code sets (Iczelion, td_win32asm).
I largely started this thread to try and get info to refine my system further. The easier it is for me to build peoples' example code, the easier it is for me to work with and learn from it.
Queue
Edit: I had to modify masm32rt.inc to remove the hard-coded paths as my preprocessor only handles the main ASM file, not the includes. It and macros.asm are the only two MASM32 includes that have hard-coded paths in them, aren't they?
Quote from: Rockoon on June 24, 2010, 01:14:37 AMThe current situation ISNT newbie friendly
There's an obvious solution: install to the default directory and build with the default options! :green2
But for real, look into NT 'junctions' - you can just hardlink one path with another, and solve all the problems at once
Quote from: redskull on June 24, 2010, 01:39:09 AM
Quote from: Rockoon on June 24, 2010, 01:14:37 AMThe current situation ISNT newbie friendly
But for real, look into NT 'junctions' - you can just hardlink one path with another, and solve all the problems at once
It is still the case that even with the junction HACK, cross-drive assembling is still broken, unless you propose creating a junction on every drive?
This is the most serious of the issues. That newbie that we are worried about, even if (s)he installs to the default and recommended location, (s)he will still eventually (maybe immediately) run into the cross-drive problem.
As I have noted before, qeditor remains completely silent about the problem. No error messages, no indication of any problems whatsoever. No executable generated. No object file generated. Silence. That is most definately NOT newbie friendly.
So the whole thing is circular.. hard coded paths BECAUSE of newbies, and then slug newbies in the face BECAUSE of hard coded paths.
Just get rid of the hard coded paths. If it means environment variables, there are simple newbie friendly solutions for that.
i prefer using the environment to control assembly
in my opinion, it is how it was meant to be used
and, it allows you to easily switch libraries, etc
but, there are 2 problems if you want to use that technique with the masm32 package
1) you'd have to change a lot of files to make it work :P
and, you'd have to do it each time the package was updated
you could probably write a little program to do it for you, though
2) code you authored would not be usable in the context of the forum
most other forum members are set up to use the hard-coded paths
if you wanted to post code or example programs in the forum, you'd have to modify them each time in order for others to use it
this would open up a lot of opportunities for errors
apparently, some of the inital users were confused by the variables
that meant there were problems setting up the package in the beginning
Hutch wanted the package to be a simple "install and use it" type package
thus, we have hard-code paths
Quote from: redskullBut for real, look into NT 'junctions' - you can just hardlink one path with another, and solve all the problems at once
For Windows Vista and Windows 7, a directory symbolic link will work just as well (see the MKLINK command), I do that with my source code on my D: drive and it works just fine. For earlier versions of NT based Windows you will have to use a junction.
[Edit] You can use Sysinternals
junction.exe to create a junction.
Examples: D:\MKLINK /D masm32 C:\masm32
JUNCTION D:\masm32 C:\masm32
Quote from: dedndave on June 24, 2010, 03:51:40 AM
2) code you authored would not be usable in the context of the forum
most other forum members are set up to use the hard-coded paths
Source code that references the include files in a hard coded manner may fail to assemble, but only if the user has put them someplace else, and then the source code that has failed to assemble can be modified.
Source code that references the include files in a soft coded manner may fail to assemble, but only if the user doesn't have environment variables set up, and then he can edit the source code, or modify his environment variables, or even supply the correct path to the assembler on the command line.
The alternative situation is that the include files themselves use hard coded paths, and if the source code doesnt assemble because of it, the user has to edit or move the include files!
That could break other source code!All bets are off with regards to interoperability as soon as the user is coerced into messing with those supplied include files, which must remain unchanged and unremarkable if we desire interoperability.
Let me re-iterate that
qeditor will not assemble a source file located on a drive where masm32 does not reside, unless that source file does not reference any of the offending hard-coded-paths-inside include files in any way. Furthermore, qeditor could *easily* supply the proper environment variables. This insistence on hard coded paths is completely irrational:
A development console could be easily supplied that has the proper environment variables.
qeditor could easily be modified so that is produces the proper environment variables.
make.bat templates could easily be given that include the proper command line to locate the include files.
hard coded paths that are NOT in include files are NOT the problem.
The correct solution is to just modify the .bat files so they include "/I<include-path>" for ml, and "/LIBPATH:<lib-path>" for link. 'include' and 'includelib' statements then require no path (just the bare filename), and everything works no matter where anyone has masm installed.
..as long as the .bat paths are set correctly, which could easily be produced on install; or they can use %MASM_HOME% and its value could be set once upon install.
Only masm32rt.inc and macros.asm contain \masm32\ includes, and the one in macros.asm is pretty insignificant (the uselib macro). So, if you want to use MASM32's includes from a non-default location, modify masm32rt.inc yourself and you'll be set.
After some more tinkering I've set myself up thusly:
- MASM folder containing BUILDASM.BAT
- - BIN subfolder containing programs for assembling, linking, etc.
- - INC subfolder containing .INC files, macros.asm and resource.h
- - - LIB subfolder under the INC subfolder containing .LIB files
Now, the first odd thing you'll notice is I put the LIB folder in the INC folder, instead of beside it in the MASM folder. This is so I can use a trick to save environment variable space. Instead of defining my INCLUDE and LIB environment variables at the same time, I only define INCLUDE. After assembling, but before linking, I use a single line in a batch file to set the LIB environment variable to %INCLUDE%\LIB and empty the INCLUDE environment variable, which effectively renames INCLUDE to LIB and adds \LIB.
The following two lines are the same number of characters as eachother:
INCLUDE=X:\Folders\INC
LIB=X:\Folders\INC\LIB
Why do I go through that trouble? By default, Win9x has really limited environment variable space, and one of my computers is still running Win98SE.
The batch file assumes everything it will need is in the BIN subfolder, so it sets that PATH environment variable to that folder exclusively.
I've got this working off a flash drive, so I wanted it to be capable of setting environment variables on demand, and I had to do a fair bit of weird things to get the batch file to behave consistently on Win98SE, XP and Vista, but as is, I can now plug in to any given computer, drag-and-drop ASM files onto BUILDASM.BAT and have them assemble and link properly. To save space, I only include the .INC and .LIB files that are in masm32rt.inc; all others are generated on demand by my VBScript ASM file preprocessor (and Vortex's Scan and inc2lib utilities, and POLIB). Obviously, that only works for import libraries and not libraries that contain actual code, but that accounts for most of them.
In relation to my original 5 questions, and some of the feedback I got, I've done the following:
1) If a .DEF file exists with the same name as the .ASM file, assume we're making a DLL, and link with the appropriate settings.
2) I haven't improved object file linking order, right now it's using a wildcard to make sure it gets any that exist within the naming scheme I'm using ("Project Name.*.obj") but this doesn't let you specify the order.
3) I simply check for both "Project Name.rc" and rsrc.rc; that covers both situations.
4) I'm still using LINK by default for now, rather than POLINK, but I have POLINK and found the output it generates easier to hex edit by hand than LINK's output.
5) A manually edited masm32rt.inc plus an ASM preprocessor handles the \masm\etc include and includelib entries, in addition to any ASM files with completely hard-coded include paths (C:\etc).
I think in the case of number 2), the only real solution is manual linking; I can't think of any way to determine a ''proper'' link order for object files.
Queue
well - we aren't talking about all that much environment space
that is, unless you use some really crazy folder names :P
but, you can enlarge the environment table in DOS and Win9x
i used to set my MASM 5.10 up with permanent variables
LIB and INCLUDE are the important ones
of course, your BIN folder should be in the PATH variable
MASM was another variable that pointed to the BIN folder, also
i don't think they ever made a special one for MACROS - i used to put macros in with INC's
you can also set them up temporarily
in fact, you can create a special ASM Command Prompt (console window) shortcut with the variables all set up
that way, you don't have to put them in every batch file - only the ones where a change is made
if you wanted, you could have seperate shortcuts for assembling with different paths
i used to do something similar back in the days when the C compiler paths conflicted with the MASM paths
the environment table space is only used when you have that console window open
This is so out of my level... I just install everything in any place... And folders all over place + 1 overloaded disk driver :dance:
PS: This should be a thread called "how far would you go for a path folder"...
:bg
environment tables and variables are so simple
it is basic DOS behaviour 101
i don't know why so many people have so many problems with it
i suppose noone wants to master DOS anymore :P
<-----------------------------
how many clock cycles does it take to multiply 1024 by 33 on an abacus ???