Setting up debugger problem in RadASM 3.x

[Shooter]

To see the difference, you should use OllyDbg.

jj2007,
In "Project | Project Options", the drop down list only shows "Window Release"; in "Option | Make Options" the "..." button for selecting a debugger does absolutely nothing, so I manually have to type in the complete path pointing to "C:\Program Files\RadASM\Ollydbg\Ollydbg.exe" <---- I'm thinking this is a bug for both issues.

-Shooter

[jj2007]

Shooter,
Can't help you on that one. I use RichMasm (the MasmBasic editor), which has a different approach: If the editor finds int 3 somewhere, the assemble, link & run button (F6) interprets "run" as "run \masm32\OllyDbg\ollydbg.exe MySauce.exe". So all I do is insert a breakpoint, hit F6 and here you go, hello Olly! Maybe you can convince the RadAsm author to implement something similar ;-)

[Shooter]

I still haven't figured out how to set up the debugging options in RadASM 3.x. I've got a project that has a loop in it, so I'm trying to figure out how to set a breakpoint to figure things out, but there aren't any options to make the project into a Debug Release, let alone set breakpoints.

Everything under the "Debug" menu item is grayed out, which I'm assuming is for the RadASM's internal debugger, but I can't even set anything up for OllyDbg (I have to manually launch OllyDbg, but there aren't any symbols to reference to).

What am I doing wrong?

-Shooter

[donkey]

The debugger you set up in the "Make Options" is the external debugger, it is invoked by selecting "Run with debug" from the Make menu or Ctrl-D.

A breakpoint is simply interrupt 3:

INT 3

you can add it to your source code any where you want a breakpoint. OllyDbg allows you to add breakpoints during execution.

Not sure why you're having problems using OllyDbg from the Make menu, works fine here as long as you define an external debugger in Project Options.

The internal Debugger is for MASM only.

Edgar

[Shooter]

Not sure why you're having problems using OllyDbg from the Make menu, works fine here as long as you define an external debugger in Project Options.

Edgar,
Did you mean the "Option | Make Options" dialog? That's the only place I've found where I can specify a debugger, and yeah, I've got a full path pointing to OllyDbg.exe. (This is also the location where the button next to the debugger textbox ("...") does nothing that I mentioned before.)

-Shooter

(Side question... Do I have to have an image on a website somewhere in order to add it to a post here on the forum?)

[donkey]

Hi Shooter,

Yes, when you create a project it will use the external debugger from the Options::Make Options entry. If the project is existing and the debugger has not been set you can specify it in Project::Project Options.

Yes, you need to park the image somewhere (I use Imageshack) then is the IMG tags to display it.

Edgar

[Shooter]

If the project is existing and the debugger has not been set you can specify it in Project::Project Options.

Edgar,
I have just got to be blind, for I can not for the life of me figure out where you mean.




This is the only place I have found to set up a debugger, and as far as I can tell, it's not doing anything.



-Shooter

[donkey]

Hi Shooter,

Yeah, you're right. Not sure what I was thinking.

Edgar

[Shooter]

So, am I being dense and just can't see the solution, or do I have a real problem? (I don't mind being called 'dense' just so long as the darn thing works. :dazzled:)

[donkey]

Hi Shooter,

I have no idea why it isn't working for you, it works just fine here and has since I installed RadAsm 3. At some point I would consider wiping out your entire RadAsm installation and starting over from scratch with the defaults Ketil has set then work from there to get it properly configured.

[Shooter]

At some point I would consider wiping out your entire RadAsm installation and starting over from scratch with the defaults Ketil has set then work from there to get it properly configured.

Given all the headaches, frustrations, and problems, I'm going to take that advice. I'm deleting everything RadASM related (backing up 1st, of course), and am going to reinstall just RadASM 3.x and give that a go. However, if memory serves, I had to do something with the call to GFL.txt and add your list of includes somewhere. Do you know if this has already been incorporated in the latest build set?

-Shooter

[donkey]

gfl.txt is not used with the header project, you can delete it and remove any reference to it from the command line. There should no longer be any reference to it in the current distribution of RadAsm. Instead you just include the following line before Windows.h is included:

#DEFINE LINKFILES

Edgar

[Shooter]

#DEFINE LINKFILES

Is that a GoASM thing, or does that apply to MASM32 as well?

-Shooter

[donkey]

It has nothing to do with MASM32, it is a switch for the GoAsm headers that passes the name of the appropriate DLL to golink at compile time through the #dynamiclinkfile directive. MASM (MASM32 is not MASM) does not have the capacity to directly import functions from DLL files, it uses includelib and import libraries instead. The headers for GoAsm are incompatible with MASM just as import libraries are incompatible with GoAsm.

There is almost no cross-over between MASM and GoAsm, though the syntax is similar they are very different under the hood and few of the support files for one will work properly with the other. In general any include file for MASM can be considered incompatible with GoAsm and vice versa.

Edgar

[Shooter]

In the words of that famous black man on his stand-up comic tour, "Bill Cosby Himself", "Obeekaybee".  :U

So, I've wiped it all out, re-installed everything and started a new MASM project. I got to a point where I needed to debug it (trying to read a key from the system registry, but it comes back as File_Not_Found (eax=02)) and went to make it into a Debug Version:

1) I went to "Project::Project Options" and selected "Window Debug" under the "Make Options", but the OK button was still grayed out. So, I pressed the space bar in the empty "Library" textbox and the OK button came alive (I clicked it to exit).
2) I went immediately back into "Project::Project Options" and noticed nothing had changed (still set to "Window Release").
3) I opened "Options::Make Options" and selected "Window Debug" and selected OK.
4) I went immediately back into "Options::Make Options" and again noticed nothing had changed (still set to "Window Release").

I will say that "Make::Run with Debug" now launches OllyBug like it should.

So, is it that I've got a setting wrong in some .ini file? Or maybe it's the way I've got my directory structure? Or is it possible that it might be because I'm missing some files that came along with the development of RadASM? Could it be that I'm running XP Pro SP3? Am I missing files from a Visual Studio? (I'm just throwing out ideas in the hopes something somewhere might stand out and lead to a solution.)