The MASM Forum Archive 2004 to 2012

Project Support Forums => IDE Development and Support => RadAsm Support Forum => Topic started by: Shooter on December 09, 2010, 01:37:56 AM

Title: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 09, 2010, 01:37:56 AM
Quote from: jj2007 on December 08, 2010, 12:14:19 PM
To see the difference, you should use OllyDbg.

jj2007,
In "Project | Project Options", the drop down list only shows "Window Release"; in "Option | Make Options" the "..." button for selecting a debugger does absolutely nothing, so I manually have to type in the complete path pointing to "C:\Program Files\RadASM\Ollydbg\Ollydbg.exe" <---- I'm thinking this is a bug for both issues.

-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: jj2007 on December 09, 2010, 08:15:20 AM
Shooter,
Can't help you on that one. I use RichMasm (the MasmBasic editor), which has a different approach: If the editor finds int 3 somewhere, the assemble, link & run button (F6) interprets "run" as "run \masm32\OllyDbg\ollydbg.exe MySauce.exe". So all I do is insert a breakpoint, hit F6 and here you go, hello Olly! Maybe you can convince the RadAsm author to implement something similar ;-)
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 15, 2010, 04:23:46 PM
I still haven't figured out how to set up the debugging options in RadASM 3.x. I've got a project that has a loop in it, so I'm trying to figure out how to set a breakpoint to figure things out, but there aren't any options to make the project into a Debug Release, let alone set breakpoints.

Everything under the "Debug" menu item is grayed out, which I'm assuming is for the RadASM's internal debugger, but I can't even set anything up for OllyDbg (I have to manually launch OllyDbg, but there aren't any symbols to reference to).

What am I doing wrong?

-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 16, 2010, 02:43:56 AM
The debugger you set up in the "Make Options" is the external debugger, it is invoked by selecting "Run with debug" from the Make menu or Ctrl-D.

A breakpoint is simply interrupt 3:

INT 3

you can add it to your source code any where you want a breakpoint. OllyDbg allows you to add breakpoints during execution.

Not sure why you're having problems using OllyDbg from the Make menu, works fine here as long as you define an external debugger in Project Options.

The internal Debugger is for MASM only.

Edgar
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 16, 2010, 07:09:01 PM
Quote from: donkey on December 16, 2010, 02:43:56 AM
Not sure why you're having problems using OllyDbg from the Make menu, works fine here as long as you define an external debugger in Project Options.

Edgar,
Did you mean the "Option | Make Options" dialog? That's the only place I've found where I can specify a debugger, and yeah, I've got a full path pointing to OllyDbg.exe. (This is also the location where the button next to the debugger textbox ("...") does nothing that I mentioned before.)

-Shooter

(Side question... Do I have to have an image on a website somewhere in order to add it to a post here on the forum?)
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 17, 2010, 07:08:42 AM
Hi Shooter,

Yes, when you create a project it will use the external debugger from the Options::Make Options entry. If the project is existing and the debugger has not been set you can specify it in Project::Project Options.

Yes, you need to park the image somewhere (I use Imageshack) then is the IMG tags to display it.

Edgar
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 17, 2010, 02:13:49 PM
Quote from: donkey on December 17, 2010, 07:08:42 AM
If the project is existing and the debugger has not been set you can specify it in Project::Project Options.

Edgar,
I have just got to be blind, for I can not for the life of me figure out where you mean.

(http://img831.imageshack.us/img831/1552/projectoptionsscr4.png) (http://img831.imageshack.us/i/projectoptionsscr4.png/)


This is the only place I have found to set up a debugger, and as far as I can tell, it's not doing anything.
(http://img444.imageshack.us/img444/7599/makeoptionsscr2.png) (http://img444.imageshack.us/i/makeoptionsscr2.png/)


-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 18, 2010, 10:15:40 AM
Hi Shooter,

Yeah, you're right. Not sure what I was thinking.

Edgar
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 18, 2010, 02:45:49 PM
So, am I being dense and just can't see the solution, or do I have a real problem? (I don't mind being called 'dense' just so long as the darn thing works. :dazzled:)
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 18, 2010, 03:49:36 PM
Hi Shooter,

I have no idea why it isn't working for you, it works just fine here and has since I installed RadAsm 3. At some point I would consider wiping out your entire RadAsm installation and starting over from scratch with the defaults Ketil has set then work from there to get it properly configured.
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 18, 2010, 06:18:59 PM
Quote from: donkey on December 18, 2010, 03:49:36 PM
At some point I would consider wiping out your entire RadAsm installation and starting over from scratch with the defaults Ketil has set then work from there to get it properly configured.

Given all the headaches, frustrations, and problems, I'm going to take that advice. I'm deleting everything RadASM related (backing up 1st, of course), and am going to reinstall just RadASM 3.x and give that a go. However, if memory serves, I had to do something with the call to GFL.txt and add your list of includes somewhere. Do you know if this has already been incorporated in the latest build set?

-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 19, 2010, 05:37:23 AM
gfl.txt is not used with the header project, you can delete it and remove any reference to it from the command line. There should no longer be any reference to it in the current distribution of RadAsm. Instead you just include the following line before Windows.h is included:

#DEFINE LINKFILES

Edgar
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 19, 2010, 06:38:25 AM
Quote from: donkey on December 19, 2010, 05:37:23 AM
#DEFINE LINKFILES

Is that a GoASM thing, or does that apply to MASM32 as well?

-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on December 19, 2010, 06:48:18 AM
It has nothing to do with MASM32, it is a switch for the GoAsm headers that passes the name of the appropriate DLL to golink at compile time through the #dynamiclinkfile directive. MASM (MASM32 is not MASM) does not have the capacity to directly import functions from DLL files, it uses includelib and import libraries instead. The headers for GoAsm are incompatible with MASM just as import libraries are incompatible with GoAsm.

There is almost no cross-over between MASM and GoAsm, though the syntax is similar they are very different under the hood and few of the support files for one will work properly with the other. In general any include file for MASM can be considered incompatible with GoAsm and vice versa.

Edgar
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on December 19, 2010, 10:10:33 PM
In the words of that famous black man on his stand-up comic tour, "Bill Cosby Himself", "Obeekaybee".  :U

So, I've wiped it all out, re-installed everything and started a new MASM project. I got to a point where I needed to debug it (trying to read a key from the system registry, but it comes back as File_Not_Found (eax=02)) and went to make it into a Debug Version:

1) I went to "Project::Project Options" and selected "Window Debug" under the "Make Options", but the OK button was still grayed out. So, I pressed the space bar in the empty "Library" textbox and the OK button came alive (I clicked it to exit).
2) I went immediately back into "Project::Project Options" and noticed nothing had changed (still set to "Window Release").
3) I opened "Options::Make Options" and selected "Window Debug" and selected OK.
4) I went immediately back into "Options::Make Options" and again noticed nothing had changed (still set to "Window Release").

I will say that "Make::Run with Debug" now launches OllyBug like it should.

So, is it that I've got a setting wrong in some .ini file? Or maybe it's the way I've got my directory structure? Or is it possible that it might be because I'm missing some files that came along with the development of RadASM? Could it be that I'm running XP Pro SP3? Am I missing files from a Visual Studio? (I'm just throwing out ideas in the hopes something somewhere might stand out and lead to a solution.)

(http://img638.imageshack.us/img638/2596/projectoptionsdebuggeri.png) (http://img638.imageshack.us/i/projectoptionsdebuggeri.png/)

(http://img718.imageshack.us/img718/2596/projectoptionsdebuggeri.png) (http://img718.imageshack.us/i/projectoptionsdebuggeri.png/)
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on January 05, 2011, 06:23:09 AM
RadASM 3.0.0.7i fixed the "..." button I was having problems with, but I'm still having other problems. In order for me to actually get a debugged compiled version I had to copy the strings from the debug release to the standard release. Not sure why I can't get it to work still.

-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on January 05, 2011, 06:39:54 AM
Hi Shooter,

This may seem like a dumb question but after you add the debug build type in Project Options and before you build the project you do select the build type on the toolbar right ? The drop down combo box at the end of the toolbar is where you select how you want to build the project, it contains the build types you added in Project Options.
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on January 05, 2011, 09:15:32 AM
Quote from: donkey on January 05, 2011, 06:39:54 AM
This may seem like a dumb question but after you add the debug build type in Project Options and before you build the project you do select the build type on the toolbar right ? The drop down combo box at the end of the toolbar is where you select how you want to build the project, it contains the build types you added in Project Options.

As a point of clarification, would you mind providing a screen shot as to this?

Thanks,
-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on January 05, 2011, 10:59:35 PM
My RadAsm 3 is heavily modified through addins that I'm testing, it won't look much like yours. The drop down combo is on the end of the main toolbar and should say "Windows Release" at startup. Press the down arrow and you will find a number of different build types that were added in the project options dialog. Select the build type you want to use and then Make:Build the project.
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on January 06, 2011, 12:21:48 AM
Quote from: donkey on January 05, 2011, 10:59:35 PM
My RadAsm 3 is heavily modified through addins that I'm testing, it won't look much like yours. The drop down combo is on the end of the main toolbar and should say "Windows Release" at startup. Press the down arrow and you will find a number of different build types that were added in the project options dialog. Select the build type you want to use and then Make:Build the project.

To tell the truth, I'd forgotten about it since 3.0.0.7g. I just tried it again and all seems to work as it should. Thanks for reminding me about it.
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on January 06, 2011, 03:52:12 AM
Quote from: Shooter on January 06, 2011, 12:21:48 AM
Thanks for reminding me about it.

You're welcome. I've been using RadAsm since the days of 1.x and there are times that I forget how to do certain things, it has to be exponentially more frustrating for a new user to try to remember all the ins and outs of the IDE. It's a large complex program but once you get a few things down its hard to find one more powerful.
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: Shooter on January 06, 2011, 06:39:34 AM
Quote from: donkey on January 06, 2011, 03:52:12 AM
You're welcome. I've been using RadAsm since the days of 1.x and there are times that I forget how to do certain things, it has to be exponentially more frustrating for a new user to try to remember all the ins and outs of the IDE. It's a large complex program but once you get a few things down its hard to find one more powerful.

Haha, no doubt.

Two questions:
1) It was mentioned in another thread about an addin to make RadASM launch a debugger that automatically load the current executable... how's that done? Addin's are completely new to me.

2) What is VKim?
Thanks,
-Shooter
Title: Re: Setting up debugger problem in RadASM 3.x
Post by: donkey on January 06, 2011, 07:39:37 AM
Quote from: Shooter on January 06, 2011, 06:39:34 AM
1) It was mentioned in another thread about an addin to make RadASM launch a debugger that automatically load the current executable... how's that done? Addin's are completely new to me.

Not sure what you're talking about here, if its VKim's, that's not what it does. Its an inline debugging tool.

Quote2) What is VKim?

Who is Vkim. A programmer who wrote an excellent debugging tool. It's been a while since he has hung around the boards though, a few years at least.