Debugging a DLL

[Robert Collins]

If I write a C/C++ program and compole it to an .EXE in the Debug mode I can single step through it using the compiler.

Now, if I write a C/C++ program and compile it to a .DLL and use that DLL in a VB program for example, is there any way to also single step through the DLL just like I am able to do with the .EXE?

I am not a real experienced C/C++ programmer (my experience is mainly on a need to know basis and I do write in C/C++ alot because of my job duties where I have to make my own DLLs to be part of my many VB assignments) but I do need to test my DLLs as part of my VB projects and I am getting sick of always having to insert messages boxes everywhere in the DLL just to get a crap-ass feedback of where I am in the DLL and the values of variables. This method sucks but I know of no other way to trace my DLLs.

Can anyone help this poor idiot?

[PellesC]

Perhaps add one or more "int 3" instructions (breakpoint; inline assembly) and let the "just-in-time" debugger catch it? It should work...

Pelle

[Mincho Georgiev]

Hello, Robert!
Yes,there is a way, but i'be never tryed that in VStudio. My way is with OllyDbg. BUT : the executable MUST attach the dll process,nomatter if is with LoadLibrary or else. When you load the .exe in olly dbg, go to View->executable modules and double 
click on your dll,attached by the application,loaded in debugger. Then find what you looking for and set a breakpoint. When the app reach that BPX, you can debugging the dll. Always set a hardware breakpoints(on execution). I dont deal with visual basic but i'm sure that is not a problem if the executable is VB written.
Something nasty with that method is that every time you want to debug the dll, you have to set the same (or other) breakpoint over and over again. But you can do it like that :
1.Loading the exe in  olly dbg.
2.View->executable modules and select your dll (it must be attached by the app!!!)
3.Debug -> Hardware Breakpoints -> follow the previous set!

thats all!

[Mincho Georgiev]

if both - exe and dll are in C++ (or some of them is in asm) is easy, you just have to attach the dll to the exe in visual studio,but you propably know how...

[Robert Collins]

Well, all three of you had decent answers but I don't think I can take advantage of any of them.....

As for the "int 3". Problem: I still need to manually insert these instructions and when it does break
wouldn't the "jit" debugger be in machine code rather than C code?

As far as OllyDbg. I can't use OllyDbg because this is stuff I'm doing at work and I am not allowed to install any outside
software.

As far as both in C. No, the main .EXE is in VB and the DLL is C.

[Mincho Georgiev]

Ok ,i'll explain to you step by step.

1st Buld you library with a debugging configuration in VC++

2nd Build you VB Project (Make sure that you've declared everything correctly)

3th Buld your VB as .exe

4th place the exe in the directory (\DEBUG) of your VC++ DLL's Project (where the dll already exist offcourse)

5th open your VC++ Workspace of the DLL

6th Set A Breakpoint where you wish to debug

7 After the Dlg for the attachment is on ,browse your exe in the dll's (\debug) directory - this is the executable for the debugging session ,nomatter that the exe does not contain a debugging symbols - we want to debug the dll only and just the dll must have PDB. (that's way i've said "Debugging configuration)

8th When you Start the exe from the VC++ Debugger, it will breaks automatically at the breakpoint that we had set - derectly in the dll's source!

That's all!
I hope you understand my point!
:U

here is an simple example - just build them both and place the exe in the \debug directory of the dll
It does not matter if the .exe is written in any language. The only condition is the dll to be able for debugging!


[attachment deleted by admin]

[Robert Collins]

OK, that works great!!!!

The only thing that I don't understand is this:

I unzipped your sample in a directory on my 'C' Drive....

C:\DEBUG_DLL

Then I went into the 'src' directory of the VB program and compiled it into an .EXE
Then I went into the 'testlib' directory and built the DLL into the ..\Debug directory
I copied the .EXE into the ..\Debug directory
In the VC++ IDE I did what you said.

Now, for some reason, everything wound up in the following directory...

D:\PROBA\testlib\Debug

Where and how did this occur?

Oh, well, it works, it is exactly what I needed, and I thank you very much for your help. 

[Mincho Georgiev]

Sorry about that,the dsw - Workspace has a default output folder and is a d:\proba..bla bla
you can just change it or make a new project with new workspace and put the source files in!
You're welcome!

[PBrennick]

Hi Robert,
Here is another method.  It is a lot simpler and is the one I use.  The DLL is nothing more than an EXE that is not able to self execute so you can wrap any of your procedures in your DLL into a test piece and build it as an exe and test it in any debugger as usual.  You just set up the test piece to call the procedures in the same way as you would once it is a DLL.  The only difference is that it is all in one EXE, instead.  As you get good at this method, you will learn how to create what I call a 'boilerplate' EXE that has a stock set of call already embedded into it so all you have to do is add the procedures, compile it, test it and make adjustments to the procedure as necessary.  Once it is fully tested, copy it into your existing DLL source, build it and forget it as it will just work.

Paul

[sluggy]

When using Visual Studio, you don't need to start the exe from the IDE - you can run the exe normally, then use the menu options in VS to attach to the process you want to debug. As long as your dll is compiled in debug mode, not release, then you can load the source file into the IDE and set your breakpoints before attaching to the process, and the debugger will break when you hit those breakpoints (as normal, the debugger expects the pdb file to be in the same folder that the dll is run from). If you explore the options a little (in the breakpoints window) you will see also that you can set conditional breakpoints - ie. you only break if the specified condition is true, this is very useful.

[Mincho Georgiev]

However, i defenately prefare WinDbg and personally for me is the best debugging tool from microsoft!
VS6 does not supprot the new pdb format from 7.0 and 8.0,WinDbg does.