I am using Joergen Ibsen's tiny archiver for creating sfx archives. Out of curiosity, I switched on DEP on my Win XP SP2, and bang :tdown a GPF for apack.exe. Quite a harmless app, but somewhere it seems to confound .data and .code sections. In the meantime, I switched to 7-zip (faster and more compact output, although I still have some open questions...), but I wonder whether somebody else ran into the DEP trap?
More at "Hardware DEP has a backdoor", http://blog.fabriceroux.com/index.php?blog=1&title=hardware_dep_has_a_backdoor&more=1&c=1&tb=1&pb=1
Yes,
I had to rebuild just about every EXE in MASM32 because of the change in specification with DEP. I would have a look at JIBZ's site to see if he has done a later version that works under DEP, its probably a compressed EXE file which will cause the problem if it has not been designed for a DEP enabled OS.
Did you try something like (I have just this piece of code written in my scribble book, probably from OpenRCE forums, but I don't know for sure who is the author; when I find, I'll put the credits here): I've found ("I feel lucky!" :bg), here is the authority where I got the info: "http://www.uninformed.org/?v=2&a=4&t=txt"
ExecutionFlags dd 2
lea eax, ExecutionFlags
push 4 ;sizeof(DWORD)
push eax ;Address of value 2, which translates to MEM_EXECUTE_OPTION_ENABLE
push 022h ;Magic :-) In fact, value from enum {} which is called "ProcessExecuteFlags"
push -1 ;The value that GetCurrentProcess returns on Windows
invoke ntdll.dll!NtSetInformationProcess
;Here check the status
Another approach is to create the registry key under
HKEY_LOCAL_MACHINE\SOFTWARE[\Wow6432Node]\Microsoft\Windows NT\CurrentVersion\Image File Execution Options
which will be named after your file and add under it REG_DWORD value named "ExecuteOptions" with value 2.
Other values worth mentioning are REG_DWORD "GlobalFlag", REG_SZ (with possible values "0" or "1") named "DisableHeapLookAside", REG_SZ "Debugger" which lets you choose what is the debugger for particular file and for the end, obviously very usefull option "ApplicationGoo" which is REG_BINARY and yet undocumented :green