The MASM Forum Archive 2004 to 2012

General Forums => The Laboratory => Topic started by: donkey on December 27, 2004, 05:33:22 PM

Title: Check for valid PE file
Post by: donkey on December 27, 2004, 05:33:22 PM
I recently wanted to reliably determine if a given file was a valid PE file and this is what I came up with, it is in GoAsm syntax. It will return 0 if the file is a valid PE. If not it returns -1 and more information can be extracted using GetLastError.

IsPEFile FRAME pFilename
uses edi
LOCAL hPEFile :D
LOCAL hPEMapFile :D
LOCAL pPEMapFile :D
LOCAL fSize :D
LOCAL cbHigh :D

; Checks the file for indications that it is a valid Win32 PE file
; Returns 0 if the file is a valid Win32 executable
; -1 if there is an error, use GetLastError to obtain more information

invoke SetLastError,0
invoke CreateFileA,[pFilename],GENERIC_READ,FILE_SHARE_READ, \
NULL,OPEN_EXISTING,NULL,NULL
or eax,eax
jns >
ret
:
mov [hPEFile],eax

invoke GetFileSize,eax,offset cbHigh
mov [fSize],eax

invoke CreateFileMappingA,[hPEFile],0,PAGE_READONLY,0,0,0
mov [hPEMapFile],eax
or eax,eax
jnz >
invoke CloseHandle,[hPEFile]
xor eax,eax
dec eax
ret
:

invoke MapViewOfFile,[hPEMapFile],FILE_MAP_READ,0,0,0
mov [pPEMapFile],eax
or eax,eax
jnz >
invoke CloseHandle,[hPEMapFile]
invoke CloseHandle,[hPEFile]
xor eax,eax
dec eax
ret
:

mov edi,eax

cmp W[edi+IMAGE_DOS_HEADER.e_magic],"MZ"
jne >.INVALID_PE

mov edi,[edi+IMAGE_DOS_HEADER.e_lfanew]
add edi,[pPEMapFile]
jc >.INVALID_PE

; Get the size of the mapped file
mov eax,[pPEMapFile]
add eax,[fSize]
cmp edi, eax
ja >.INVALID_PE

cmp D[edi+IMAGE_NT_HEADERS.Signature],"PE"
jne >.INVALID_PE

invoke UnmapViewOfFile,[pPEMapFile]
invoke CloseHandle,[hPEMapFile]
invoke CloseHandle,[hPEFile]
xor eax,eax
RET

.INVALID_PE
invoke UnmapViewOfFile,[pPEMapFile]
invoke CloseHandle,[hPEMapFile]
invoke CloseHandle,[hPEFile]
invoke SetLastError,193 ; %1 is not a valid Win32 application. ERROR_BAD_EXE_FORMAT
xor eax,eax
dec eax
ret
ENDF


In case you don't have all the defs, here they are...
#Define FILE_MAP_READ 4h
#Define GENERIC_READ 080000000h
#Define FILE_SHARE_READ 1h
#Define OPEN_EXISTING 3
#Define PAGE_READONLY 2
#Define IMAGE_NUMBEROF_DIRECTORY_ENTRIES 16

IMAGE_DOS_HEADER STRUCT
  e_magic           DW
  e_cblp            DW
  e_cp              DW
  e_crlc            DW
  e_cparhdr         DW
  e_minalloc        DW
  e_maxalloc        DW
  e_ss              DW
  e_sp              DW
  e_csum            DW
  e_ip              DW
  e_cs              DW
  e_lfarlc          DW
  e_ovno            DW
  e_res             DW   4 dup 0
  e_oemid           DW
  e_oeminfo         DW
  e_res2            DW  10 dup 0
  e_lfanew          DD
IMAGE_DOS_HEADER ENDS

IMAGE_OPTIONAL_HEADER32 STRUCT
  Magic                         DW
  MajorLinkerVersion            DB
  MinorLinkerVersion            DB
  SizeOfCode                    DD
  SizeOfInitializedData         DD
  SizeOfUninitializedData       DD
  AddressOfEntryPoint           DD
  BaseOfCode                    DD
  BaseOfData                    DD
  ImageBase                     DD
  SectionAlignment              DD
  FileAlignment                 DD
  MajorOperatingSystemVersion   DW
  MinorOperatingSystemVersion   DW
  MajorImageVersion             DW
  MinorImageVersion             DW
  MajorSubsystemVersion         DW
  MinorSubsystemVersion         DW
  Win32VersionValue             DD
  SizeOfImage                   DD
  SizeOfHeaders                 DD
  CheckSum                      DD
  Subsystem                     DW
  DllCharacteristics            DW
  SizeOfStackReserve            DD
  SizeOfStackCommit             DD
  SizeOfHeapReserve             DD
  SizeOfHeapCommit              DD
  LoaderFlags                   DD
  NumberOfRvaAndSizes           DD
  DataDirectory                 DQ IMAGE_NUMBEROF_DIRECTORY_ENTRIES dup
IMAGE_OPTIONAL_HEADER32 ENDS

IMAGE_FILE_HEADER STRUCT
  Machine               DW
  NumberOfSections      DW
  TimeDateStamp         DD
  PointerToSymbolTable  DD
  NumberOfSymbols       DD
  SizeOfOptionalHeader  DW
  Characteristics       DW
IMAGE_FILE_HEADER ENDS


IMAGE_NT_HEADERS STRUCT
  Signature         DD
  FileHeader        IMAGE_FILE_HEADER         <>
  OptionalHeader    IMAGE_OPTIONAL_HEADER32   <>
ENDS


Title: Re: Check for valid PE file
Post by: Jibz on December 29, 2004, 07:37:47 PM
Nice work :U.

Not that it's likely to be a problem, but you might want to check that e_lfanew is not something evil like 0xffffff00 -- you only check if the resulting pointer is above the mapped image. Also if you want to be certain it doesn't crash, you should make sure there's room for the 4 bytes you compare there.

Btw, you can use or eax,-1 instead of xor eax,eax; dec eax.
Title: Re: Check for valid PE file
Post by: Relvinian on December 29, 2004, 08:07:02 PM
Quote from: Jibz on December 29, 2004, 07:37:47 PM
Btw, you can use or eax,-1 instead of xor eax,eax; dec eax.

Jibz,

Does this kind of trick work for setting it to 1 also?

or eax, 1  = 1 ????   instead of xor eax, eax, inc eax

Relvinian
Title: Re: Check for valid PE file
Post by: rea on December 29, 2004, 08:26:08 PM
I not think that, normally a -1 in complement 2 will be the size byte, word, dword... but will all bits sets..

-1 = 0x11..11 Then the or will set all :).. my question in this case why not only mov eax,-1?
Title: Re: Check for valid PE file
Post by: Jibz on December 29, 2004, 08:45:24 PM
Relvinian,

or only works for setting it to -1, i.e. all bits set.

rea,

You could use mov eax,-1 just the same, the only difference is size -- or eax,-1 and xor eax,eax; dec eax are 3 bytes, mov eax,-1 is 5 bytes.
Title: Re: Check for valid PE file
Post by: hutch-- on December 29, 2004, 10:50:05 PM
 :bg

In times past I have used a barbarian approach that worked pefectly for determining if a file was a PE file or not, scan the first 1k for the signature "PE00". I don't remember an inctance where it ever failed.

I like Donkey's code, its clear, simple and can be used for many purposes.
Title: Re: Check for valid PE file
Post by: donkey on December 30, 2004, 12:27:20 AM
Quote from: Jibz on December 29, 2004, 07:37:47 PM
Nice work :U.

Not that it's likely to be a problem, but you might want to check that e_lfanew is not something evil like 0xffffff00 -- you only check if the resulting pointer is above the mapped image. Also if you want to be certain it doesn't crash, you should make sure there's room for the 4 bytes you compare there.

Btw, you can use or eax,-1 instead of xor eax,eax; dec eax.

Hi Jibz,

Thanks, I did consider negative numbers,that is the reasoning behind ja instead of jg, if a negative number is added the compare is an unsigned one. As long as the value is beyond the end of the file or there are at least 4 bytes of data there it will not fail. Since memory maps are allocated in pages, I am taking the chance that the end of the PE in not directly at a page boundary so I can read beyond it just a bit.
Title: Re: Check for valid PE file
Post by: Jibz on December 30, 2004, 10:30:23 AM
Quote from: donkey on December 30, 2004, 12:27:20 AM
Thanks, I did consider negative numbers,that is the reasoning behind ja instead of jg, if a negative number is added the compare is an unsigned one. As long as the value is beyond the end of the file or there are at least 4 bytes of data there it will not fail. Since memory maps are allocated in pages, I am taking the chance that the end of the PE in not directly at a page boundary so I can read beyond it just a bit.

The problem is that a value like 0xffffff00 will not result in a negative number in the comparison, it will just give you a pointer to right before your mapped area. I cannot get the attachment function to work, so I'll e-mail you an example that crashes your function.

On a side note, a valid PE file can also have a 'ZM' signature in the DOS header.
Title: Re: Check for valid PE file
Post by: donkey on December 30, 2004, 11:17:46 AM
Hi Jibz,

Yes, I see now, I will fix up the code when I get home from work. Thanks  :U

By the way, it seems I am not the only one who has the problem. Both Hotmail and my firewall refused to download the stub, I had to turn off my firewall in order to get it...

image removed
Title: Re: Check for valid PE file
Post by: Jibz on December 30, 2004, 11:39:32 AM
Unknown virus scanner failure virus .. I wonder if that's related to the much feared No good virus scanner programmer virus :green.

The stub program is a valid DOS 16-bit executable, so if they flag that then they probably flag other valid DOS programs as well based on their code being buggy .. nice approach ;).
Title: Re: Check for valid PE file
Post by: donkey on December 30, 2004, 11:47:49 AM
Yeah, probably means exactly that  :toothy

The solution is adding a JC to check to see if the address has wrapped...

mov edi,[edi+IMAGE_DOS_HEADER.e_lfanew]
add edi,[pPEMapFile]
jc >.INVALID_PE
Title: Re: Check for valid PE file
Post by: donkey on December 30, 2004, 11:53:49 AM
Since this bug was in Files.LIB I have updated that library on my website with the jump if carry (jc)
Title: Re: Check for valid PE file
Post by: sheep on December 30, 2004, 07:50:42 PM
SetLastError is a pretty simple API, you could avoid the call overhead by inlining it:


mov eax,fs:[18h]
mov DWORD PTR [eax+34h],193
Title: Re: Check for valid PE file
Post by: Jibz on December 30, 2004, 08:00:30 PM
sheep,

I think that would only work on NT+.

It's better to use standard methods in a library function like that. And since there are plenty of other API calls in that function I think you'd gain little except for possible incompatibilities with future or past windows versions :U.
Title: Re: Check for valid PE file
Post by: donkey on December 30, 2004, 11:48:19 PM
Quote from: sheep on December 30, 2004, 07:50:42 PM
SetLastError is a pretty simple API, you could avoid the call overhead by inlining it:


mov eax,fs:[18h]
mov DWORD PTR [eax+34h],193


There are actually quite a few of those, I don't generally like to use them unless I am absolutely forced to, for the reasons Jibz gave, they may be changed in future versions of Windows. Especially the sparsely documented TIB and PDB. I posted a few others on the Win32ASM board a while back...

http://board.win32asmcommunity.net/viewtopic.php?t=18797

BTW, that is for NT+ only, for Win9x it's...

fs mov eax, [18h]
mov eax, [eax+60h]
Title: Re: Check for valid PE file
Post by: Tedd on January 04, 2005, 02:01:22 PM
Isn't the MZ stub optional? I mean, it's not strictly necessary and is only there in case you try to run it from DOS.
Anyway, point being that this code requires it and I don't think the PE does.
Title: Re: Check for valid PE file
Post by: hutch-- on January 04, 2005, 05:10:16 PM
No, A member of the MZ header is required to locate the PE header. The PE specification includes the MZ 16 bit real mode header.
Title: Re: Check for valid PE file
Post by: Jibz on January 04, 2005, 05:25:23 PM
Yes, the MZ header is required .. at least enough of it that the Windows PE loader interprets it as such -- i.e. if the MZ signature and the e_lfanew members are correct, the PE file should work. Of course it's best to use a valid stub executable to be safe :U.
Title: Re: Check for valid PE file
Post by: billy on January 05, 2005, 01:34:30 AM
Check out the code in the MS .NET beta 2005 for the crt stub that gets called upon loading (it calls a C/C++ application's main() function). It contains the code to scan a PE, and will even determine if it has .NET managed code in it. FYI, 2005 beta is a free download, but you must agree to send feedback.
Title: Re: Check for valid PE file
Post by: hutch-- on January 05, 2005, 01:42:43 AM
The 2005 beta does not allow distribution of code developed using it. The licence is specific to testing and giving fedback to Microsoft.
Title: Re: Check for valid PE file
Post by: billy on January 05, 2005, 01:54:03 AM
Quote from: hutch-- on January 05, 2005, 01:42:43 AM
The 2005 beta does not allow distribution of code developed using it. The licence is specific to testing and giving fedback to Microsoft.

I didn't mean to distribute the code, but it can be studied to see how the PE header is layed out.
Title: Re: Check for valid PE file
Post by: Tedd on January 05, 2005, 01:11:59 PM
Quote from: hutch-- on January 04, 2005, 05:10:16 PM
No, A member of the MZ header is required to locate the PE header. The PE specification includes the MZ 16 bit real mode header.

Yes, but if the stub isn't there, then the PE header would start at offset 0 :bg (...usually)
Title: Re: Check for valid PE file
Post by: hutch-- on January 05, 2005, 02:32:41 PM
 :bg

Quote
Yes, but if the stub isn't there, then the PE header would start at offset 0  (...usually)

This would be fine if you wrote your own EXE file loader but Microsoft PE specs require an MZ header, properly a dos stub and without it, it ain't a PE file. Try it through any version windows loader and it will go bang.
Title: Re: Check for valid PE file
Post by: Tedd on January 06, 2005, 01:21:52 PM
Fairy nuff :8)
Title: Re: Check for valid PE file
Post by: chetnik on January 06, 2005, 08:09:27 PM
PE executable should start with MZ header, and that's what it sais in PE specification. I haven't seen, yet, PEexecutable without MZ at the begining.

Best regards =)

p.s.
I don't like MapViewOfFile =))) I like old Unix file handling =))))))))))
Title: Re: Check for valid PE file
Post by: Polizei on January 23, 2005, 01:09:57 AM
Well donkey, you could install a SEH frame when checking the file for a VALID additional header (LE/PE/etc.) i had some files on my PC that don't have a GOOD DOS-header and the [e_lfanew] member was junk :( so that causes a "General Protection Fault" (Win98) By the way, I use the same code when openning PE files in my PElib :>
Title: Re: Check for valid PE file
Post by: chetnik on January 24, 2005, 03:06:38 PM
Yap I do the same thing in my file finder/check valid PE code, install SEH, and if progy fails to read from memory where lefnew is pointing(bad mz header) it just unmaps file, closes handles and continues with search  :dazzled: :dazzled:

Best regards