Big Files in resources

[ossama]

i am writing a setup program , i use the files in resource file,the files are big in size,i use the LoadResource function ,but if the size of file is bigger then the free memory it will fail,
how can i extract fies from the setup executable ? (files are in the order of giga bytes)

[ramguru]

I suggest not using resource-section for storing big files. Real installer does it in much different way. Maybe you'll consider using poasm and its feature (INCBIN):

Code Select Expand


.data
ptr_file1:
INCBIN setup\file1.exe
ptr_file2:
INCBIN setup\file2.txt

So in this case you already have valid pointers

[donkey]

I agree with ramguru here, resource files is not the way to do this. GoAsm has INCBIN available as well or you can concatenate the file to your executable and memory map it or read it in chunks. Using INCBIN has disadvantages in that it will include the file in the data section, not the best place for multi-gigabytes for sure. If it was up to me I would be using multiple files, a 1 or 2 gig install file is a bit much IMHO, a whole lot of cabs would be much easier to handle and update. You should really be looking at a different scheme.

Donkey

[ossama]

you can concatenate the file to your executable and memory map it or read it in chunks

yes,good idea,
is there a way to add the files data using - i dont know - some instructions in masm and let the linker do the concatenate process for me?

[donkey]

Copy from a DOS window works fine. Or if you like you can use the file functions from the API, read the executable and write it to a new file, write the rest of the data at the end of it.

Donkey

[asmfan]

but if the size of file is bigger then the free memory it will fail

Resources are the only correct way IMHO. Who said that if you store your files in other section it won't fail on loading? Resource section is discardable thus on loading each file separately you will discard one after other consuming less memory than using files in ".data" section. Nevertheless if there will be insufficient physical and pagefile memory - your program will fail aniway.

[Vortex]

Hi ossama,

If you are coding your project with Masm, you can convert your file to MS COFF object module using Hutch's file data assembler. It comes with the Masm32 package. After, you can link this object file with your main module. This method allows you to embed binary data ( file ) into the data section of your final executable. Another alternative is to convert your file to compiled resource file ( . res ) using bin2res

[donkey]

but if the size of file is bigger then the free memory it will fail

Resources are the only correct way IMHO. Who said that if you store your files in other section it won't fail on loading? Resource section is discardable thus on loading each file separately you will discard one after other consuming less memory than using files in ".data" section. Nevertheless if there will be insufficient physical and pagefile memory - your program will fail aniway.

I don't completely agree here, if the files are very large which they seem to be as he says they are in the GB range, they should be stored separately in cabs and extracted from those. Imagine a 2 GB install executable, I wouldn't be downloading it. For smaller more manageable files say up to 100 MB the resource section would work but it is better to just append it to the end of the executable and read it directly.

Edgar

[Vortex]

I agree with Edgar. Since it's the case with very large files, it would be preferable to manage compressed files. Appending the data to the end of the executable is another good solution. Peter Kankowski has a nice article about this method.

[Jackal]

i have done this.. this should work for you or be close.

Code Select Expand



WriteAppFile proc FileID:DWORD, FileName:DWORD
   Invoke lstrcpy, ADDR Buffer, ADDR AppPathBuff
   Invoke CreateDirectory, addr Buffer, 0
   Invoke lstrcat, addr Buffer, addr BackSlash
   Invoke lstrcat, addr Buffer, FileName

   Invoke FindResource, NULL, FileID, RT_RCDATA
   mov [hResource],eax
   Invoke SizeofResource, NULL, hResource
   mov MEMSIZE, eax 
   invoke GlobalAlloc,GMEM_FIXED, MEMSIZE
   mov hMem, eax
   Invoke LoadResource, NULL, hResource
   Invoke LockResource, eax
   mov hMem, eax
   invoke CreateFile, addr Buffer, GENERIC_WRITE, 0, 0, CREATE_ALWAYS, FILE_ATTRIBUTE_ARCHIVE, 0
   mov hFile, eax
   Invoke WriteFile, hFile, hMem, MEMSIZE, addr SizeReadWrite, NULL
   invoke CloseHandle, hFile
   Invoke FreeResource, hResource
   Invoke GlobalFree, hMem
   ret
WriteAppFile endp


[ossama]

thank you Jackal very much,
but - after the  many replies to this topic - i prefer to use the concatenation of files to the setup executable , with this way  i can read the files in chunks (small chunks of 4kb for example) so for big files i will save memory.but i use resources it will ,not let me take this opportunity.

[Jackal]

i would imagine you can do the same thing with resources increasing the pointer. I may be wrong but that would be a thought in my mind. either way good luck with what ever method you choose. The real fun begins when you write the uninstaller that self deletes ;) I have one working and know a few methods to do this if you want to know how i did it.

[ossama]

i would imagine you can do the same thing with resources increasing the pointer. I may be wrong but that would be a thought in my mind. either way good luck with what ever method you choose. The real fun begins when you write the uninstaller that self deletes ;) I have one working and know a few methods to do this if you want to know how i did it.

1) how can you increase the pointer (to read small chunks of file data) using resources.
2) about the uninstaller,you said a fiew methods to do that, please post your methods (new topic about installer/uninstaller in masm is good idea).
regards OSSAMA

[donkey]

DEP mode has unfortunately killed my method of self deleting files, pushing the deletion code onto the stack, exiting the DLL then deleting it on RET. This is no longer a valid way of creating self deleting DLL's using RunDLL32, I will have to look at others some day but it is not a pressing issue for me.

[ossama]

all your posts about installers/uninstaller are welcome here:
http://www.masm32.com/board/index.php?topic=8441.0
regards OSSAMA