in Iczelion's tutorial, he taught the way to read file into memory. but he only mentioned the case that 1 file was read into memory.
what if i need 100 files read into memory?
should each file has a "hFileRead HANDLE" "pMemory DWORD" "hMapFile HANDLE"? which makes 300 dwords in memory ?
can i use these 3 dwords to read all the files?
like this:
read file1 , get handles(3 dwords) , get memory address for the content of file1 , free handles
read file2 , get handles(3 dwords) , get memory address for the content of file2 , free handles
read file3 , get handles(3 dwords) , get memory address for the content of file3 , free handles
......
in this way i save 200 dwords of memory space. can i ?
PS: i only need to read files, i don't write to them.
icezlions tut on memory mapped files isn't the best cause he has you map a view of the entire thing at once which isn't practical for very large files. a cache of the file is kept in mem too, so lots of files being mapped would clog up your systems memory even after your program is closed ;\, here's a much more intelligent approach, this memory maps files in typically 64kilobyte parts and it disables caching, you can work with files up 18 exabytes in size(the 64bit hardware limit) as a result, and is great for multiple files.
this function calls a callback of your choice with the current memory it has a view mapped off and its size
include masm32rt.inc
MyCallBack proto :DWORD,:DWORD
MapLargeFile proto :DWORD,:DWORD
.data
thefile db "yourfile.txt",0
.code
start:
invoke MapLargeFile,addr thefile,addr MyCallBack
invoke ExitProcess,0
MapLargeFile proc iFile:DWORD,iFunc:DWORD
LOCAL bAlign:SYSTEM_INFO
LOCAL hFile:DWORD
LOCAL pbFile:DWORD
LOCAL iMapFile:DWORD
LOCAL qwFileSize:LARGE_INTEGER
LOCAL qwFileOffset:LARGE_INTEGER
LOCAL dwBytesInBlock:DWORD
invoke GetSystemInfo,addr bAlign
mov qwFileSize.LowPart,0
mov qwFileSize.HighPart,0
mov qwFileOffset.LowPart,0
mov qwFileOffset.HighPart,0
invoke CreateFile,iFile,GENERIC_READ,FILE_SHARE_READ,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_ARCHIVE or FILE_FLAG_NO_BUFFERING or FILE_FLAG_OVERLAPPED,0
cmp eax,INVALID_HANDLE_VALUE
je @Error
mov hFile,eax
invoke CreateFileMapping,hFile,NULL,PAGE_READONLY,0,0,NULL
mov iMapFile,eax
invoke GetFileSize,hFile, addr qwFileSize.HighPart
mov qwFileSize.LowPart,eax
.while TRUE
;if the file is larger than AllocationGranularity(usually 64kb) then use AllocationGranuarity to start
;otherwise use the file size
mov eax,qwFileSize.HighPart
.if eax == 0
mov eax, qwFileSize.LowPart
mov edx, bAlign.dwAllocationGranularity
.if eax < edx
mov dwBytesInBlock,eax
.else
mov dwBytesInBlock,edx
.endif
.else
mov edx,bAlign.dwAllocationGranularity
mov dwBytesInBlock,edx
.endif
invoke MapViewOfFile,iMapFile,FILE_MAP_READ,qwFileOffset.HighPart,qwFileOffset.LowPart,dwBytesInBlock
mov pbFile,eax
push dwBytesInBlock
push pbFile
call iFunc
;we unmap the current view and mapview the next AllocationGranularity(usually 64kbs)
invoke UnmapViewOfFile,pbFile
;Increment Offset quadword
mov ecx,qwFileOffset.LowPart
mov edx,qwFileOffset.HighPart
mov eax,dwBytesInBlock
mov ebx,0
add ecx,eax
adc edx,ebx
mov qwFileOffset.LowPart,ecx
mov qwFileOffset.HighPart,edx
;Decrement Size quadword
mov ecx,qwFileSize.LowPart
mov edx,qwFileSize.HighPart
mov eax,dwBytesInBlock
mov ebx,0
sub ecx,eax
sbb edx,ebx
mov qwFileSize.LowPart,ecx
mov qwFileSize.HighPart,edx
.if edx == 0
.if ecx == 0
.break
.endif
.endif
.endw
invoke CloseHandle,iMapFile
invoke CloseHandle,hFile
@Error:
ret
MapLargeFile endp
MyCallBack proc iMem:DWORD,iLen:DWORD
;do whatever with mem
ret
MyCallBack endp
end start
keep in mind memory mapped files may not be the best choice here either, for top to bottom reading, ReadFile is much faster, for random access memory map is better, a big problem with memory mapped files too is they use aton of CPU.
Unless you need all 100 files open at once, you can just open them one at a time, and re-use the variables for holding the handles.
I wouldn't recommend using memory-mapping unless the files are potentially quite large (at least a few MB) and you require random access. If they're not too big and you just need to read them start to end, allocate a block of memory (GlobalAlloc, or HeapAlloc), ReadFile into that, do your stuff, and then re-use the memory for the next file.
i need to read all files at once, and close them all at once when the program ends. and the files are not partially read, always in a whole. the file sizes may vary from xxKB to xxMB.
then which function is better?
Quote from: E^cube on May 10, 2010, 10:01:41 AM
icezlions tut on memory mapped files isn't the best cause he has you map a view of the entire thing at once which isn't practical for very large files. a cache of the file is kept in mem too, so lots of files being mapped would clog up your systems memory even after your program is closed ;\, here's a much more intelligent approach, this memory maps files in typically 64kilobyte parts and it disables caching, you can work with files up 18 exabytes in size(the 64bit hardware limit) as a result, and is great for multiple files.
this function calls a callback of your choice with the current memory it has a view mapped off and its size
include masm32rt.inc
MyCallBack proto :DWORD,:DWORD
MapLargeFile proto :DWORD,:DWORD
...
...
MyCallBack endp
end start
my program is quite hard coded when it comes to loading files into memory, file names are all set, load modes are all set (it's a game requiring textures, sounds, etc) .
i have a hard time understanding these codes. :eek and i like simple codes, since flexibility is not required.
i just got to know about HeapAlloc() and Fopen() , and for OPENGL texture , there is AuxDIBImageLoad() .
there are so many ways to open a file or load file into memory, which is best?
Easiest is probaly to use the InputFile Macro on smaller files
mov eax, InputFile(chr$("c:\test.txt"))
(ecx contains File Length)
For textures, you don't need the files to be open (and it only takes up extra memory) - you open them, load them into memory, extract/convert the data, close the file, and use the data to create the texture.
You can use LoadBitmap, and then GetDIBits to get the data, then you can delete the bmp returned by LoadBitmap, and return the pixel data. The pixel data can then be used directly with glTexImage2D.
Also, to avoid loading so many texture files, you should put related textures into the same image (separate them by two pixels) and fit the texture co-ordinates accordingly. And if your game has any notion of levels/sections, only load the ones for the current section at the beginning of that section - that should keep memory usage and load time down (which also means you can use bigger textures for each.)
there are so many ways to do one thing so i would stay with one way for now, then try other ways later on.
let's say now i only need 1 TGA with size of 128x128, format 32bit, BGRA , and there is only 1 level in my game (i have just started the project)
i have successfully loaded all data of the TGA file into memory. say address = 0x07000000-0x07010030 , pMapFile dd contains the address 0x07000000 . the data in memory are like these:
x x x x x x x x x x x x x x x x x x 00 00 ff ff 00 ff 00 ff ff 00 00 ff ...... (header 18 bytes, color of each pixel follows)
i am following NEHE's OGL tutorial:
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=06 (loading bmp image)
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=24 (loading TGA image)
i tried to convert the codes into ASM (omitting some functions such as: test the file exist or not, test the format of image, etc). but the outcome is wrong.
my code for creating GL texture:
=============================
loadtexture proc
invoke glGenTextures,1,gltexture[0]
invoke glBindTexture,GL_TEXTURE_2D,gltexture[0]
invoke glTexParameteri,GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR
invoke glTexParameteri,GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR
mov eax,pMapFile
add eax,18
invoke glTexImage2D,GL_TEXTURE_2D,0,4,128,128,0,GL_BGRA_EXT,GL_UNSIGNED_BYTE,eax
ret
loadtexture endp
=============================
drawscene proc
...
invoke glBindTexture,GL_TEXTURE_2D,gltexture[0]
invoke glBegin,GL_QUADS
invoke glTexCoord2i,0,0
invoke glVertex3f,0bf800000h,0bf800000h,3f800000h ; ---- note that 0bf800000h means float value -1.0
invoke glTexCoord2i,0,127
invoke glVertex3f,3f800000h,0bf800000h,3f800000h
invoke glTexCoord2i,127,127
invoke glVertex3f,3f800000h,3f800000h,3f800000h
invoke glTexCoord2i,127,0
invoke glVertex3f,0bf800000h,3f800000h,3f800000h
invoke glEnd
...
drawscene endp
=============================
what is wrong ?
i don't understand the purpose of:
glGenTexture(1,&texture[0].texID)
glBindTexture(GL_TEXTURE_2D,texture[0].texID)
glTexImage2D(...... , texture[0].imagedata)
what are " &texture[0].texID" "texture[0].texID" "texture[0].imagedata"
are they memory addresses? or just ID numbers?
how do they relate to the memory address 0x07000000 ?
in my program, the polygon only shows red/black strips , while the TGA contains red/green/blue colors.