Hi I am a little confused about what I am doing wrong.
I have some code like this . . .
include \masm32\include\masm32rt.inc
.data
m_inFile BYTE "test.txt",0
m_inArray DWORD 0
.code
start:
print chr$("Test output one.",13,10)
mov m_inArray, arrfile$(addr m_inFile)
print chr$("Test output two.",13,10)
exit
end start
. . . and a file (test.txt) like this . . .
one b
two w
three w
four w
five b
six w
seven w
eight w
nine b
ten w
elven b
twelve w
thirteen b
fourteen w
fifteen b
sixteen w
seventeen b
eighteen w
. . . if I end my file on one of the lines that ends with a w it works fine. If I end it one of the lines that ends with a b it does not work at all. It seems like the line number is what affects if it works or not. Any idea how I could get it to work for files with any number of lines.
I am using masm32 on windows 7. I thought it said I had downloaded version 10 but when I look at the editor it says "quick editor 4".
Thanks for your help.
Try this, you must allocate the array first. Check the reference material in the HLHELP help file.
IF 0 ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
Build this template with "CONSOLE ASSEMBLE AND LINK"
ENDIF ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
include \masm32\include\masm32rt.inc
.code
start:
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
call main
inkey
exit
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
main proc
LOCAL array :DWORD
LOCAL rval :DWORD
LOCAL ptxt :DWORD
mov array, arralloc$(64) ; allocate the array
test arrset$(array,1,"item 1"), 0 ; load text into array
test arrset$(array,2,"item 2"), 0
test arrset$(array,3,"item 3"), 0
test arrset$(array,4,"item 4"), 0
test arrset$(array,5,"item 5"), 0
test arrset$(array,6,"item 6"), 0
test arrset$(array,7,"item 7"), 0
test arrset$(array,8,"item 8"), 0
mov ptxt, arrget$(array,8) ; display results in reverse order
print ptxt,13,10
mov ptxt, arrget$(array,7)
print ptxt,13,10
mov ptxt, arrget$(array,6)
print ptxt,13,10
mov ptxt, arrget$(array,5)
print ptxt,13,10
mov ptxt, arrget$(array,4)
print ptxt,13,10
mov ptxt, arrget$(array,3)
print ptxt,13,10
mov ptxt, arrget$(array,2)
print ptxt,13,10
mov ptxt, arrget$(array,1)
print ptxt,13,10
mov rval, arrfree$(array) ; free the array memory
ret
main endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
end start
Seems here that a debugger will show what happen.
Check Recall in MasmBasic (http://www.masm32.com/board/index.php?topic=12460.0)'s help file at \masm32\MasmBasic\MbGuide.rtf
Quote from: hutch-- on August 03, 2011, 08:36:50 AM
Try this, you must allocate the array first. Check the reference material in the HLHELP help file.
The help file says "The array is created by this function and the return value is the handle of the array." So it seems like I do not need to allocate it first, but I will try it out. The code sample you showed is not using the arrfile macro. Thanks for your help.
there is an example that uses arrfile$ in \masm32\examples\exampl10\dynarray\loadfile
the macro is defined in \masm32\macros\macros.asm
arrfile$ MACRO file_name ;; load multiline text file into array
EXITM <rv(arrfile,reparg(file_name))>
ENDM
the arrfile function is defined in \masm32\m32lib\arrfile.asm
arrfile proc file_name:DWORD
LOCAL arr :DWORD
LOCAL hMem :DWORD
LOCAL flen :DWORD
LOCAL lcnt :DWORD
LOCAL pbuf :DWORD
LOCAL spos :DWORD
LOCAL void :DWORD
push ebx
mov hMem, InputFile(file_name)
mov flen, ecx
mov lcnt, rv(get_line_count,hMem,flen)
mov arr, arralloc$(lcnt)
mov pbuf, alloc(flen)
mov spos, 0
mov ebx, 1
@@:
mov spos, rv(readline,hMem,pbuf,spos)
mov void, arrset$(arr,ebx,pbuf)
add ebx, 1
cmp ebx, lcnt
jle @B
free pbuf
free hMem
mov eax, arr
pop ebx
ret
arrfile endp
as you can see, the arrfile function calls arrset$
but it looks like you still need to use arrget$ :P
bhy56,
I did write you the example for a reason, trying to re-interpret the help file yielded the results you already had that did not work. There is an overview in the help file that tells you how it works, one array of pointers in fixed memory which holds pointers to either separate allocated memory for each member or a place holder for an empty member.
the example in \masm32\examples\exampl10\dynarray\loadfile makes it look pretty easy :P
Quote from: hutch-- on August 04, 2011, 01:30:48 AM
bhy56,
I did write you the example for a reason, trying to re-interpret the help file yielded the results you already had that did not work. There is an overview in the help file that tells you how it works, one array of pointers in fixed memory which holds pointers to either separate allocated memory for each member or a place holder for an empty member.
I still do not understand what I need to do to make my code open all files. It opens files with a even number of lines and I can work with them and change the strings and print them out. But it crashes when opening a file with a odd number of lines, and does not even get out of the arrfile macro. I tried to allocate the array first with a line like this: "mov hArray, arralloc$(4096)", but it did not help. Sorry I am really new at this. Thanks for your help.
Quote from: dedndave on August 04, 2011, 01:36:34 AM
the example in \masm32\examples\exampl10\dynarray\loadfile makes it look pretty easy :P
It does make it look easy, but I do not see what they are doing that I need to do in order to make it work. I have not yet figured out what GetTickCount does. Do you know where it is defined? Thanks for pointing out this example.
GetTickCount
http://msdn.microsoft.com/en-us/library/ms724408%28v=vs.85%29.aspx
they are using it for demo purposes to show how long it takes :bg
the main points of interest in that demo....
arrfile$("\masm32\include\windows.inc") ;returns a handle
arrcnt$(hArr) ;returns element count
arrget$(hArr,ebx) ;returns an element address (ebx = number)
arrfree$(hArr) ;frees the allocation
Quote from: dedndave on August 04, 2011, 04:25:52 AM
GetTickCount
http://msdn.microsoft.com/en-us/library/ms724408%28v=vs.85%29.aspx
they are using it for demo purposes to show how long it takes :bg
If I use the example (\masm32\examples\exampl10\dynarray\loadfile) to load my file it brakes with a odd number of lines also, just like my code above does. I loaded masm32 on another computer and with both the sample code and the code I posted above, it does not work with a odd number lines in the file.
it may be you have found a real bug :U
i'll play with it tomorrow
it looks as though it searches for carriage return/line feed pairs
is the last line terminated with CR/LF ?
Quote from: dedndave on August 04, 2011, 04:52:36 AM
it may be you have found a real bug :U
i'll play with it tomorrow
it looks as though it searches for carriage return/line feed pairs
is the last line terminated with CR/LF ?
I think I tested for that, and if I remember right it does not seem to make any difference if there is a CR/LF at the end of the file.
I have just read a little the source code.He need to be rewrite.There is things useful with text files he don't do and there isn't a structure to avoid a too big number of parameters in use.
bhy56,
Attack a couple of test files, one that works for you and one that crashes for you. It take the guesswork out of trying to solve your problem.
Hutch,
If you modify the code add this things:
**** a stucture who gives at least the maximum size of the lines (to avoid memory leak).
**** a variable who said if the last line is ended by a 13,10 or not.
Here is a test piece with two files, one with 4 lines, the other with 3 lines. Works fine here.
IF 0 ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
Build this template with "CONSOLE ASSEMBLE AND LINK"
ENDIF ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
include \masm32\include\masm32rt.inc
.code
start:
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
call main
inkey
exit
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
main proc
LOCAL array :DWORD
LOCAL acnt :DWORD
LOCAL rval :DWORD
LOCAL ptxt :DWORD
LOCAL cntr :DWORD
; ------------------------------------------
mov array, arrfile$("4line.txt")
mov acnt, arrcnt$(array)
print str$(acnt)," line count",13,10
mov cntr, 1
@@:
mov ptxt, arrget$(array,cntr)
print ptxt,13,10
add cntr, 1
mov eax, acnt
cmp cntr, eax
jle @B
mov rval, arrfree$(array)
; ------------------------------------------
mov array, arrfile$("3line.txt")
mov acnt, arrcnt$(array)
print str$(acnt)," line count",13,10
mov cntr, 1
@@:
mov ptxt, arrget$(array,cntr)
print ptxt,13,10
add cntr, 1
mov eax, acnt
cmp cntr, eax
jle @B
mov rval, arrfree$(array)
; ------------------------------------------
ret
main endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
end start
Test file 1. 3line.txt
line 1
line 2
line 3
Test file 2. 4line.txt
line 1
line 2
line 3
line 4
Output looks like this.
4 line count
line 1
line 2
line 3
line 4
3 line count
line 1
line 2
line 3
Press any key to continue ...
Yves, it does not need either, the array count is stored in the array at member zero and is retrieved by a function from the address. The memory is deallocated by a function "arrfree$".
The max size off the lines is needed by the user if he want to copy the line in a buffer.
i am able to repeat bhy56's failure
it seems to be with the arrfile function
more specifically, the get_line_count function
i may have a "handle" on it :lol
let me play some more....
You guys must not read documntation.
Yves,
arrlen$
arrlen$ MACRO arr,indx
gets the stored length of each array member. It is stored 4 bytes below the start address like normal with OLE strings.
Dave,
arrcnt$
arrcnt$ MACRO arr
As in the example above.
well, get_line_count does not seem to be te culprit, as i thought
i'll come back to that.....
i did find a little potential bug in arrfile
mov hMem, InputFile(file_name)
mov flen, ecx
mov lcnt, rv(get_line_count,hMem,flen)
mov arr, arralloc$(lcnt)
if the file does not end with a carriage return/line feed pair, get_line_count appends a pair and adds 2 to the size
it returns the updated length in ECX
the "flen" value does not reflect that change, however
this simple change would take care of that
mov hMem, InputFile(file_name)
mov lcnt, rv(get_line_count,hMem,ecx)
mov flen, ecx
mov arr, arralloc$(lcnt)
it may not make a difference, really
when the array is filled, it probably does not need the last cr/lf
but, it makes the function more useful :8)
now, for the crash :P
very strange - i am not sure i understand what is happening, here
i will have to look at the arralloc proc a little closer - it may be overwriting memory that doesn't belong to it
maybe there is something going on that is machine or OS dependant - i dunno
at any rate, by allocating the pbuf buffer before allocating the arr buffer, the crash goes away
original arrfile code (crashes)
mov hMem, InputFile(file_name)
mov flen, ecx
mov lcnt, rv(get_line_count,hMem,flen)
mov arr, arralloc$(lcnt)
mov pbuf, alloc(flen)
modified arrfile code (does not crash)
mov hMem, InputFile(file_name)
mov lcnt, rv(get_line_count,hMem,ecx)
mov flen, ecx
mov pbuf, alloc(flen)
mov arr, arralloc$(lcnt)
it can be simplified a little more....
mov hMem, InputFile(file_name)
mov lcnt, rv(get_line_count,eax,ecx)
mov flen, ecx
mov pbuf, alloc(ecx)
mov arr, arralloc$(lcnt)
now, back to get_line_count
get_line_count proc mem:DWORD,blen:DWORD
; mem = address of loaded file
; blen = length of loaded file
mov ecx, blen
mov edx, ecx
sub edx, 2
mov eax, mem
cmp WORD PTR [eax+edx], 0A0Dh ; if no trailing CRLF
je cntlf
mov WORD PTR [eax+ecx], 0A0Dh ; append CRLF
mov BYTE PTR [eax+ecx+2], 0 ; add terminator
add blen, 2 ; correct blen by 2
cntlf:
or ecx, -1
xor eax, eax
mov edx, mem
@@:
add ecx, 1
cmp BYTE PTR [edx+ecx], 0
je @F
cmp BYTE PTR [edx+ecx], 10 ; count the line feed character
jne @B
add eax, 1
jmp @B
@@:
mov ecx, blen ; return count in EAX, length in ECX
ret
get_line_count endp
not sure why a null is written with the added cr/lf :P
the allocation is zero-init
the loop looks a little strange
maybe i am off, here, but a single access to memory seems like it would be more efficient
get_line_count proc mem:DWORD,blen:DWORD
; mem = address of loaded file
; blen = length of loaded file
mov edx,mem
push ebx
mov ecx,blen
mov eax,0A0Dh
xor ebx,ebx
cmp ax,[edx+ecx-2]
jz cntlf
mov [edx+ecx],ax
add ecx,2
cntlf: mov eax,ebx
jmp short cloop1
cloop0: inc edx
cmp bl,0Ah
jnz cloop1
inc eax
cloop1: mov bl,[edx]
or ebx,ebx
jnz cloop0
pop ebx
ret
get_line_count endp
This counter of lines have the same entries as get_line_count proc mem:DWORD,blen:DWORD.
He is faster .Number of lines in eax
Quote
CompteurLignes PROC uses ebx edi esi pmem:DWORD,taille:DWORD
Local Nblines:DWORD,count,reste
local theEnd:dword
;init
mov Nblines,0
mov reste,0
mov edx,pmem
add edx,taille
mov theEnd,edx
mov edx,pmem
mov esi,edx
and edx,0Fh
.if edx != 0
;search lines in the non align memory
mov ecx,16
sub ecx,edx
@@:
.if byte ptr [esi] != 0
.if word ptr [esi] == 0A0Dh
inc Nblines
.endif
.else
mov eax,Nblines
jmp FindeCompteurLignes
.endif
inc esi
dec ecx
jnz @B
.endif
;esi point on a 16 aligned memory
;count the number of 32 bytes parts
mov edx,0
mov eax,theEnd
sub eax,esi
.if eax == 0
mov eax,Nblines
jmp FindeCompteurLignes
.endif
.if eax < 32
mov reste,eax
mov eax,Nblines
jmp EndNonaligned
.endif
mov edx,0
mov ecx,32
div ecx
mov count,eax
mov reste,edx
;-------------------------- search in aligned part -------------
;init of various register
mov eax, 0d0d0d0dh ; Ascii 10, linefeed
movd xmm6, eax
pshufd xmm6, xmm6, 0 ; linefeeds for comparison in xmm2
mov eax,Nblines ;line counter
;ready
NewBloc:
;------ align 16 needed ----------
;1731187 cycles for 22274 lines
movdqa xmm1,xmm6 ;charge 13
movdqa xmm2,xmm6 ;charge 13
pcmpeqb xmm1,[esi] ;cmp with memory align 16
pcmpeqb xmm2,[esi+16] ;cmp with memory ,align 16
pmovmskb ecx, xmm1 ; result in ecx
pmovmskb edx, xmm2 ; result +16 edx
shl edx,16
add ecx,edx
jz suite
NbLineBreak:
bsf edx, ecx
jz suite
.if word ptr [edx+esi] == 0A0Dh
inc eax
.endif
btr ecx, edx
jmp NbLineBreak
suite:
lea esi,[esi+32]
dec count
jnz NewBloc
EndNonaligned:
.if reste != 0
mov ecx,reste
@@:
.if word ptr [esi] == 0A0Dh
inc eax
.endif
inc esi
dec ecx
jnz @B
.endif
FindeCompteurLignes:
ret
CompteurLignes endp
Here the same sample with other routines ( improved)
Made a modify to view the mx size
i like it, Yves :bg
but - it does not use the oleaut32 functions
which - i am not sure what the advantage is in doing so - lol
i read some documents about BSTR's and i now know less than i did before :red
i can't afford to read many such documents, as i am losing IQ points, fast
perhaps i can enlight you a little on the BSTR.
First they are unicode strings
Second : they are not put anywhere in memory.Its a memory accessible by com object,like dll implementing an interface and having hard need of shared memory.There is a lot of theorie on the subject.
It's not the case here (hopefully).
Quote from: hutch-- on August 04, 2011, 05:59:48 AM
bhy56,
Attack a couple of test files, one that works for you and one that crashes for you. It take the guesswork out of trying to solve your problem.
I tested from one to twenty lines. I did not test with long ( 40 +char ) lines. It does not seem to make any difference if there is a empty line at the end or not. I made these files in notepad. When it does not work I get a little pause and then "t01.exe has stopped working. Windows can check online for a solution to the problem." I attached two files one that works and one that does not. Notice that I ended the lines with a b if it broke the program, and a w if it worked (when that was the last line in the file). The files look something like this. Notice that there are a few odd numbered lines that worked (namely three and seven) .
one b
two w
three w
four w
five b
six w
seven w
eight w
nine b
ten w
elven b
twelve w
thirteen b
fourteen w
fifteen b
sixteen w
seventeen b
eighteen w
nineteen b
twenty w
well - there is something wrong with the arr macros
i fixed one thing, and another pops up
i suspect that the spos macro is crashing for some reason
the oleaut32 stuff seems to be very picky about BSTR's and allocation/deallocation
to make matters worse, the bugs may be OS dependant
0 one b
1 two w
2 three w
3 four w
4 five b
5 six w
6 seven w
7 eight w
8 nine b
9 ten w
10 elven b
11 twelve w
12 thirteen b
13 fourteen w
14 fifteen b
15 sixteen w
16 seventeen b
17 eighteen w
18 nineteen b
19 twenty w
0 one b
1 two w
2 three w
3 four w
4 five b
5 six w
6 seven w
7 eight w
8 nine b
9 ten w
10 elven b
11 twelve w
12 thirteen b
13 fourteen w
14 fifteen b
15 sixteen w
16 seventeen b
17 eighteen w
18 nineteen bThe textfiles are ok.
Quoteinclude \masm32\MasmBasic\MasmBasic.inc ; download (http://www.masm32.com/board/index.php?topic=12460.0)
Init
Recall "testworked.txt", L1$()
For_ n=0 To eax-1
Print Str$(n), Tb$, L1$(n), CrLf$
Next
Recall "testbrok.txt", L2$()
For_ n=0 To eax-1
Print Str$(n), Tb$, L2$(n), CrLf$
Next
Inkey "bye"
Exit
end start
You have got the good sample who broke.
I have it with the sample of hutch and the textfile testbrok.txt
Quote
FAULTING_SOURCE_CODE:
10:
11: ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
12:
13: call main
> 14: inkey
15: exit
16:
17: ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
18:
Here the result given by windg.I m not sure of it
I can duplicate the crash but it does not occur while the code is running, only on exit. This is on the file with the brok uneven line count.
REPEAT 1024
call main ; this works fine
ENDM
invoke ExitProcess,0 ; crashes here. comment out and it stops at "inkey"
inkey
exit ; then crashes here which is just ExitProcess.
Changed this line in "arrfile" and it no longer crashes on exit.
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
arrfilex proc file_name:DWORD
LOCAL arr :DWORD
LOCAL hMem :DWORD
LOCAL flen :DWORD
LOCAL lcnt :DWORD
LOCAL pbuf :DWORD
LOCAL spos :DWORD
LOCAL void :DWORD
push ebx
mov hMem, InputFile(file_name)
mov flen, ecx
add flen, 2 ; <<<<<<<<<<<<<<<<<<<<<<<<<<<< added this
mov lcnt, rv(get_line_count,hMem,flen)
mov arr, arralloc$(lcnt)
mov pbuf, alloc(flen)
mov spos, 0
mov ebx, 1
@@:
mov spos, rv(readline,hMem,pbuf,spos)
mov void, arrset$(arr,ebx,pbuf)
add ebx, 1
cmp ebx, lcnt
jle @B
free pbuf
free hMem
mov eax, arr
pop ebx
ret
arrfilex endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
Quote from: hutch-- on August 05, 2011, 01:39:31 AM
Changed this line in "arrfile" and it no longer crashes on exit.
I added the line, but could you tell me how to re-assemble this so that it is using the new macro?
Thanks
Quote
I added the line, but could you tell me how to re-assemble this so that it is using the new macro?
The arrfile.asm is part of the masm32 lib,(\masm32\m32lib) and you need to rebuild the library.
or - just copy the code into your source file, name it something unique, and modify it
that isn't exactly in the spirit of the license - lol
but it works to test the code
it isn't that dificult to modify, assemble, and replace a single module in the masm32 LIB file
it just takes a little experience with LIB, which is handled by the LINKer, i guess
if you go to the command prompt and type
LIB /?
it will show you the switches
MSDN also has a page on LIB - google it
bhy56,
Just try the last algo I posted directly in your source code, you will have to manually invoke the procedure and MOV eax to the same return value but this is the easiest way to test it and see if it does the job. I might also say thanks for finding such an obscure bug, when I wrote this system about 4 years ago I tested the hell out of it but never found the bug you reported.
Let us know how it turns out.
:bg
Don't bve afraid to let us know if the solution worked OK for you. Feedback is important when it comes to fixing obscure bugs in code.
Quote from: hutch-- on August 06, 2011, 02:17:24 AM
:bg
Don't bve afraid to let us know if the solution worked OK for you. Feedback is important when it comes to fixing obscure bugs in code.
I will test it out a little more and see if works for me and then let you know, but it will probably be next week. Thanks for your help so far.
:U
i gave you a shitload of feedback
but you ignored it
mov hMem, InputFile(file_name)
mov lcnt, rv(get_line_count,eax,ecx)
mov flen, ecx
mov pbuf, alloc(ecx)
mov arr, arralloc$(lcnt)
No I didn't, I read it all and modified the code. :red
ok - i have a solution that seems to work
it is odd (PUN) that it has to be this way :P
the arralloc function starts off like this
mov eax, mcnt ; load the member count into EAX
add eax, 1 ; correct for 1 based array
lea eax, [eax*4] ; multiply it by 4 for memory size
that code makes perfect sense
element 0 is used for the count value
elements 1 through n are for the bstr's
for some strange reason, the spos function wants an even number of elements to work with (not including element 0)
these 3 things makes it work for me....
1) adjust the flen value
2) allocate the pbuf before the arr buff
this code fixes both of those issues
mov hMem, InputFile(file_name)
mov lcnt, rv(get_line_count,eax,ecx)
mov flen, ecx
mov pbuf, alloc(ecx)
mov arr, arralloc$(lcnt)
3) make the arralloc function always return an even number of elements (not including element 0)
mov eax, mcnt ; load the member count into EAX
add eax, 1 ; correct for 1 based array
or eax, 1 ; usable element count always even
lea eax, [eax*4] ; multiply it by 4 for memory size
here is a test piece...
Dave,
As best as I can tell at the moment, the problem was a memory overwrite out of range and it appears to be within the granularity of how OLE memory is allocated, my guess is the even numbered counts were fitting into the extra couple of bytes from 4 byte granularity while the odd counts were going out of bounds. OLE memory is no joy to track but its my guess that its a GP fault problem, the error i was getting on exit was an attempt to address 00000000h which is not a valid protected mode address range and it may be because one less than the required count is being allocated and it faults on the last one.
i kind of gave up trying to isolate it any further :bg
the OLE functions are known to crash programs unexpectedly if one little hair is out of place
that describes the symptom rather well :P
the only other times i have seen programs crash silently (no dr watson) is when there is a stack imbalance
it may have something to do with the way "null" bstr's are allocated and deallocated
An write outside the allocate memory is more current than a defaut of the OLE memory.
Why use it ?
A HeapAlloc is perfect here.There is no need of shared memory (shared between further processes) and no need of OLE.
OLE string memory in either the unicode or ansi forms are designed for high count small allocations and also have their length stored below the start address. HeapAlloc() and most of the other strategies have serious fragmentation problems while VirtualAlloc() is not well suited for large count small allocations due to granularity problems. The pointer array is in GlobalAlloc() fixed memory as you get linear address but where every new or changed string involves a de-allocation then re-allocation, OLE is the correct strategy to use.
very interesting, Hutch
it sounds ideal for something like a text editor, where individual lines or groups of lines grow, shrink and get moved around or deleted
just hafta get past the BaSTRd thing :P
Quote
HeapAlloc() and most of the other strategies have serious fragmentation problems
Only if size of the buffers are in bytes,round of the size to 16,32 give better performances.
Quote
but where every new or changed string involves a de-allocation then re-allocation, OLE is the correct strategy to use.
HeapReAlloc made this very well.Here it is just a question of time, What is the faster ?
HeapAlloc is always faster, ToutEnMasm. By the way, folks, you realise that this is all old stuff - see Is there any reason why this code should fail? (http://www.masm32.com/board/index.php?topic=14268.0) :bg
HeapAlloc() may be faster but that will not help you with the fragmentation problems when you have 5 million strings of variable length. The cost of aligning large counts to an arbitrary alignment starts to become very high in terms of memory usage. Remember that with a dynamic array design of this type you are performing an allocation for every string that contains content in the array.
Quote from: hutch-- on August 05, 2011, 05:33:17 AM
bhy56,
Just try the last algo I posted directly in your source code, you will have to manually invoke the procedure and MOV eax to the same return value but this is the easiest way to test it and see if it does the job. I might also say thanks for finding such an obscure bug, when I wrote this system about 4 years ago I tested the hell out of it but never found the bug you reported.
Let us know how it turns out.
Thanks for writing these and thanks for your help. I tried to set up a test with arrfile as a local function but do not seem to be getting it right, because now it will not work at all. I am not that good at assembly (yet) and think I must be calling the function wrong. Any help would be appreciated. Thanks
code attached
Quote
HeapAlloc() may be faster but that will not help you with the fragmentation problems when you have 5 million strings of variable length. The cost of aligning large counts to an arbitrary alignment starts to become very high in terms of memory usage
I have study a little the code in the arr*.asm.EACH line of the text files is put in a separated buffer.This is a disaster in terms of time.As say microsoft , allocating and desallocating memory is slow.
If you test your proc on a text of 5 million strings ,only one things is granted.You have got here a good pass time.
It's for that than the sample i have posted avoid a too much number of memory access.
bhy56,
i have looked at your "t01.asm" file
first, at the beginning of the file, you will want a prototype for arrfile2...
arrfile2 proto :DWORD
now, you may use INVOKE to call it
now, to call it, you have passed the address of a pointer to the filename
just pass the address of the filename string
.data
m_inFile BYTE "test.txt",0
m_inArray DWORD 0
.code
start:
print chr$("Test output one.",13,10)
lea EBX, m_inArray
push EBX
call arrfile2
pop EBX
mov m_inArray,EAX
print chr$("Test output two.",13,10)
exit
also, there is no need to balance the stack after the call
that is because the StdCall convention is used
so, try this...
include \masm32\include\masm32rt.inc
arrfile2 PROTO :DWORD
get_line_count PROTO :DWORD,:DWORD
.data
m_inFile db "test.txt",0
m_inArray dd 0
.code
start:
print chr$("Test output one.",13,10)
INVOKE arrfile2,offset m_inFile
mov m_inArray,eax
print uhex$(eax),13,10
arrfree$ m_inArray
print chr$("Test output two.",13,10)
exit
i have added the arrfree$ to deallocate the array before exit
if you don't free allocated memory, you may have a "memory leak"
Yves,
With a design that handles variable length strings, you have no other choice than to perform an allocation for every string that contains data. Now this can be from a couple of bytes to many hundreds of megabytes. Fixed arrays are simple and much faster and you can allocate the pointers and the data slots in one allocation but it either must use massive amounts of memory where each slot is at least large enough to hold the longest string or you must limit the string length to preserve memory.
This is why you also have a variable length string array that will handle far larger string arrays than a fixed array can handle. The price is it is a lot slower when allocated and deallocated but it can do what a fixed array cannot do.
Quote
With a design that handles variable length strings, you have no other choice than to perform an allocation for every string that contains data.
Two soluces to do that.
*** Made an array with the pointers and the lenght of the variables (the example I have posted do that)
data are not re-copied.This method is usable for Textfiles. The translatorr use that and all my other applies also.
*** copy the variables in a single buffer (grow dynamically by increment) and made at the same time the same array as upper.
This method is usefull when adding variables one by one.
You need just a QWORD to find the adress and the size of the variable.
Just for fun, here a little testbed comparing arrfile$ and Recall on a) 73*windows.inc and b) 7*a file composed of all includes, with about 9 MB, which implies it does not fit into the cache. Results are pretty similar for both cases. String #2 is printed in all cases to check if the code works.
Create the alltest.inc from the command line of \masm32\include with copy *.inc alltest.inc
Testing arrfile$ on Windows.inc
WINDOWS.INC for 32 bit MASM (Version 1.4c RELEASE April 2008)
1984 milliseconds for arrfile$
Testing Recall on Windows.inc
WINDOWS.INC for 32 bit MASM (Version 1.4c RELEASE April 2008)
235 milliseconds for Recall
Testing arrfile$ on alltest.inc
; ===========================================
2000 milliseconds for arrfile$
Testing Recall on alltest.inc
; ===========================================
296 milliseconds for Recall
I like HeapAlloc. It is fast :bg