This example tests recursively calling new thread up to a specified thread count. The example uses 1024 and from testing XP SP3 runs out of threads at 2048 so it should run OK. Each thread is suspended when it calls the next with WaitForSingleObject() and with all of the threads loaded and waiting for a keypress the processor usage stays around the same percentage as when it started so it appears to be reasonably efficient.
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
include \masm32\include\masm32rt.inc
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
comment * -----------------------------------------------------
Build this template with
"CONSOLE ASSEMBLE AND LINK"
----------------------------------------------------- *
start_in_thread PROTO :DWORD,:DWORD
THREAD_ARGS STRUCT
flag dd ? ; thread start flag
stak dd ? ; optional thread stack size
arg1 dd ?
arg2 dd ?
arg3 dd ?
arg4 dd ?
arg5 dd ?
arg6 dd ?
arg7 dd ?
arg8 dd ?
THREAD_ARGS ENDS
td equ <1024> ; thread recursion depth
.code
start:
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
call main
inkey "Thats all folks, press any key to exit"
exit
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
main proc
LOCAL thst :THREAD_ARGS
LOCAL hThread :DWORD
LOCAL p_proc :DWORD
mov p_proc, OFFSET work_thread
; -------------------------------
; load the THREAD_ARGS structure
; -------------------------------
mov thst.arg1, 0
mov thst.arg2, 0
mov thst.arg3, 0
mov thst.arg4, 0
mov thst.stak, 1024 ; 1k stack for first thread
mov hThread, rv(start_in_thread,p_proc,ADDR thst)
invoke WaitForSingleObject,hThread,INFINITE
ret
main endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
start_in_thread proc pproc:DWORD,pstruct:DWORD
LOCAL tID :DWORD
LOCAL hThread :DWORD
push esi
mov esi, pstruct
mov (THREAD_ARGS PTR [esi]).flag, 1 ; set the flag
mov hThread, rv(CreateThread,NULL,(THREAD_ARGS PTR [esi]).stak,pproc,pstruct,NULL,ADDR tID)
; ---------------------------------------------
; block new threads from starting until args
; are saved in the current thread being created
; ---------------------------------------------
spinlock:
cmp (THREAD_ARGS PTR [esi]).flag, 0 ; loop until created thread clears the flag
jne spinlock
; ---------------------------------------------
mov eax, hThread
pop esi
ret
start_in_thread endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
work_thread proc pstruct:DWORD
LOCAL args[4] :DWORD
LOCAL thst :THREAD_ARGS
LOCAL hThread :DWORD
LOCAL p_proc :DWORD
mov p_proc, OFFSET work_thread
; --------------------------------------------------------------
; load the args to local array before the caller flag is cleared
; --------------------------------------------------------------
mov eax, pstruct
m2m args[0], (THREAD_ARGS PTR [eax]).arg1
; ------------------------------------
; release the thread starter procedure
; ------------------------------------
mov (THREAD_ARGS PTR [eax]).flag, 0 ; clear the flag
; ***********************************************************************
; ***********************************************************************
m2m thst.arg1, args[0]
add thst.arg1, 1 ; increment the thread depth indicator
print "Thread depth "
print str$(thst.arg1)," starting",13,10
cmp thst.arg1, td ; see if it has reached the thread depth limit yet
jge leaving
mov thst.stak, 64
mov hThread, rv(start_in_thread,p_proc,ADDR thst)
invoke WaitForSingleObject,hThread,INFINITE
leaving:
.if thst.arg1 == td ; only wait on the highest thread
inkey "Press a key to exit all of the threads"
.endif
print "Thread depth "
print str$(thst.arg1)," leaving",13,10
; ***********************************************************************
; ***********************************************************************
ret
work_thread endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
end start
Hmm ? It took about 3 seconds but it got to 1024 just fine. Seems to be slow until 75 or so then just jumps to 1024. Memory usage/Private working set is reported at 28.8MBs. This is on Win7 x64.
The threads exit fine in about half a second :toothy
P.S.: You must love it when you make my PC/OS drop down and give you 1024 :cheekygreen:
Changed from 1024 to 4096 and got
QuoteThread depth 1598 starting
then it did nothing but sit there (25% cpu) - q6600,win7 x64
Mine stopped at 1598 as well, obiously reached a safety limit. Maybe it would be interesting to test a native x64 executable that does the exact same thing ! Anyone up to the challenge to convert to x64 ?
I am running 32 bit XP Sp3 and I get this.
Thread depth 2027 starting
Thread depth 2028 starting
Thread depth 2029 starting
Thread depth 2030 starting
Thread depth 2031 starting <<< stops here
This machine has 4 gig of memory installed but I am not sure if this matters. I would imagine that there is more overhead in 64 bit due to the double data size.
Under Windows 2000 with 512MB it stops at 2023.
From the SDK
Quote
The number of threads a process can create is limited by the available virtual memory. By default, every thread has one megabyte of stack space. Therefore, you can create at most 2,028 threads.
At most eh? 2023,2028,2031...pick a number
I dropped the stack size down to a small number and it does not make any difference, the 2k limit seems to be for other reasons, probably the system is not designed to handle more.
PS: I should have closed the thread after each is signalled.
invoke WaitForSingleObject,hThread,INFINITE
invoke CloseHandle,hThread
http://blogs.msdn.com/oldnewthing/archive/2005/07/29/444912.aspx
running xp mce2005 sp2 with 1 Gb
1024 threads, Hutch
(that's all folks)
i suppose i should look into upgrading to 4 Gb, huh
Tested on Windows XP Pro Sp3 with 1 Gb RAM :
1024 Threads
Thanks Slugsnack, that a good article.
Just tweaked the example with the info in the site that Slugnack referred and I get this result.
Thread depth 16377 starting
Thread depth 16378 starting
Thread depth 16379 starting
Thread depth 16380 starting
Thread depth 16381 starting
Thread depth 16382 starting
Thread depth 16383 starting
Thread depth 16384 starting
Press a key to exit all of the threads
I have tried it up to just under 32k and it runs out of puff there.
This is the code mod.
mov hThread, rv(CreateThread,NULL,(THREAD_ARGS PTR [esi]).stak,pproc,pstruct,00010000h,ADDR tID)
Its the constant 00010000h that does the work here so that the specified stack size is used.
are you sure that's the only change Hutch ?
i made the mod and get the same result - 1024 threads
Dave,
This is the test piece with the minor changes done.
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
include \masm32\include\masm32rt.inc
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
comment * -----------------------------------------------------
Build this template with
"CONSOLE ASSEMBLE AND LINK"
----------------------------------------------------- *
start_in_thread PROTO :DWORD,:DWORD
THREAD_ARGS STRUCT
flag dd ? ; thread start flag
stak dd ? ; optional thread stack size
arg1 dd ?
arg2 dd ?
arg3 dd ?
arg4 dd ?
arg5 dd ?
arg6 dd ?
arg7 dd ?
arg8 dd ?
THREAD_ARGS ENDS
td equ <16384> ; thread recursion depth
.code
start:
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
call main
inkey "THATS ALL FOLKS, press any key to exit ...."
exit
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
main proc
LOCAL thst :THREAD_ARGS
LOCAL hThread :DWORD
LOCAL p_proc :DWORD
mov p_proc, OFFSET work_thread
; -------------------------------
; load the THREAD_ARGS structure
; -------------------------------
mov thst.arg1, 0
mov thst.arg2, 0
mov thst.arg3, 0
mov thst.arg4, 0
mov thst.stak, 128 ; 1k stack for first thread
mov hThread, rv(start_in_thread,p_proc,ADDR thst)
invoke WaitForSingleObject,hThread,INFINITE
invoke CloseHandle,hThread
ret
main endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
start_in_thread proc pproc:DWORD,pstruct:DWORD
LOCAL tID :DWORD
LOCAL hThread :DWORD
push esi
mov esi, pstruct
mov (THREAD_ARGS PTR [esi]).flag, 1 ; set the flag
mov hThread, rv(CreateThread,NULL,(THREAD_ARGS PTR [esi]).stak,pproc,pstruct,00010000h,ADDR tID)
; ---------------------------------------------
; block new threads from starting until args
; are saved in the current thread being created
; ---------------------------------------------
spinlock:
cmp (THREAD_ARGS PTR [esi]).flag, 0 ; loop until created thread clears the flag
jne spinlock
; ---------------------------------------------
mov eax, hThread
pop esi
ret
start_in_thread endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
work_thread proc pstruct:DWORD
LOCAL args[4] :DWORD
LOCAL thst :THREAD_ARGS
LOCAL hThread :DWORD
LOCAL p_proc :DWORD
mov p_proc, OFFSET work_thread
; --------------------------------------------------------------
; load the args to local array before the caller flag is cleared
; --------------------------------------------------------------
mov eax, pstruct
m2m args[0], (THREAD_ARGS PTR [eax]).arg1
; ------------------------------------
; release the thread starter procedure
; ------------------------------------
mov (THREAD_ARGS PTR [eax]).flag, 0 ; clear the flag
; ***********************************************************************
; ***********************************************************************
m2m thst.arg1, args[0]
add thst.arg1, 1 ; increment the thread depth indicator
print "Thread depth "
print str$(thst.arg1)," starting",13,10
cmp thst.arg1, td ; see if it has reached the thread depth limit yet
jge leaving
mov thst.stak, 64 ; <<<<<<<<<< change done here
mov hThread, rv(start_in_thread,p_proc,ADDR thst)
invoke WaitForSingleObject,hThread,INFINITE
invoke CloseHandle,hThread
leaving:
.if thst.arg1 == td ; only wait on the highest thread
inkey "Press a key to exit all of the threads"
.endif
print "Thread depth "
print str$(thst.arg1)," leaving",13,10
; ***********************************************************************
; ***********************************************************************
ret
work_thread endp
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
end start
thanks Hutch - i got up to 16384 that time
xp mce2005 sp2 - 1 Gb ram
Hi,
Thread depth 2031 starting
Thread depth 2032 starting
^C
F:\WORK>
Windows 2000 Pro SP4
256M mem
(from system information)
261M Total physical mem
115M Available physical mem
525M virtual mem
I got up to 2025 on Windows 7 32 bit RTM.
4GB RAM. However I am not sure that the amount of physical memory will affect the memory available to an application. That is the whole point of virtual memory and paging. In theory, every usermode application should only have access to 2GB memory
the issue is the stack - each thread is alloted 1 Mb stack space by default
it has to be physical memory - not virtual
i am surprised you do not get more threads on win 7
it may be a predetermined limit
Quote from: dedndave on October 04, 2009, 05:33:41 PM
the issue is the stack - each thread is alloted 1 Mb stack space by default
it has to be physical memory - not virtual
i am surprised you do not get more threads on win 7
it may be a predetermined limit
Are you sure stack has to be physical memory ? If it's the case that the stack is the bottleneck here because of the fact is has to be in physical memory. Then going by that theory, surely once we exhaust physical memory with this program, no other programs can create new threads ?
Slugsnack,
Did you use the later mod that used your reference to MSDN, I added the constant so it then used the preset stack size and it went from 2k to 16k with no problems. I am using XP SP3 32 bit with 4 gig memory. I ran it with a much larger number but it stops creating threads at about 31k.
Quote from: hutch-- on October 04, 2009, 09:47:01 PM
Slugsnack,
Did you use the later mod that used your reference to MSDN, I added the constant so it then used the preset stack size and it went from 2k to 16k with no problems. I am using XP SP3 32 bit with 4 gig memory. I ran it with a much larger number but it stops creating threads at about 31k.
Yes, assembled the source from your post : http://www.masm32.com/board/index.php?topic=12445.msg95526#msg95526
Just tested again, stops at 2026. Also, in case it's relevant.. Page file size = 3582MB
well - i have learned (the hard way - lol) that stack memory may be assigned to a thread or procedure
but not dedicated to it until it uses it - so until you actually use some of the huge stack-space, it is "available" to other threads
it seems illogical to have stack space virtualized to disk - it would constantly be switching in and out
I would be surprised if stack space was shoved out to virtual memory as it would involve some serious performance hits but the Microsoft reference material appears to indicate it is a memory based limit so it appears that stack stays loaded in memory when other allocation types are swapped to disk.
It has been an interesting aside that you can create about 2000 threads but the original purpose of the test piece was to test thread nesting and the necessary overhead in the OS keeping track of all of the suspended threads so with 2000 threads nested you have 1999 wait states pending thread termination signalling which is impressive in its capacity and speed.
Watching the test app in performance monitor, the process virtual bytes rapidly increase from ~12,000,000 to ~2,000,000,000 and the process page file bytes steadily increase from ~270,000 to ~27,000,000.