How do I find where my large two_dimension arrays is on my computer - memory or

[Coral88]

How do I find where my large two_dimension arrays is on my computer - memory or hd?

I am dimensioning a two dimension arrays in Powerbasic(see URL below and reference to MASM) ::)
DIM AR8 (1 TO 20000000,1 TO 2) AS LOCAL LONG ' twenty millions by two

I can see the HD light coming up when reading and writing to this large arrays.

The reading and writing is getting slower than before when the array was only
dimensioned to 100000 by two.
When dimensioning 100000 by two NO lights flashed on my HD.

I think part of the array now is spilling over onto the virtual memory(VM) of my hard drive.

I've 1 Gig memory RAM with an Hyperthreading Celeron 2.4 Ghz.
WIN 2000 Pro, Version 5, build 2195 with service pack 4.
Here is the config of my VM-> 
http://home.brisnet.org.au/~aufempen/PB/PB_forum_post003259.jpg

Question: Is there a way of making sure this array stays in the 1G memory RAM instead of
spilling over into the virutal memory which is the hard drive?
:cheekygreen:

THis question has been posted to but could not be answered: :boohoo:
http://www.powerbasic.com/support/forums/Forum5/HTML/003259.html
http://www.powerbasic.com/support/forums/Forum5/HTML/003259-3.html

It has been suggested by one of the Forum Powerbasic member to contact MASM forum.
Coral88

[sluggy]

AFAIK there is no way to ensure this under modern Windows - it is the memory manager, and you have to rely on its discretion whether your app's memory goes to the swap file or not.
Your memory request is large but not excessive: you are requesting space for 40 million longs, a long is 8 bytes (2 dwords), you want 320 million bytes, which is 312MB. An app i am currently working on extracts & manipulates data between two databases, and routinely uses between 500 and 600 MB while executing (due to all the hash tables that are set up to transform data).

Of course, if you are too slow processing whatever it is that you are storing in memory, then you could always quit using Power Basic and use pure assembly  :bdg :toothy :P

[MichaelW]

A 160MB (or 320MB?) array on a 1GB system running Windows 2000 should not be a problem. What is the memory usage statistic for your program in Task Manager? You might be able to get a better idea of what is happening with System Monitor. There are ~~1000 counters available, spread over ~45 performance objects. For starters you could try Memory - Available Mbytes, Paging File - % Usage, and perhaps Cache - Fast Read Not Possibles/sec.

[zooba]

And any counter with 'Bytes' in it in the Process object (you'll have to have your program running first before you can add them though)

[Coral88]

Of course, if you are too slow processing whatever it is that you are storing in memory, then you could always quit using Power Basic and use pure assembly  :bdg :toothy :P

Sluggy


In 1997 I bought a
A86/A386 Macro Assembler -
D86/D386 Debugger
from Eric Isaacson.

I still have the diskette with assembler and debugger plus the manual

I did a bit of work with it .  Very beginners stuff and I liked it.

How much this has changed from A86/A386 to MASM?   Heaps I would say?

Just a trivial question:
From PowerBasic I can easely call a  Machine Language( ML) subroutine.
Could I make a couple of essential subroutine in ML to accelerate the process?

Where would you the see the benefit of such subroutine being written in MASM?
Even if I can double the process speed of the subroutine  I'll be very happy.
Would you think I could  double the speed of execution of my subroutine if they were writen in MASM?

By the way, the large array I am using is for Neural_network_back-propagation crunching.

Coral88

[Coral88]

A 160MB (or 320MB?) array on a 1GB system running Windows 2000 should not be a problem. What is the memory usage statistic for your program in Task Manager? You might be able to get a better idea of what is happening with System Monitor. There are ~~1000 counters available, spread over ~45 performance objects. For starters you could try Memory - Available Mbytes, Paging File - % Usage, and perhaps Cache - Fast Read Not Possibles/sec.

MIcheal
My memory usage in the Task manager passes from 226688K to 384164K  when I run my array dimensioned at:
DIM AR2 (1 TO 20000000,1 TO 2) AS LOCAL LONG

Sluggy has calculated " you are requesting space for 40 million longs, a long is 8 bytes (2 dwords), you want 320 million bytes, which is 312MB."

How come my Memory monitor is telling me I am using only 157476K.
1 long integer is 4 bytes=(32-bits)
A Dword is 32 bits
What's this 157476K?
I am lost now

====
Michael you mentioned
"System Monitor. There are ~~1000 counters available, spread over ~45 performance objects. "

How do you find these 1000 counters available? even 10 would be OK  :clap:

You know something that  never tried before - Could you please explain a newbie like me how to get these 1000 counters?
It see at the bottom of my Task Manager

Physical memory
Total 1048044K
Avalaible 720712 (constantly changes by a small value)
System Cache 311648  (constantly changes by a small value)

Kernel memory
Total 81096
Paged 63200
Non paged 17896

coral88

[zooba]

Sluggy has calculated " you are requesting space for 40 million longs, a long is 8 bytes (2 dwords), you want 320 million bytes, which is 312MB."

How come my Memory monitor is telling me I am using only 157476K.
1 long integer is 4 bytes=(32-bits)
A Dword is 32 bits
What's this 157476K?
I am lost now

...

How do you find these 1000 counters available? even 10 would be OK  :clap:

Sluggy calculated his figure assuming a 'LONG' was 8-bytes, as it is for recent implementations of C et al. Older implementations of languages only have 4-bytes for a LONG. Since 156476K is approximately half of Sluggy's figure and your LONG is half the size of his LONG, the numbers work out fine.

Check out Control Panel/Administrative Tools/Performance for the performance counters :U

[MichaelW]

Code Select Expand


DIM AR2 (1 TO 20000000,1 TO 2) AS LOCAL LONG


Does the LOCAL specify Thread Local Storage? If so, could this be part of the problem?

[Coral88]

Check out Control Panel/Administrative Tools/Performance for the performance counters :U

Thanks for the tip.
I did not realized all these counters were there.  :U

Coral88