measuring performance

[juliomacedo]

I'd like to know if this is a good way of comparing different algorithms (or different implementations of the same algorithm) using HLA:

var

myTimer: timer;

(...)

myTimer.start();
//statements that implements one of the solutions
myTimer.stop();
stdout.put(eax); //eax contains the time in milliseconds taken to execute statements

Thanks

Julio Macedo

[Sevag.K]

It's not the most accurate and it does have a tiny bit of overhead, but it's great for most tasks.  If you want more accureate, you'll have to go to the processor level.  eg:

Code Select Expand


static
    timeStart   :int64;
    timeEnd     :int64;

...
rdtsc();
mov( eax, (type dword timeStart));
mov( edx, (type dword timeStart[4]));

// your test routine here...

rdtsc();
sub( (type dword timeStart), eax );
mov( eax, (type dword timeEnd) );
sbb( (type dword timeStart[4]), edx );
mov( edx, (type dword timeEnd[4]));
stdout.put( "Time :", timeEnd, nl );

...


[MichaelW]

Julio,

This test code (don't laugh, I've had only about 30 minutes with HLA):

Code Select Expand


program timertest;

#include( "stdlib.hhf" );
#include( "timer.hhf" );

var
    myTimer:timer;
    prev:dword;

begin timertest;

    myTimer.create();
    myTimer.start();

    for( mov( 0, ebx ); ebx < 1_000_000; inc( ebx )) do
      myTimer.checkPoint();
      //stdout.put( eax, nl );
      if( eax != prev ) then
        push( eax );
        sub( prev, eax );
        stdout.put( eax, nl );
        pop( prev );
    endif;
    endfor;

    myTimer.stop();
 
    stdout.put( "Press Enter to exit . . ." );
    stdin.flushInput();
    stdin.a_gets();

end timertest;


Shows that despite the one-millisecond precision the timer has an effective resolution of 10 milliseconds. For reasonable timing accuracy with such a timer the timed interval must be at least several hundred milliseconds, or more probably several seconds. Most of the code I have timed required not more than a few hundred processor clock cycles, and at today's clock speeds this translates to an execution time well under one microsecond. To get the timed interval up to something reasonable you can run the code in a loop with a suitable loop count. You can calculate the time for a single loop as the total time divided by the loop count. The calculated time will include the loop overhead. If this is not acceptable you can compensate by timing an empty loop, and subtracting the time for the empty loop from the time for the working loop, before dividing by the loop count.

Or you could use a higher resolution timer, the high-resolution performance counter for example, or do the timing by counting processor clock cycles.

hth

[juliomacedo]

It seems RDSTC is really the best way of measuring code performance but when making several consecutive tests using RDSTC, I noted that results varies a lot...

Shouldn't it give us a constant value since a given cpu instruction takes always the same number of cpu cycles to be executed?

Why does it vary too much?

How much is RDSTC measures affected by other tasks being executed?

[juliomacedo]

Of course, I meant RDTSC, not RDSTC...  :green

[MichaelW]

How much is RDSTC measures affected by other tasks being executed?

AFAIK it is the other tasks, or more accurately, multitasking that is responsible for most of the variation. There are several methods you can use to minimize the variation. Take a look at the "Code timing macros" topic in the Laboratory. The macros as they currently are generally produce very repeatable results, but I have timed a few pieces of code where I could not get repeatable results no matter how I arranged the code, or how many times I looped the code, even with REALTIME_PRIORITY_CLASS. Several knowledgeable members have suggested methods of correcting this and other problems (Frank and RATCH to name two, but there may have been others). Sooner or later I will get around to trying the suggestions.

I wonder if HLA provides any method of passing a MASM macro through to MASM.

[Sevag.K]

I wonder if HLA provides any method of passing a MASM macro through to MASM.

If your're using ML.exe as the back end, you can pass MASM code using #asm .. #endasm blocks.  Don't quiet know if this will work with macros, but it's worth a try.

[juliomacedo]

Michael,

I read the topic about "code timing macros" and there are several interesting comments there.

I was wandering about a way of using your macro in HLA code and I thought a way must be using #asm...#endasm... I need to try it...

I also need to understand your macros and why it is better than simply using RDTSC and reading value in EDX:EAX...