able/windows-nt
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
windows-nt/Source/XPSP1/NT/sdktools/perftool/stat/readme

620 lines
22 KiB
Plaintext
Raw Normal View History

Add source files
2020-09-26 03:20:57 -05:00
This directory and its subdirectory contain source and
binary files for the statitics support packages that can
be run across multiple platforms.
The directory is organized as follows:
1) .\ ->
a) stat.c (single c source)
b) makefile.rst (common for windows, OS2 16, and OS2386)
c) makefile (for Windows NT)
d) sources (sources file for Windows NT)
e) The header file, teststat.h, required for building
the dlls is under ..\inc\.
2) .\win (FOR WINDOWS)
a) .\src (contains the remaining .asm file, and the
module def file)
b) .\bin (the binary statwin.dll file)
3) .\WIN32 (FOR WIN32 APPS)
a) .\src (contains the .def file and an i386 sub-dir.
and a mips subdir, each containing an asm file)
b) .\bin (the binary file)
4) .\os2286 (FOR 16 bit OS/2 Cruiser and Sloop apps)
a) .\src (the module def file)
b) .\bin (the binary stat286.dll file)
5) .\os2386 (FOR 32 bit OS/2 Cruiser apps)
a) .\src (the module def file)
b) .\bin (the binary stat386.dll file)
**********************************************************************
To build an application that uses a statistics DLL:
--------------------------------------------------
To use one of the above binaries, please read the USAGE NOTES at the
end of this document. Please copy the teststat.h file from this
directory to the directory where you are building your application.
Copy the relevant .dll to your libpath.
It is essential that you define the type of system you are building
your application for, since the header file uses some special types
that are dependent on the system. While compiling your application,
add the following flag: -DXXX where XXX stands for one of:
WIN - for Windows applications
OS2286 - for 16 bit OS/2 applications
OS2386 - for 32 bit OS/2 applications
WIN32 - for Win32 applications.
**********************************************************************
To build one of the dlls:
------------------------
If building a Windows, OS2 16 or OS/2 32 bit dll:
--------------------------------------------------------
a) Copy the stat.c file found under this directory
and teststat.h from ..\inc\. to a local directory.
Copy the .asm file from win\src if building for WIN.
b) Also copy the "makefile.rst" from here to the same directory.
c) From ???\src copy the remaining files to the local directory,
where ??? represents win, os2286 or os2386.
d) Edit "makefile.rst" to define the system that you are making the
dll for. Eg. if you are making the dll for windows, remove
the comment sign (#) from the line "WIN=TRUE" in the makefile
and ensure that the other system defines (OS2286 and OS2386)
are commented out.
f) Type "nmake -f makefile.rst" and the dll will be created for
you. (Ensure that your development environment is set up for
the right system).
If building the Win32 dll:
-------------------------
a) Copy stat.c, makefile and sources files found under this
directory and teststat.h from ..\inc\. to a local directory.
b) tc the win32\src directory to your local directory. This
will create an i386 (or mips) sub-directory containing an asm
file on your local machine.
c) From the directory where you have your sources file, type
"build -xxx statw32" from the command line, where xxx represents
your target system. It is 386 by default.
d) A binary file "statw32.dll" will be created along with the
.obj file under .\xxx\obj where xxx is your target system.
It is i386 by default.
In case you have any questions, or if you run into any problems,
contact vaidy (936-7812).
*****************************************************************
USAGE NOTES
-----------
This is the user's guide to using TestStat.dll, the
statistical package. In case of questions contact vaidy (936-
7812).
This document describes the use of each of the functions
available through this module and then demonstrates the use
of these routines through an example.
This module provides basic statistical routines which can be
used to compute average, min, max, standard deviation, and
statistical convergence. Statistical convergence of the
average is determined by the number of test iterations required
for the average to converge to a "stable" value. The number of
iterations required is computed on the fly as the data is
collected, so that the caller is informed when enough data is
collected (the average is stable). Stable averages obtained in
this way can be compared to other stable averages obtained under
different experimental conditions with known confidence levels.
Notes in this document describe the meanings of stability and
confidence more formally.
In addition to the functionlity described above, this module
also provides routines that generate normally distributed
random numbers. Three routines are provided that return
random numbers within a specified boundary, a set of uniformly
distributed random numbers within the range 0 to 1 and a normally
distributed set of numbers around a mean, which satisfy a given
mean and standard deviation.
1) TestStatOpen:
-------------
Description: Allocates an instance data array for the data set
and other global data structures required by the high level
functions.
USHORT FAR PASCAL
TestStatOpen (
USHORT usMinIterations,
USHORT usMaxIterations
);
usMinIterations - The minimum number of iterations
that the calling application has to run before
the convergence algorithm may be used.
usMaxIterations - The maximum number of iterations
that the test program may run. The maximum
acceptable value is 64K. An internal data array
of usMaxIterations of ULONGs is allocated. The
caller should bear this in mind when setting this
parameter.
Remarks: This routine should be called before the first
call to TestStatInit. If usMinIterations is zero, an error
code is returned. If usMinIterations is greater than
usMaxIterations an error code is returned. If usMinIterations
is equal to usMaxIterations, TestStatConverge will return TRUE
after that many iterations. This function frees the caller
from the responsibility of allocating any data storage or
book-keeping.
Return Value: 0 if the call succeeded.
An error code indicating a failure. The
error code may be one of:
STAT_ERROR_ILLEGAL_MIN_ITER
STAT_ERROR_ILLEGAL_MAX_ITER
STAT_ERROR_ALLOC_FAILED
See also: TestStatInit, TestStatConverge, TestStatValues,
TestStatClose.
2) TestStatInit:
-------------
Description: Initializes variables required by the convergence
and statistics routines.
VOID FAR PASCAL
TestStatInit (
VOID
);
Remarks: This routine should be called before the first
call to TestStatConverge and after each call to
TestStatValues, if you want to converge on a new set of
data.
Return Value: None.
See also: TestStatOpen, TestStatClose, TestStatConverge,
TestStatValues.
3) TestStatConverge:
-----------------
Description: Automatically computes number of iterations
required for 95% confidence in data obtained.
BOOL FAR PASCAL
TestStatConverge (
ULONG ulNewDataPoint,
);
ulNewDataPoint - The data point obtained for the
current iteration.
Remarks: This routine should be called for each
iteration of the test. The first call to this routine
should be preceded by a call to TestStatInit. The test
program should check for the return value and should stop
the test as soon as a TRUE is returned.
In making tests of significance, sometimes errors will be
encountered in the results concerning an hypothesis tested.
The hypothesis is that the difference between the actual
mean in one experiment and the actual mean in a second
experiment is less than a specified value. This
difference is expressed as a percentage of the first
experiment's mean. We call this difference, the "precision"
of the comparison.
If the assumption is true and the results of the tests leads one
to believe that it is false, the condition is described as a
TYPE I error. If the assumption is false and the test results
show that the two means are within the prescribed difference, the
condition is described as a TYPE II error.
The probability of TYPE I error is set by the significance level
of the test. Choosing a small probability of one type of error,
increases the probability of the other type.
The routines in this module operate on the following set of
assumed parameters:
95% confidence that if the means differ by less than 5% they
are really the same, and 85% confidence that if the means
differ by more than 5% that they are really different.
The algorithm in this module uses these assumptions to determine
the number of iterations needed to achieve these levels of
confidence.
The reason for emphasizing TYPE II error is that a TYPE I error
indicates that the means differ, when in fact, they are the same.
If they differ, we will usually explore why, and in doing so, will
discover that they are not really different after all. If, on the
other hand, we get a TYPE II error, then this means that the
results show no difference, whereas the means really are
different. This is to be avoided since if means don't differ
from one run to the next, we are unlikely to look further into
the problem.
When additional iterations are forced by a high usMinIterations,
then the resulting precision will usually be less than 5%.
Conversely, when usMaxIterations are reached without converging,
then the precision will be greater than 5%. The precision returned
by TestStatValues will indicate how meaningful the comparisons of
two means will be.
Return Value: FALSE if further iterations are required for
the test to converge or usMinIterations has
not been reached.
TRUE if already converged or maximum limit
on iterations has been reached.
See also: TestStatOpen, TestStatInit, TestStatValues,
TestStatClose.
4) TestStatValues:
----------------
Description: Automatically computes a number of useful
statistical values for a given set of data.
VOID FAR PASCAL
TestStatValues (
PSZ pszOutputString,
USHORT usOutlierFactor,
PULONG * pulDataArray,
PUSHORT pcusElementsInArray,
PUSHORT pcusDiscardedValues,
);
pszOutputString - A pointer to a string buffer to which
output data may be returned. The minimum size of
the buffer should be 81 bytes. The string will
be a NULL terminated ascii string.
usOutlierFactor - Factor that defines the range of
acceptable data values. A value of zero will ignore
this factor and all data will be considered
valid.
pulDataArray - A pointer to the data array. If the outlier
factor has been chosen, this array has as many elements
as there were good data points in the data set. Else,
all the data points are contained in the data array.
pcusElementsInArray - The number of elements in the array
pointed to by puDataArray.
pcusDiscardedValues - pointer to the number of data
points discarded based upon the outlier factor.
Remarks: This routine should be called only once for
each test, normally after TestStatConverge has returned
TRUE. Any call to this should be followed by a call to
TestStatInit before the next call to TestStatConverge,
if you want to converge on a new set of data. The Outlier
factor decides the range of acceptable values in the data set.
The format of the returned string will be (as in C):
"%4u %10lu %10lu %10lu %6u %5u %10lu %4u %2u ".
These represent the mode number, mean, minimum, maximum,
the number of iterations completed, the precision, the standard
deviation, number of points discarded, and, the outlier factor
from the data set. The mode number will always be zero.
The precision will be 5% in case test results converged before
the limit on the maximum iterations is reached. Otherwise,
it returns the precision of the results gathered.
The precision value in this case assumes that the Type I error
and Type II error probabilities are 85% and 95% respectively.
The outlier factor determines along with the standard deviation
any abnormal data points. Any data point that does not satisfy:
[Mean - (SDev * OF)] < Data Point < [Mean + (SDev * OF)],
where SDev is the standard deviation computed with good data
points and OF is the outlier factor, is left out in the
statistics computation. The standard deviation is
recomputed and this process is repeated until there are no
abnormal data entries in the data set. The number of
outliers that were discarded is also returned to the calling
program. To ignore the outlier factor and this process of
elimination, the outlier factor may be set to zero.
Otherwise, the outlier factor should be at least 2 in order
for the results to be meaningful.
Return Value: None
See also: TestStatOpen, TestStatClose, TestStatInit,
TestStatConverge
5) TestStatClose:
-------------
Description: Deallocates instance data structures and
all memory allocated by TestStatOpen and TestStatInit.
VOID FAR PASCAL
TestStatClose (
VOID
);
Remarks: This routine should be called after the last
call to TestStatValues. A call to this must be followed by a
call to TestStatOpen and TestStatInit, in that order, before
the application calls TestStatConverge and TestStatValues.
Return Value: None
See also: TestStatOpen, TestStatInit, TestStatConverge,
TestStatValues.
------------------------------------------------------------
Usage of Statistical routines for convergence and values: TestApp
------------------------------------------------------------------
#define MIN_ITERATION 3
#define MAX_ITERATION 200
#define OUTLIER_FACTOR 4
Body of test application
{
USHORT usMinIteration = MIN_ITERATION;
USHORT usMaxIteration = MAX_ITERATION;
ULONG ulDataForCurrentIter;
ULONG far *pulDataArray; // make sure you have the "far" for 16 bit.
char chOutputBufferForString [81];
USHORT usOutlierFactor = OUTLIER_FACTOR;
USHORT cusDiscardedValues;
USHORT cusElementsInArray;
:
:
if (!TestStatOpen (usMinIteration,
usMaxIteration)) {
// Data Array could not be allocated.
// Cannot do convergence/statistics routines;
// Check parameters to call;
}
do { // for each test or if need to run convergence again
TestStatInit ()
// Initialize test variables;
:
do { // convergence loop; do until a
// TRUE is returned
// Start the timer;
// Test operation;
// Stop the timer;
ulDataForCurrentIter = // get the elapsed time for
// operation;
} while (!TestStatConverge (ulDataForCurrentIter));
// the data set has converged. Call the Statistics
// routine for the values and output data
TestStatValues (OutputBufferForString,
usOutlierFactor,
&pulDataArray,
&cusDiscardedValues,
&cusElementsInArray,
);
// the OutputBufferForString array has all the data.
// iDiscardedValues has the number of discarded values
//
} while (//more tests or need to converge on new data set )
:
:
TestStatClose();
:
}
-------------------------------------------------------------------
Random Number Generation Routines:
6) TestStatUniRand:
---------------
Description: Returns a number within the range of 0 to 1 based on
a starting seed.
double FAR PASCAL
TestStatUniRand (
VOID
);
Remarks: This routine returns a set of uniformly distributed numbers
between 0 and 1, on being, called repeatedly. TestStatUniRand makes
use of the multiplicative congruential algorithm discussed in Knuth,
Vol. II, Chapter 3. A starting seed is chosen along with a
multiplier and a modulus values. The seed for the next iteration is
computed from these values as follows:
Temp Value = X * A, where,
X is the current seed value and A is the multiplier. The remainder
of the division of this value by the modulus identifier is
determined. This will be the seed for the next iteration. This
value is divided by the modulus value to obtain a normalized value
(that lies between 0 and 1). This normalized value is returned to
the caller.
Through experiments, Sullivans, W. L has determined that a good set of
values is returned by selecting one of the 9 following values as
starting seeds:
32347753, 52142147, 52142123, 53214215, 23521425, 42321479,
20302541, 32524125, 42152159.
TestStatUniRand uses 32347753 as the starting seed. A good set of
values, mentioned above, implies that for the given seed, it takes
a very large number of iterations, before the set of returned values
is repeated. The following values have been chosen for the
multiplier and the modulus by M.C. Pike and I.D. Hill (reference):
Multiplier - 3125
Modulus id - 67108864
Return Value: A double float between 0 and 1.
See also: TestStatShortRand, TestStatRand, TestStatNormDist.
7) TestStatShortRand:
-----------------
Description: Returns a number within the range of 0 to 65535 based on
a starting seed.
USHORT FAR PASCAL
TestStatShortRand (
VOID
);
Remarks: This routine returns a set of uniformly distributed numbers
between 0 and 65535, on being, called repeatedly. TestStatShortRand makes
use of the multiplicative congruential algorithm discussed in Knuth,
Vol. II, Chapter 3. A starting seed is chosen along with a
multiplier and a modulus values. The seed for the next iteration is
computed from these values as follows:
Temp Value = X * A, where,
X is the current seed value and A is the multiplier. The remainder
of the division of this value by the modulus identifier is
determined. This will be the seed for the next iteration. This
value is multiplied by 65535 and divided by the modulus value
to obtain a value between 0 and 65535. This value is returned to
the caller.
Through experiments, Sullivans, W. L has determined that a good set of
values is returned by selecting one of the 9 following values as
starting seeds:
32347753, 52142147, 52142123, 53214215, 23521425, 42321479,
20302541, 32524125, 42152159.
TestStatShortRand uses 32347753 as the starting seed. A good set of
values, mentioned above, implies that for the given seed, it takes
a very large number of iterations, before the set of returned values
is repeated. The following values have been chosen for the
multiplier and the modulus by M.C. Pike and I.D. Hill (reference):
Multiplier - 3125
Modulus id - 67108864
Return Value: A USHORT between 0 and 65535.
See also: TestStatUniRand, TestStatRand, TestStatNormDist.
8) TestStatRand:
------------
Description: Returns a uniformly distributed random number within
a specified range.
ULONG FAR PASCAL
TestStatRand (
ULONG ulLower,
ULONG ulUpper
);
ulLower - Specifies the lower boundary of the desired random
number. Should be atleast 1 in value.
ulUpper - Specifies the upper boundary of the desired random
number. May not exceed 67108863.
Remarks: TestStatRand calls TestStatNorm for obtaining a normalized
random number. The value obtained from TestStatNorm is then
multiplied by the range (i.e. the difference between ulUpper and
ulLower). The computed value is then added to the lower limit
and the resulting number is returned. It should be noted that both
ulLower and ulUpper are included in the range of returned random
numbers.
Return Value: A random number within the specified range.
See Also: TestStatShortRand, TestStatUniRand, TestStatNormDist.
9) TestStatNormDist:
----------------
Description: With every call, returns a number that forms a set
of points whose mean is approximately the input mean and whose
standard deviation is nearly equal to the input standard deviation.
A normally distributed set of points is generated.
LONG FAR PASCAL
TestStatNormDist (
ULONG ulMean,
USHORT usSDev
);
Remarks: This routine uses a formula discussed in 'Random Number
Generation and Testing', IBM Data Processing Techniques, C20-8011
and 'Tuning an Operating System for General Purpose Use', Russell
P. Blake, Online Conferences (info. to be filled in).
TestStatNormDist makes use of TestStatShortRand to get a set of
uniformly distributed numbers. It generates a point around the
input mean using the following formula:
14
_
lRetVal <- Mean + ( -7 + >_ TestStatShortRand ()) * Std. Dev
i=1
The set of points generated with several calls to this routine
will be uniformly distributed with a mean of about the input mean
and a standard deviation of approximately the input standard
deviation. The returned value may be negative, too, depending
upon the values returned by TestStatShortRand and the input standard
deviation!
Return Value: A long integer.
See also: TestStatShortRand, TestStatUniRand, TestStatRand.
Reference in a new issue Copy permalink