Results Analysis Guide

Boest Results Analysis Guide

This guide looks at how to analyse the results of a Performance Test using the results database.

Overview

Download the results database from Sourceforge.net. This includes scripts for building the tables and stored procedures that process the results. Currently this is available for SQL Server.

The stored procedures are utility procedures for loading the Boest log file and the output from Performance Monitor into the database, the data is then processed and loaded into the main tables of the database and from there we can analyse our results.

Installing the Database

Download an uzip the package.

Using SQL Server Management Studio create a new database. Logon to this database and run the SQL package in the file ddl_tables_views.sql. This will create the required tables and views.

Edit the SP_LoadStaging.sql file and update the value of the @sourcefolder variable and enter a path to a folder on your SQL server where you can put the Boest log file and Performance Monitor trace files. Then run the SQL in the following SQL files to create the stored procedures.

SP_LoadStaging
SP_NewTestRun
SP_ProcessResults
SP_ProcessViews

Load Database

After running a stress test we can load the results into the database and then analyse the results. To load the database,

Copy the performance monitor trace file and the Boest log file to your source folder, identified by the @sourcefolder above.
Edit the trace files produced by Performance Monitor and remove all double quoues (").
From SQL Server Management Studio execute the ProcessResults stored procedure providing the following parameters (Note, name and description can be taken from first row of the Boest log file),

@name         a name for test for example 'Webi Small Doc'
@description  details of test parameters 'Number Users=0, Increment=25, repeat=2'
@statsfile    name of Boest log file
@tracefile    name of performance monitor trace file
@sourcetype   either 'webi' or 'crystal' depending on type of file being loaded

Analyse Results

The following SQL will display the results for a given test run. It displays the maximum memory averaged over the repeated test runs for a set number of users. It also displays average maximum CPU and average of the avergae CPU.

SELECT
  tr.test_run_id, tr.test_name, tb.test_set_id, tb.number_users,
  avg(cast(tb.max_memory as float)) / (1024*1024) as 'Max Memory',
  avg(tb.max_CPU) as 'Max CPU',
  avg(tb.average_cpu) as 'Avg CPU'
FROM
  test_run tr, test_batch tb
WHERE tb.test_run_id = tr.test_run_id
GROUP BY
  tr.test_run_id, tr.test_name, tb.test_set_id, tb.number_users

Troubleshooting

If a load fails then you can remove data and try. Test_Run_ID is common to all tables and so to cleanup a failed load execute,

DELETE * FROM test_run WHERE test_run_id = X;
DELETE * FROM test_batch WHERE test_run_id = X;
DELETE * FROM test_workflow WHERE test_run_id = X;

Staging tables are always cleared prior to be loaded so we don't need to worry about them. If we need to reload Perf_Trace then you need to delete records based on the name of the trace file loaded.

Back Next