MySQL Reference Manual - 4.0.20

[ < ]

[ > ]

[ << ]

[ Up ]

[ >> ]

[Top]

[Contents]

[Index]

[ ? ]

7.1.4 The MySQL Benchmark Suite

This section should contain a technical description of the MySQL benchmark suite (and crash-me), but that description has not yet been written. Currently, you can get a good idea of the benchmarks by looking at the code and results in the `sql-bench' directory in any MySQL source distribution.

This benchmark suite is meant to tell any user what operations a given SQL implementation performs well or poorly.

Note that this benchmark is single-threaded, so it measures the minimum time for the operations performed. We plan to add multi-threaded tests to the benchmark suite in the future.

The following tables show some comparative benchmark results for several database servers when accessed through ODBC on a Windows NT 4.0 machine.

Reading 2000000 rows by index	Seconds	Seconds
mysql	367	249
mysql_odbc	464
db2_odbc	1206
informix_odbc	121126
ms-sql_odbc	1634
oracle_odbc	20800
solid_odbc	877
sybase_odbc	17614

Inserting 350768 rows	Seconds	Seconds
mysql	381	206
mysql_odbc	619
db2_odbc	3460
informix_odbc	2692
ms-sql_odbc	4012
oracle_odbc	11291
solid_odbc	1801
sybase_odbc	4802

For the preceding tests, MySQL was run with an index cache size of 8MB.

Note that Oracle is not included because it asked to be removed. All Oracle benchmarks have to be passed by Oracle! We believe that makes Oracle benchmarks very biased because the above benchmarks are supposed to show what a standard installation can do for a single client.

To use the benchmark suite, the following requirements must be satisfied:

The benchmark suite is provided with MySQL source distributions. You can either download a released distribution from http://dev.mysql.com/downloads/, or use the current development source tree (see section Installing from the Development Source Tree).
The benchmark scripts are written in Perl and use the Perl DBI module to access database servers, so DBI must be installed. You will also need the server-specific DBD drivers for each of the servers you want to test. For example, to test MySQL, PostgreSQL, and DB2, you must have the DBD::mysql, DBD::Pg, and DBD::DB2 modules installed. See section Perl Installation Notes.

After you obtain a MySQL source distribution, you will find the benchmark suite located in its `sql-bench' directory. To run the benchmark tests, build MySQL, then change location into the `sql-bench' directory and execute the run-all-tests script:

shell> cd sql-bench
shell> perl run-all-tests --server=server_name

server_name is one of supported servers. To get a list of all options and supported servers, invoke this command:

shell> perl run-all-tests --help

The crash-me script also is located in the `sql-bench' directory. crash-me tries to determine what features a database supports and what its capabilities and limitations are by actually running queries. For example, it determines:

What column types are supported
How many indexes are supported
What functions are supported
How big a query can be
How big a VARCHAR column can be

You can find the results from crash-me for many different database servers at http://dev.mysql.com/tech-resources/crash-me.php. For more information about benchmark results, visit http://dev.mysql.com/tech-resources/benchmarks/.