With the continuing relevance of PC-clusters, SMP-clusters and the development of chip architectures into memory hierarchies and internal parallel processing, computer architectures have become more and more complex. Parallelism and the growing capacity of memory and storage has led to an increase of the growth of today's problems.
At present we all miss reliable system benchmarks (other than the Linpack performance used in TOP500), which support the user in rating and obtaining computer systems - especially parallel systems. Existing parallel benchmarks, like the NAS parallel benchmarks, are no longer meaningful these days. Applications, which are not numerical and yet using parallel systems, are still not covered. With the growing relevance of computing grids the situation will even become more aggravated in a few years.
In cooperation with colleagues at
IPACS wants to define a new
basis for benchmarks measuring system performance of distributed systems.
The following topics are the main work packages in IPACS:
Development and propagation of scalable, portable and realistic benchmarks,
Development of methods of measurement for parameters which are specific to
applications and hardware,
Prediction of performance of commercial codes and
Tools that support the user.
The goal here is to develop methods of measurement based on new applications,
low level, and grid benchmarks, which also allow performance prediction.
A further significant element of the new benchmark environment is represented
in the usability of the benchmarks. Mechanisms for the automatic installation
and execution of benchmarks are being strived for.
With the help of grid-adaptive benchmarks it should be possible to check if an existing grid infrastructure is suited for a certain application. Another intent of grid benchmarks is to develop technologies for grid-adaptive applications and to put them at user's disposal. These different approaches should result in a simple, but salable benchmark, which might have the potential to supersede the Linpack benchmark.
These developments should also lead the user in tools for optimizing performance.