diff --git a/README.md b/README.md index 35a08fffa7318c842ea543731cb7b4a2a84df2f8..ac24a777f40d69e2158d4e087980b20454c71166 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,24 @@ -## Build the docker image and run it +# MPI-BugBench +MPI-BugBench [[MBB24](#References)] is an benchmark suite to evaluate the classification quality of MPI correctness tools. It covers various error classes in MPI while incorporating a broad range of real-world MPI usage scenarios. + + +## Preparation +For simple reproduction, all workflows are executed within Docker containers. + +### Build the docker image and run it ``` docker build -f Dockerfile.<parcoach|itac|must> -t mbb:latest . docker run -v `pwd`:/MBB -it mbb:latest bash ``` -## Generate the codes - +## Generating the codes +MBB uses a generation infrastructure written in Python to generate all test codes and supports different test generation levels (for details, see our [paper](#References)). +To generate the test cases for a given generation level (1-5) run the following commands: ``` python3 MBB.py -c generate --level 2 ``` -Or +or ``` cd /MBB tar -xf real_world_data.csv.tar.gz @@ -31,15 +39,18 @@ python3 MBB.py -c run -x <parcoach|itac|must> -t <timeout in seconds> -l <logs o python3 MBB.py -c run -x parcoach -t 180 -l mylogdir -n 4 ``` -## To get stats on a tool +## Analyzing results +### CSV Results ``` -cd /MBB python3 MBB.py -x <parcoach|itac|must> -l <logs output dir> -c csv ``` -## Troubleshooting for ITAC +### Plots +``` +python3 MBB.py -x <parcoach|itac|must> -l <logs output dir> -c plots +``` + -If you see an error like "Waiting for cache lock: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 3509 (apt)" -Try: apt clean -And retry the command to run the tool: python3 MBI.py -x itac -c run +## References +[MBB24] Tim Jammer, Emmanuelle Saillard, Simon Schwitanski, Joachim Jenke, Radjasouria Vinayagame, Alexander Hück, Christian Bischof. 2024. MPI-BugBench: A Framework for Assessing MPI Correctness Tools]. In EuroMPI'24, September 25-27, 2024, Perth, Australia. Springer. \ No newline at end of file