Skip to content
Snippets Groups Projects
Select Git revision
  • toolcoverage
  • usertypes
  • fortran
  • main default protected
  • tools
  • must-json
  • merged
  • tools-parallel
  • coll
  • rma
  • dtypes
  • p2p
  • infrastructure-patch-3
  • infrastructure-patch2
  • devel-TJ
  • infrasructure-patch-1
  • devel-ES
  • instructionblock-lists
  • mbi
19 results

mpi-bugbench

  • Clone with SSH
  • Clone with HTTPS
    • Name Last commit Last update
    Data
    img
    scripts
    .dockerignore
    .gitignore
    Dockerfile.itac
    Dockerfile.must
    Dockerfile.parcoach
    MBB.py
    Pipfile
    Pipfile.lock
    README.md
    real_world_data.csv.tar.gz
    README.md

    MPI-BugBench

    MPI-BugBench [MBB24] is an benchmark suite to evaluate the classification quality of MPI correctness tools. It covers various error classes in MPI while incorporating a broad range of real-world MPI usage scenarios.

    Preparation

    For simple reproduction, all workflows are executed within Docker containers.

    Build the docker image and run it

    docker build -f Dockerfile.<parcoach|itac|must> -t mbb:latest .
docker run -v `pwd`:/MBB -it mbb:latest bash

    Generating the codes

    MBB uses a generation infrastructure written in Python to generate all test codes and supports different test generation levels (for details, see our paper). To generate the test cases for a given generation level (1-5) run the following commands:

    python3 MBB.py -c generate --level 2

    or

    cd /MBB
tar -xf real_world_data.csv.tar.gz
cd /MBB/scripts
python3 generate.py --generator_dir errors --level 2 --real_world_data ../output.csv

    Run a tool on the codes

    cd /MBB
python3 MBB.py -c run -x <parcoach|itac|must> -t <timeout in seconds> -l <logs output dir> -n <number of workers>

    Example for PAROACH

    python3 MBB.py -c run -x parcoach -t 180 -l mylogdir -n 4

    Analyzing results

    CSV Results

    python3 MBB.py -x <parcoach|itac|must> -l <logs output dir> -c csv

    Plots

    python3 MBB.py -x <parcoach|itac|must> -l <logs output dir> -c plots

    References

    [MBB24] Tim Jammer, Emmanuelle Saillard, Simon Schwitanski, Joachim Jenke, Radjasouria Vinayagame, Alexander Hück, Christian Bischof. 2024. MPI-BugBench: A Framework for Assessing MPI Correctness Tools]. In EuroMPI'24, September 25-27, 2024, Perth, Australia. Springer.