Commit 729b67f6 authored by mal539's avatar mal539
Browse files

Merge branch 'chunksize' of https://gitlab.pop-coe.eu/kernels/bem4i-solver into HEAD

Conflicts:
	run_tests_localmtf_queue.sh

merge commit
parents 2f4a99c0 7c24bb30
This is a bem4i-solver (or just bem4i) repository. It will have pop-db orphan branch.
\ No newline at end of file
This is a bem4i-solver (or just bem4i) repository. It will have pop-db orphan branch.
# bem4i kernel
This kernel reproduces a matrix-vector multiplication within the GMRES solver for a very specific matrix arose in multi-trace formulation f Helmholz equation implemented using boundary element method (BEM). The global system matrix is composed of four matrices that are further made of blocks. Its application requires application for all blocks of each sub-matrix and one exra multiplication over all degress of freedon on the system.
All this is a part of the BEM4I library, that is a library of parallel boundary element based solvers developed at IT4Innovations National Supercomputing Center. It supports solutions of the Laplace, Helmholtz, Lame, and wave equations. The library implements OpenMP and hybrid OpenMP/MPI parallelization. The development is focused on an efficient implementation utilizing multi- and many-core architecture. System matrices assembled within the BEM are generally dense and the library uses Adaptive Cross Approximation technique to approximate them.
The resulted linear system is solved by the appropriate iterative solver based on the quality of the system matrix. For Helmholtz and wave equations, the solver is the GMRES method, for Laplace and Lame it can be the CG method.
## Structure of the repository
* `./dist/` contains the final executables after a build process
* `./build/` contains the built files
* `./nbproject/` contains makefiles for compilation of this project
* `./src/` contains the main kernel source files
* `OMP/`
* `Examples/Helmholtz/testMTF_parallel.h` contains function main - this is actually run
* `./input/` contains input file with an initial mesh
* `./output/` contains results generted during the application run
## How to built the kernel
Create direcotires `dist` and `build`
````bash
mkdir dist
mkdir build
````
Modify file `./remake_mpi.sh` to setup a compiler for C++ and MPI, libraries METIS, BLAS and Eigen.
We have tested this kernel with Intel compiler and Intel MKL. For other compilers, please modify also the makefile `nbproject/Makefile-release_intel.mk`.
Run `./remake_mpi.sh` script to build an executable.
## How to run the kernel
To run the application you can execute the `run_tests_localmtf.sh` script or manually set the appropriate parameters and run the executable from `./dist/release_intel/Intel-Linux/bem4i_classic`. Total problem size is controlled by two refinement parameters `refine4` and `refine9`. For smaller run use smaller values.
......@@ -2,53 +2,13 @@
ml Eigen
ml intel/2018a
#ml intel/2017c
#ml intel
#ml intel
#ml imkl
ml METIS
#ml VTune/2018_update4
#ml TotalView
#ml Score-P/4.1-intel-2018a
#ml Score-P/5.0-intel-2018a
ml Valgrind/3.15.0-intel-2018.04
#ml GCCcore/8.3.0
#ml GCC/8.3.0-2.32
#ml binutils/2.32-GCCcore-8.3.0
#ml ifort/2019.4.227-GCC-8.3.0-2.32
#ml icc/2019.4.227-GCC-8.3.0-2.32
#ml iccifort/2019.4.227-GCC-8.3.0-2.32
#ml intel/2019.03-GCC-8.3.0-2.32
#ml impi/2019.3.199-iccifort-2019.4.227-GCC-8.3.0-2.32
#ml imkl/2019.3.199-iimpi-2019.03-GCC-8.3.0-2.32
#ml impi/2018.4.274-iccifort-2018.5.274-GCC-6.3.0-2.27
#ml Forge/18.2.3
#ml Score-P/5.0-intel-2018a
#ml Scalasca/2.5-intel-2018a
#export SCOREP_EXPERIMENT_DIRECTORY=Score-P/scorep_sum
#export SCOREP_TOTAL_MEMORY=128384000
#export SCOREP_PROFILING_MAX_CALLPATH_DEPTH=87
#export SCOREP_PROFILING_ENABLE_CORE_FILES=1
#cp ${HOME_DIR}dist/release_intel/Intel-Linux/bem4i ${HOME_DIR}dist/release_intel/Intel-Linux/bem4i_comm_test
#test_prefix=COMM_
#executable="${HOME_DIR}dist/release_intel/Intel-Linux/bem4i_comm_test"
test_prefix=OVERLAP_
executable="${HOME_DIR}dist/release_intel/Intel-Linux/bem4i_novel"
#executable="${HOME_DIR}dist/release_intel_scorep/Intel-Linux/bem4i"
test_prefix=BLOCKING_
executable="${HOME_DIR}dist/release_intel/Intel-Linux/bem4i_classic"
# # APPLICATION PARAMETERS
test_index=1
......@@ -94,16 +54,7 @@ export OMP_NUM_THREADS=6
export I_MPI_ASYNC_PROGRESS=0
export I_MPI_ASYNC_PROGRESS_THREADS=0
##export LD_LIBARY_PATH=$LD_LIBRARY_PATH:/home/mal539/POP/bem4i_kernel_reduction/bem4i_MK/Score-P
##echo $LD_LIBRARY_PATH
#export SCOREP_ENABLE_PROFILING=true
#export SCOREP_ENABLE_TRACING=false
#export SCOREP_EXPERIMENT_DIRECTORY=profile
echo mpirun -np 4 ${executable} ${arguments}
#mpirun -np 4 ${executable} ${arguments} > /home/kra568/__BEM4I/mereni/MTF/OVERLAP_1_P12_MPI002_N00002_MS10_BS10000_ACAEPS1e-6_SOLVERT0_PREC1.txt
mpirun -np 4 valgrind --track-origins=yes ./${executable} ${arguments}
mpirun -np 4 ./${executable} ${arguments}
#mpirun -np 2 -env LD_PRELOAD $EBROOTVALGRIND/lib/valgrind/libmpiwrap-amd64-linux.so valgrind ./dist/release_intel/Intel-Linux/bem4i ${arguments}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment