Compilation Error NaluWind & trilinos

[lylclclyl]

We speculate that it may be that it is likely to be a compilation problem with Trilinos, and it is likely that individual modules are not enabled and the dependent functions are missing when compiling Nalu-Wind. Is the compilation method of 12.12.1 Trilinos and 1.2.0 Nalu-Wind. Here is our running file, ask for help.
nalu-wind.sh(.dat) & trilinos.sh(.dat)
trilinos.txt
nalu-wind.txt

[lylclclyl]

[nalu-wind]
#!/bin/bash -l

Instructions:

Make a directory in the Nalu-Wind directory for building,

Copy this script to that directory and edit the

options below to your own needs and run it.

#rm -rf CMakeCache.txt CTestCustom.cmake Makefile Nalu-WindTargets.cmake Testing CMakeFiles CTestTestfile.cmake make.log reg_tests unit_tests cmake_install.cmake DartConfiguration.tcl include Nalu-WindConfig.cmake src
CXX_COMPILER=mpiicpc
C_COMPILER=mpiicc
FLAGS="-O2 -march=native -mtune=native -std=c++11 -L/public1/soft/boost/boost174-para/lib -L/public1/soft/Trilinos/12.12.1-icc-original/lib -lstk_mesh_base -ltpetra -lboost_program_options -lmuelu -lstokhos_muelu -lmuelu-interface -lmuelu-adapters -lifpack2"
OVERSUBSCRIBE_FLAGS="--use-hwthread-cpus --oversubscribe"

set -e

TRILINOS_ROOT_DIR=/public1/soft/Trilinos/12.12.1-icc-original
YAML_CPP_ROOT_DIR=/public1/soft/yaml-cpp/0.6.2-gcc730

Clean before cmake configure

set +e
rm -rf CMakeFiles
rm -f CMakeCache.txt
set -e

Extra TPLs that can be included in the cmake configure:

-DENABLE_PARAVIEW_CATALYST:BOOL=ON \

-DPARAVIEW_CATALYST_INSTALL_PATH:PATH=${CATALYST_IOSS_ADAPTER_ROOT_DIR} \

-DENABLE_OPENFAST:BOOL=ON \

-DOpenFAST_DIR:PATH=${OPENFAST_ROOT_DIR} \

-DENABLE_HYPRE:BOOL=ON \

-DHYPRE_DIR:PATH=${HYPRE_ROOT_DIR} \

-DENABLE_TIOGA:BOOL=ON \

-DTIOGA_DIR:PATH=${TIOGA_ROOT_DIR} \

(set -x; cmake
-DCMAKE_CXX_COMPILER:STRING=${CXX_COMPILER}
-DCMAKE_CXX_FLAGS:STRING="${FLAGS}"
-DCMAKE_Fortran_COMPILER=mpiifort
-DMPI_Fortran_COMPILER=mpiifort
-DMPI_CXX_COMPILER:STRING=${CXX_COMPILER}
-DMPIEXEC_PREFLAGS:STRING="${OVERSUBSCRIBE_FLAGS}"
-DTrilinos_DIR:PATH=${TRILINOS_ROOT_DIR}
-DYAML_DIR:PATH=${YAML_CPP_ROOT_DIR}
-DCMAKE_BUILD_TYPE:STRING=RELEASE
-DENABLE_DOCUMENTATION:BOOL=OFF
-DHYPRE_INCLUDE_DIRS=/public1/soft/hypre/2.25.0/include
-DHYPRE_LIBRARIES=/public1/soft/hypre/2.25.0/lib/libHYPRE.a
-DENABLE_TESTS:BOOL=ON
..)
#vim build/CMakeFiles/unittestX.dir/flags.make
#vim build/CMakeFiles/naluX.dir/flags.make
#vim build/CMakeFiles/nalu.dir/flags.make

#make VERBOSE=1 2>&1 | tee make.log
#(set -x; nice make -j 16)

[lylclclyl]

[trilinos]
#!/bin/bash

The base directory where mpi is located.

From here you should be able to find include/mpi.h bin/mpicxx, bin/mpiexec, etc.

MPI_ROOT_DIR=/public1/soft/oneAPI/2022.1/mpi/latest
NALU_ROOT_DIR=/PathToNaluProjectDir

Note: Don't forget to set your LD_LIBRARY_PATH to $mpi_base_dir/lib

You may also need to add to LD_LIBRARY_PATH the lib directory for the compiler

used to create the mpi executables.

TPLS needed by trilinos, possibly provided by HomeBrew on a Mac

#BOOST_ROOT_DIR=/usr/local/Cellar/boost/1.56.0/include/boost/
#SUPERLU_ROOT_DIR=/usr/local/Cellar/superlu/4.3
BOOST_ROOT_DIR=/public1/soft/boost/boost174-para
SUPERLU_ROOT_DIR=/public1/soft/SuperLU/5.2.1
NETCDF_ROOT_DIR=/public1/soft/netcdf/4.6.1-parallel-intel17
HDF5_ROOT_DIR=/public1/soft/hdf5/1.10.5-ips17
PARALLEL_NETCDF_ROOT_DIR=/public1/soft/pnetcdf/1.11.1
ZLIB_ROOT_DIR=/public1/soft/zlib/1.2.7-21-lzy
TRILINOS_ROOT_DIR=/public1/soft/Trilinos/12.12.1-icc-original

EXTRA_ARGS=$@

Cleanup old cache before we configure

Note: This does not remove files produced by make. Use "make clean" for this.

find . -name "CMakeFiles" -exec rm -rf {} ;
rm -f CMakeCache.txt

CC=icc cmake ..
-DCMAKE_INSTALL_PREFIX=${TRILINOS_ROOT_DIR}
-DCMAKE_BUILD_TYPE:STRING=RELEASE
-DMPI_USE_COMPILER_WRAPPERS:BOOL=ON
-DMPI_CXX_COMPILER:FILEPATH=mpiicpc -DMPI_C_COMPILER=mpiicc -DMPI_Fortran_COMPILER=mpiifort
-DCMAKE_Fortran_COMPILER=mpiifort -DCMAKE_C_COMPILER=mpiicc -DCMAKE_CXX_COMPILER=mpiicpc
-DKokkos_ENABLE_DEPRECATED_CODE:BOOL=OFF
-DTpetra_INST_SERIAL:BOOL=ON
-DTrilinos_ENABLE_CXX11:BOOL=ON
-DTrilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON
-DTpetra_INST_DOUBLE:BOOL=ON
-DTpetra_INST_COMPLEX_DOUBLE:BOOL=OFF
-DTrilinos_ENABLE_TESTS:BOOL=OFF
-DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=OFF
-DTrilinos_ASSERT_MISSING_PACKAGES:BOOL=OFF
-DTrilinos_ALLOW_NO_PACKAGES:BOOL=OFF
-DTrilinos_ENABLE_Epetra:BOOL=OFF
-DTrilinos_ENABLE_Tpetra:BOOL=ON
-DTrilinos_ENABLE_KokkosKernels:BOOL=ON
-DTrilinos_ENABLE_ML:BOOL=OFF
-DTrilinos_ENABLE_MueLu:BOOL=ON
-DXpetra_ENABLE_Kokkos_Refactor:BOOL=ON
-DMueLu_ENABLE_Kokkos_Refactor:BOOL=ON
-DTrilinos_ENABLE_EpetraExt:BOOL=OFF
-DTrilinos_ENABLE_AztecOO:BOOL=OFF
-DTrilinos_ENABLE_Belos:BOOL=ON
-DTrilinos_ENABLE_Ifpack2:BOOL=ON
-DTrilinos_ENABLE_Amesos2:BOOL=ON
-DTrilinos_ENABLE_Zoltan2:BOOL=ON
-DTrilinos_ENABLE_Ifpack:BOOL=OFF
-DTrilinos_ENABLE_Amesos:BOOL=OFF
-DTrilinos_ENABLE_Zoltan:BOOL=ON
-DTrilinos_ENABLE_STK:BOOL=ON
-DTrilinos_ENABLE_Stokhos:BOOL=ON
-DTrilinos_ENABLE_Teuchos:BOOL=ON
-DTrilinos_ENABLE_Tpetra:BOOL=ON
-DTrilinos_ENABLE_TpetraCore:BOOL=ON
-DTrilinos_ENABLE_ThyraEpetraAdapters:BOOL=OFF
-DTrilinos_ENABLE_ThyraEpetraExtAdapters:BOOL=OFF
-DTrilinos_ENABLE_ThyraTpetraAdapters:BOOL=OFF
-DTrilinos_ENABLE_EpetraExt:BOOL=OFF
-DTrilinos_ENABLE_AztecOO:BOOL=OFF
-DTrilinos_ENABLE_Amesos:BOOL=OFF
-DTrilinos_ENABLE_Ifpack:BOOL=OFF
-DTrilinos_ENABLE_ML:BOOL=OFF
-DTrilinos_ENABLE_Teko:BOOL=OFF
-DTrilinos_ENABLE_Epetra:BOOL=OFF
-DTrilinos_ENABLE_ROL:BOOL=ON
-DTrilinos_ENABLE_TPL:BOOL=ON
-DTrilinos_ENABLE_Gtest:BOOL=ON
-DTrilinos_ENABLE_SEACASExodus:BOOL=ON
-DTrilinos_ENABLE_SEACASEpu:BOOL=ON
-DTrilinos_ENABLE_SEACASExodiff:BOOL=ON
-DTrilinos_ENABLE_SEACASNemspread:BOOL=ON
-DTrilinos_ENABLE_SEACASNemslice:BOOL=ON
-DTrilinos_ENABLE_SEACASIoss:BOOL=ON
-DTPL_ENABLE_MPI:BOOL=ON
-DTPL_ENABLE_Boost:BOOL=ON
-DBoostLib_INCLUDE_DIRS:PATH=${BOOST_ROOT_DIR}/include
-DBoostLib_LIBRARY_DIRS:PATH=${BOOST_ROOT_DIR}/lib
-DBoost_INCLUDE_DIRS:PATH=${BOOST_ROOT_DIR}/include
-DBoost_LIBRARY_DIRS:PATH=${BOOST_ROOT_DIR}/lib
-DTPL_ENABLE_SuperLU:BOOL=ON
-DSuperLU_INCLUDE_DIRS:PATH=${SUPERLU_ROOT_DIR}/include
-DSuperLU_LIBRARY_DIRS:PATH=${SUPERLU_ROOT_DIR}/lib
-DTPL_SuperLU_LIBRARIES=/public1/soft/SuperLU/5.2.1/lib/libsuperlu_5.1.a
-DTPL_ENABLE_Netcdf:BOOL=ON
-DNetCDF_ROOT:PATH=${NETCDF_ROOT_DIR}
-DTPL_Netcdf_PARALLEL:BOOL=ON
-DTPL_ENABLE_Pnetcdf:BOOL=ON
-DPNetCDF_ROOT:PATH=${PARALLEL_NETCDF_ROOT_DIR}
-DPnetcdf_INCLUDE_DIRS:PATH=${PARALLEL_NETCDF_ROOT_DIR}/include
-DPnetcdf_LIBRARY_DIRS:PATH=${PARALLEL_NETCDF_ROOT_DIR}/lib
-DTPL_ENABLE_HDF5:BOOL=ON
-DHDF5_ROOT:PATH=${HDF5_ROOT_DIR}
-DHDF5_NO_SYSTEM_PATHS:BOOL=ON
-DTPL_ENABLE_Zlib:BOOL=ON
-DZlib_INCLUDE_DIRS:PATH=${ZLIB_ROOT_DIR}/include
-DZlib_LIBRARY_DIRS:PATH=${ZLIB_ROOT_DIR}/lib
-DTPL_ENABLE_BLAS:BOOL=ON
-DCMAKE_CXX_FLAGS="-L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib/release/ -L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib -lmpi -ltvmpi -L/public1/soft/hdf5/1.10.5-ips17/lib -lhdf5_hl -L/public1/soft/oneAPI/2022.1/compiler/2022.0.0/linux/compiler/lib/intel64_lin -lsvml -lirc -std=c++11"
-DCMAKE_C_FLAGS="-L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib/release/ -L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib -lmpi -ltvmpi -L/public1/soft/hdf5/1.10.5-ips17/lib -lhdf5_hl -L/public1/soft/oneAPI/2022.1/compiler/2022.0.0/linux/compiler/lib/intel64_lin -lsvml -lirc"
-DCMAKE_Fortran_FLAGS="-L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib/release/ -L/public1/soft/oneAPI/2022.1/mpi/2021.5.0/lib -lmpi -ltvmpi -L/public1/soft/hdf5/1.10.5-ips17/lib -lhdf5_hl -L/public1/soft/oneAPI/2022.1/compiler/2022.0.0/linux/compiler/lib/intel64_lin -lsvml -lirc"

$EXTRA_ARGS

[psakievich]

Is this separate from #1211?

[lylclclyl]

Is this separate from #1211?

Yes! This is a problem that we have installed on the same computer and encountered.

[psakievich]

@lylclclyl we suggest building with spack going forward. We have updated several package recipes and will be supporting builds.