DRAFT PR: inflow-outflow boundary condition #1066

mukul1992 · 2024-05-18T01:33:41Z

Summary

Enabling the mass_inflow_outflow (MIO) BC which allows for both inflow and outflow cells in a boundary.

Pull request type

Please check the type of change introduced:

Checklist

The following is included:

new unit-test(s)
new regression test(s)
documentation for new capability

This PR was tested by running:

the unit tests
- on GPU
- on CPU
the regression tests
- on GPU
- on CPU (macOS M3, gcc-13)

Additional background

This functionality would be required for coupling with ERF and was started after discussions with @gdeskos and @mchurchf, along with @asalmgren, as it was a desired functionality within AMR-Wind too. Additionally, @hgopalan has expressed interest, and @marchdf has helped along the way.

Description

The changes required can be organized into six categories:

adding MIO as one of the BCs, mapping it to a custom math BC, and adding the mechanics required for enabling UDFs.
the new TwoLayer UDF for testing, currently it implements a simple two-layer x-direction flow in opposite directions as a test, but this UDF can be modified to test progressively more complex cases, finishing with a regression test case such as vortex initialization and advection across the boundary.
the custom Neumann function for filling the outflow boundary cells in MassInflowOutflowBC.cpp
adding the inflow and outflow conditions in the WENO upwinding functions to map to ext_dir and foextrap behavior respectively.
calling the enforceSolvability routine in AMReX-Hydro before the mac-projection, which adjusts the outflux to match with influx (will link to the corresponding PR in Hydro).
the cmake scripts and input files for testing, these can be reorganized and cleaned up ultimately into a proper regression test as development completes. The input file being used for testing is input_inout.

marchdf

Made a first quick pass.

amr-wind/boundary_conditions/MassInflowOutflowBC.cpp

marchdf · 2024-05-22T14:07:08Z

amr-wind/boundary_conditions/MassInflowOutflowBC.cpp

+    const int nlevels = m_field.repo().num_active_levels();
+    const bool ib = (idim == 0), jb = (idim == 1), kb = (idim == 2);
+
+    amrex::Print() << "***** Applying MIO custom Neumann fills at orientation: " << idx << std::endl;


remove debug statement

yes, I am keeping them until all the bugs are ironed out an will clean things up when it's ready to merge

marchdf · 2024-05-22T14:09:40Z

amr-wind/boundary_conditions/MassInflowOutflowBC.cpp

+    const auto islow = m_ori.isLow();
+    const auto ishigh = m_ori.isHigh();
+    const int nlevels = m_field.repo().num_active_levels();
+    const bool ib = (idim == 0), jb = (idim == 1), kb = (idim == 2);


these bools are implicetly cast to ints later. I would prefer something like const amrex::IntVect ivp = {blah} and then you can just iv + ivp in the parallelfor loops

amr-wind/boundary_conditions/BCInterface.H

amr-wind/boundary_conditions/scalar_bcs.H

marchdf · 2024-05-22T14:14:31Z

amr-wind/convection/incflo_godunov_ppm.H

-        if (bclo == BCType::ext_dir) {
+        if (bclo == BCType::ext_dir ||
+            (bclo == BCType::user_1 && velp >= 0.0)) {
+            //Print() << "trans_xbc_domlo_extdir triggered at " << i << " " << j << " " << k << " " << n


when we get close to merging, let's remove all these print statements.

marchdf · 2024-05-22T14:16:38Z

cmake/amr-wind-utils.cmake

@@ -79,6 +79,7 @@ macro(init_amrex_hydro)
  else()
    set(CMAKE_PREFIX_PATH ${AMReX-Hydro_DIR} ${CMAKE_PREFIX_PATH})
    find_package(AMReX-Hydro CONFIG REQUIRED)
+    add_subdirectory(${AMReX-Hydro_DIR} build)


Is this needed? Feels strange

I needed it to compile with an external AMReX-Hydro, but it may just be my inexperience with CMake. I will look into it..

marchdf · 2024-05-22T14:17:23Z

amr-wind/physics/udfs/TwoLayer.H

+{
+    struct DeviceOp
+    {
+        // clang-format off


remove this block

marchdf · 2024-05-22T14:18:12Z

amr-wind/physics/udfs/TwoLayer.cpp

+
+namespace amr_wind::udf {
+
+TwoLayer::TwoLayer(const Field& /*fld*/)


clean up this cpp and H files from comments and clang format off stuff

marchdf · 2024-05-22T14:18:36Z

amr-wind/projection/incflo_apply_nodal_projection.cpp

@@ -6,6 +6,9 @@
 #include "amr-wind/core/field_ops.H"
 #include "amr-wind/wind_energy/ABL.H"

+// for debugging


remove these later

marchdf · 2024-05-22T14:26:45Z

bctest/input_inflow

@@ -0,0 +1,71 @@
+# This is a 2D poiseuille flow when run to steady state


move these into test/test_files and then add to test/CMakeLists.txt

yes, makes sense

marchdf · 2024-05-22T14:28:26Z

Most of my comments right now are just noting the obvious things to clean up before actually merging. Thank you for taking this task on!!

mbkuhn · 2024-05-22T15:12:16Z

Speaking of merging, it looks like there are some complicated merge conflicts between this branch and main. It may be helpful to rebase your branch to upstream/main to remove some of those conflicts and get the unrelated parts of this branch up-to-date. Removing the merge conflicts will also allow the CI tests to run.

mukul1992 · 2024-05-22T18:20:52Z

@marchdf Yes, I will certainly clean up all the debug statements and incorporate other changes you recommended.

@mbkuhn Yes, I will do the rebase over the next week along with cleaning things up.

Thanks for the early feedback!

mukul1992 · 2024-06-07T17:28:33Z

PS. I have been able to rebase this code with the latest changes in the Exawind/amr-wind repo. I have also updated the amrex submod in the index (let me know if I did it incorrectly). And have updated the BCType from user_1 to direction_dependent in the code.
And finally, here's the draft PR for AMReX-Hydro that includes the outflow-correction for enforcing solvability.

…ls in the same boundary

…jection

…nodal projection

… solvability

marchdf requested review from marchdf, mbkuhn and moprak-nrel May 20, 2024 14:23

marchdf marked this pull request as draft May 20, 2024 14:24

marchdf requested changes May 22, 2024

View reviewed changes

marchdf reviewed May 22, 2024

View reviewed changes

mukul1992 force-pushed the inoutbc branch from 0fdf77a to 76a9304 Compare May 24, 2024 00:14

mukul1992 force-pushed the inoutbc branch from c0c94a4 to afff3fe Compare June 7, 2024 17:09

mukul1992 added 18 commits June 11, 2024 11:42

Initial setup for the inflow-outflow BC that manages both kind of cel…

33c9764

…ls in the same boundary

add comment for header ifdef

0c6e7b6

set up advection and projection BCs

bac24da

temporary fix to use an external hydro repo

8a2e56f

initial changes to add the outflow correction

0d84155

additional changes to enable mass in-out BC, along with udf capabilities

eb27e35

Mukul's test files for the in-out BC

53c5cd2

ignoring all plt and chk files

c6d7e42

set up garbage values at outflow for testing

7d465d0

intermediate changes for custom fill at MIO boundaries

458b4fc

added options to toggle initial iterations for testing

15e0840

further changes to implement and test the MIO Neumann fill

47c5184

adding print statements for testing

7e7542e

inplementing custom Neumann fill-patch correctly before the nodal pro…

15bbc60

…jection

intermediate changes in the BC routines for the in-out BC

024c685

debug build cmake config

26220ac

input file updates

75b4173

playing around with if-else on zero velocity values at the bndry

d83a510

mukul1992 added 24 commits June 11, 2024 11:42

more print messages for debugging and changes in Godunov_trans_xbc

0405e4f

enabling Dirichlet operations for the in-out BC

912d056

not doing the custom Neumann fills to corner cells

9dad637

input fileupdates

fe388c6

adding back the Neumann fills for corner cells

42ce07e

clean up, adding/removing print statements

3fa1812

clean up gitignore

11bea22

define new UDF TwoLayer and restore CustomVelocity for PR

be0ce16

clean up print messages etc for draft PR

5f32a81

printing information for debugging

b6e25d2

perturbing initial condition for testing and debugging

9e3c10c

resolved corner cell issue by adding a periodic boundary fill before …

0715825

…nodal projection

change std::array to amrex::Array

9cfa4d8

debugging for mac-velocity fill issue at i=0

95d02bb

pulled latest code from amrex to include the new BCs

bbfee7c

replaced user_1 with new direction_dependent BCType in amrex

9ed93e9

commenting out debug statements for umac boundary fill checking

eb72722

reactivating some plotfile printing for debugging/digging

ac87dd5

modified function calls to reflect name changes in hydro

fa9e1ed

Added function to call the hydro routine for enforcing inflow-outflow…

5dfe015

… solvability

simpler one-way flow test

16476d5

accepting vector velocities for TwoLayer udf

0e5af61

add directional conditions in y and z-directions

b028112

change input files to have flow along both x and y

58dcb35

mukul1992 force-pushed the inoutbc branch from afff3fe to 58dcb35 Compare July 3, 2024 18:24

mukul1992 added 4 commits July 8, 2024 11:14

renamed input file to match with naming of other test files

0c251c2

simplify x-y planar flow cases for debugging

0008f00

printing time from UDF fordebugging

71a5c30

simplifying mass_inflow case for asymmetry test

be9649a

mukul1992 mentioned this pull request Jul 10, 2024

Potential unidentified bug causes an asymmetry in the solution field #1126

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DRAFT PR: inflow-outflow boundary condition #1066

DRAFT PR: inflow-outflow boundary condition #1066

mukul1992 commented May 18, 2024 •

edited

Loading

marchdf left a comment

marchdf May 22, 2024

mukul1992 Jun 7, 2024

marchdf May 22, 2024

mukul1992 Jun 7, 2024

marchdf May 22, 2024

marchdf May 22, 2024

mukul1992 Jun 7, 2024

marchdf May 22, 2024

marchdf May 22, 2024

marchdf May 22, 2024

marchdf May 22, 2024

mukul1992 Jun 7, 2024

marchdf commented May 22, 2024

mbkuhn commented May 22, 2024

mukul1992 commented May 22, 2024

mukul1992 commented Jun 7, 2024


		namespace amr_wind::udf {

		TwoLayer::TwoLayer(const Field& /fld/)

		@@ -0,0 +1,71 @@
		# This is a 2D poiseuille flow when run to steady state

DRAFT PR: inflow-outflow boundary condition #1066

Are you sure you want to change the base?

DRAFT PR: inflow-outflow boundary condition #1066

Conversation

mukul1992 commented May 18, 2024 • edited Loading

Summary

Pull request type

Checklist

Additional background

Description

marchdf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marchdf commented May 22, 2024

mbkuhn commented May 22, 2024

mukul1992 commented May 22, 2024

mukul1992 commented Jun 7, 2024

mukul1992 commented May 18, 2024 •

edited

Loading