Realtime path tracing (#38)

* Initial implementation of real time path tracing * Added reprojection to path tracer using motion vectors * Fixed some MacOS issues * Added sky probe generation to path tracer * Bug fixes and path tracing denoising improvements * Further improvements to temporal denoiser * Begin work on bindless resources * More descriptor changes * More work on new descriptor management, also some bug fixes * User defined global descriptor injection works now * Bindless textures are now working * Fixes and cleanups * Some more changes to get everything working on Mac (textures are missing) * Work on fully bindless ray tracing continues * Fixed the rest of the issues * Mac works now * Fixed rest of MoltenVK/Metal isses * Ray tracing should now handle new meshes without a rebuild * Fixed a few things to make software pathtracing work in Voxel app * Some smaller improvements * Experimenting with better TLAS * Fixed a few things * Try fix build pipeline * Fixed a few more things * Try to fix build pipeline * Fix more stuff * More build pipeline changes * More build pipeline fixes * Should reduce pipeline build times * Fix MacOS binding issue * Hardware raytracing + bindless works now * Few more fixes * Update build.yml * Update build.yml * Update build.yml * Update build.yml * Make engine be able to work headlessly * Attempt to fix build pipeline * Try to get more info related to pipeline failure * More build pipeline related changes * A few more fixes * Added test project * Fix CMake issue * Should be cleaned up a bit more * Update build.yml * Update build.yml * Update build.yml * More changes * Fix build * Try again * Try again * Try to enable validation layers * Update build.yml * Update build.yml * Find out more * Log more stuff * More attempts * More fixes * More fixes * Update build.yml * Update build.yml * Update build.yml * Test directly setting env in main * Last try * Disabling validation layer if required ones are not found * Improved automated pipeline tests * Added SpirV caching on ShaderCompiler level * Should fix some issues * Fixed a bunch of things * Revert some previous BVH changes * Some small BVH changes * Some more fixes * Smaller changes * Fixed window not closing issue * Should have fixed a bunch of validation errors * More code style changes * More fixes * Build fixes * Extended tests * Added a new scene and new tests * Some SSGI tests * Did more changes to SSGI * Fixed a few things * Fixed a few things * Small fix for ocean * Fixed some issues * More tests, small fixes * Fix release pipeline * Fix one SSGI issue * Fixed a validation issue regarding shader rollback * Update thirdparty file * Some small improvements
tippesi · Jan 7, 2024 · 9893770 · 9893770
1 parent fe5b2b6
commit 9893770
Show file tree

Hide file tree

Showing 171 changed files with 5,853 additions and 1,875 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -13,6 +13,7 @@ env:
   BUILD_TYPE_DEBUG: Debug
   BUILD_TYPE_RELEASE: Release
   USE_CACHE: true
+  RUN_TESTS: true
 
 jobs:
   windows-build:
@@ -26,22 +27,35 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
+
+      - name: Restore swiftshader repository cache
+        uses: actions/cache/restore@v3
+        id: swiftshadercache-restore
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Windows
+          key: ${{ runner.os }}
+
+      - name: Checkout swiftshader repository
+        uses: actions/checkout@v4
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          repository: google/swiftshader
+          path: swiftshader
 
       - name: Setup Vcpkg
         uses: friendlyanon/setup-vcpkg@v1
         # Committish: The commit sha of the vcpkg repo, same as in vcpkg.json
         with: 
-          committish: 8568364fbeb414db3fa5b5bfc436e57b4368c55f
+          committish: 6c937c32233bdf295ab2140dbce97fd00084a5f3
           cache: ${{ env.USE_CACHE }}
 
       # This doesn't work when the Visual Studio C++ CLI was set up first (maybe needs a setup with 2019 version)
-      - name: Prepare Vulkan SDK
-        uses: humbletim/setup-vulkan-sdk@v1.2.0
+      - name: Install Vulkan SDK
+        uses: humbletim/install-vulkan-sdk@v1.1.1
         with:
-          vulkan-query-version: 1.3.204.0
-          vulkan-components: Vulkan-Headers, Vulkan-Loader
-          vulkan-use-cache: false
+          version: 1.3.261.1
+          cache: true
 
       - name: Setup Microsoft Visual C++ CLI
         uses: ilammy/msvc-dev-cmd@v1
@@ -61,6 +75,25 @@ jobs:
           ${{ github.workspace }}/vcpkg/vcpkg install --clean-after-build --triplet=x64-windows
           Remove-Item –path vcpkg_installed –recurse
 
+      - name: Build swiftshader
+        uses: ashutoshvarma/action-cmake-build@master
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          build-dir: ${{ github.workspace }}/swiftshader/build
+          source-dir: ${{ github.workspace }}/swiftshader
+          cc: "cl"
+          cxx: "cl"
+          configure-options:  -G Ninja -DSWIFTSHADER_BUILD_TESTS=OFF -DSWIFTSHADER_ENABLE_ASTC=OFF
+          parallel: 16
+          build-type: MinSizeRel
+
+      - name: Save swiftshader respository cache
+        id: swiftshadercache-save
+        uses: actions/cache/save@v3
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Windows
+          key: ${{ steps.swiftshadercache-restore.outputs.cache-primary-key }}    
+
       - name: Build ${{ matrix.build-type }} configuration with CMake
         uses: ashutoshvarma/action-cmake-build@master
         with:
@@ -69,10 +102,39 @@ jobs:
           cc: "cl"
           cxx: "cl"
           configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
-            -DATLAS_DEMO=ON -G Ninja
+            -DATLAS_DEMO=ON -DATLAS_TESTS=OFF -G Ninja
           parallel: 16
           build-type: ${{ matrix.build-type }}
 
+      - name: Build ${{ matrix.build-type }} test configuration with CMake
+        uses: ashutoshvarma/action-cmake-build@master
+        if: ${{ env.RUN_TESTS }} == true
+        with:
+          build-dir: ${{ github.workspace }}/tests/${{ matrix.build-type }}
+          source-dir: ${{ github.workspace }}
+          cc: "cl"
+          cxx: "cl"
+          configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
+            -DATLAS_TESTS=ON -DATLAS_BINDLESS=OFF -DATLAS_HEADLESS=ON -G Ninja
+          parallel: 16
+          build-type: ${{ matrix.build-type }}
+
+      - name: Run tests
+        shell: pwsh
+        if: ${{ env.RUN_TESTS }} == true
+        # Add additional scripting steps here
+        run: |
+          Copy-Item -Path "${{ github.workspace }}\swiftshader\build\Windows\*.*" -Destination "${{ github.workspace }}/bin/tests/${{ matrix.build-type }}"
+          Copy-Item -Path "${{ github.workspace }}\VULKAN_SDK\Bin\VkLayer_khronos_validation.*" -Destination "${{ github.workspace }}/bin/tests/${{ matrix.build-type }}"
+          cd ${{ github.workspace }}/bin/tests/${{ matrix.build-type }}
+          set VK_LOADER_LAYERS_ENABLE=*validation
+          set VK_ADD_LAYER_PATH=${{ github.workspace }}\VULKAN_SDK\Bin\
+          .\AtlasEngineTests.exe
+        env: 
+          VK_ICD_FILENAMES: ${{ github.workspace }}/bin/${{ matrix.build-type }}/vk_swiftshader_icd.json
+          VK_ADD_LAYER_PATH: ${{ github.workspace }}\VULKAN_SDK\Bin\;${{ github.workspace }}/bin/tests/${{ matrix.build-type }}
+          VK_LOADER_LAYERS_ENABLE: '*validation'
+
       - name: Upload artifact
         if: ${{ matrix.build-type == 'Release' }}
         uses: actions/upload-artifact@v2
@@ -85,6 +147,7 @@ jobs:
             THIRDPARTY.md
             ${{ github.workspace }}/bin/**/AtlasEngineDemo.exe
             ${{ github.workspace }}/bin/**/*.dll
+            !${{ github.workspace }}/bin/tests
             !**/CMakeFiles
 
   linux-build:
@@ -98,13 +161,27 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
+
+      - name: Restore swiftshader repository cache
+        uses: actions/cache/restore@v3
+        id: swiftshadercache-restore
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Linux
+          key: ${{ runner.os }}
+
+      - name: Checkout swiftshader repository
+        uses: actions/checkout@v4
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          repository: google/swiftshader
+          path: swiftshader
 
       - name: Setup Vcpkg
         uses: friendlyanon/setup-vcpkg@v1
         # Committish: The commit sha of the vcpkg repo, same as in vcpkg.json
         with: 
-          committish: 8568364fbeb414db3fa5b5bfc436e57b4368c55f
+          committish: 6c937c32233bdf295ab2140dbce97fd00084a5f3
           cache: ${{ env.USE_CACHE }}
 
       - name: Setup Ninja
@@ -113,13 +190,12 @@ jobs:
           # ninja version to download. Default: 1.10.0
           version: 1.10.0
 
-      - name: Prepare Vulkan SDK
-        uses: humbletim/setup-vulkan-sdk@v1.2.0
+      - name: Install Vulkan SDK
+        uses: humbletim/install-vulkan-sdk@v1.1.1
         with:
-          vulkan-query-version: 1.3.204.0
-          vulkan-components: Vulkan-Headers, Vulkan-Loader
-          vulkan-use-cache: false
-
+          version: 1.3.261.1
+          cache: true
+
       - name: Run scripts
         shell: bash
         # Add additional scripting steps here
@@ -130,6 +206,25 @@ jobs:
           ${{ github.workspace }}/vcpkg/vcpkg install --clean-after-build --triplet=x64-linux
           rm -r vcpkg_installed
 
+      - name: Build swiftshader
+        uses: ashutoshvarma/action-cmake-build@master
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          build-dir: ${{ github.workspace }}/swiftshader/build
+          source-dir: ${{ github.workspace }}/swiftshader
+          cc: "gcc"
+          cxx: "g++"
+          configure-options:  -G Ninja -DSWIFTSHADER_BUILD_TESTS=OFF -DSWIFTSHADER_ENABLE_ASTC=OFF
+          parallel: 16
+          build-type: MinSizeRel
+
+      - name: Save swiftshader respository cache
+        id: swiftshadercache-save
+        uses: actions/cache/save@v3
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Linux
+          key: ${{ steps.swiftshadercache-restore.outputs.cache-primary-key }}    
+
       # https://github.com/marketplace/actions/setup-ninja
       - name: Build ${{ matrix.build-type }} configuration with CMake
         uses: ashutoshvarma/action-cmake-build@master
@@ -139,10 +234,36 @@ jobs:
           cc: "gcc"
           cxx: "g++"
           configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
-            -DATLAS_DEMO=ON -DATLAS_DEMO=ON -G Ninja
+            -DATLAS_DEMO=ON -DATLAS_TESTS=OFF -G Ninja
+          parallel: 16
+          build-type: ${{ matrix.build-type }}
+
+      - name: Build ${{ matrix.build-type }} test configuration with CMake
+        uses: ashutoshvarma/action-cmake-build@master
+        if: ${{ env.RUN_TESTS }} == true
+        with:
+          build-dir: ${{ github.workspace }}/tests/${{ matrix.build-type }}
+          source-dir: ${{ github.workspace }}
+          cc: "gcc"
+          cxx: "g++"
+          configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
+            -DATLAS_TESTS=ON -DATLAS_BINDLESS=OFF -DATLAS_HEADLESS=ON -G Ninja
           parallel: 16
           build-type: ${{ matrix.build-type }}
 
+      - name: Run tests
+        shell: bash
+        if: ${{ env.RUN_TESTS }} == true
+        # Add additional scripting steps here
+        run: |
+          cd ${{ github.workspace }}/bin/tests/${{ matrix.build-type }}
+          ./AtlasEngineTests
+        env: 
+          VK_ICD_FILENAMES: ${{ github.workspace }}/swiftshader/build/Linux/vk_swiftshader_icd.json
+          VK_ADD_LAYER_PATH: ${{ github.workspace }}/VULKAN_SDK/lib/vulkan/layers/:${{ github.workspace }}/VULKAN_SDK/etc/vulkan/explicit_layer.d/
+          LD_LIBRARY_PATH: ${{ github.workspace }}/VULKAN_SDK/lib/
+          VK_LOADER_LAYERS_ENABLE: '*validation'
+
       - name: Upload artifact
         if: ${{ matrix.build-type == 'Release' }}
         uses: actions/upload-artifact@v2
@@ -155,6 +276,7 @@ jobs:
             THIRDPARTY.md
             ${{ github.workspace }}/bin/**/AtlasEngineDemo
             ${{ github.workspace }}/bin/**/*.so*
+            !${{ github.workspace }}/bin/tests
             !**/CMakeFiles
 
   macos-build:
@@ -170,11 +292,25 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@v2
 
+      - name: Restore swiftshader repository cache
+        uses: actions/cache/restore@v3
+        id: swiftshadercache-restore
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Darwin
+          key: ${{ runner.os }}
+
+      - name: Checkout swiftshader repository
+        uses: actions/checkout@v4
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          repository: google/swiftshader
+          path: swiftshader
+
       - name: Setup Vcpkg
         uses: friendlyanon/setup-vcpkg@v1
         # Committish: The commit sha of the vcpkg repo, same as in vcpkg.json
         with: 
-          committish: 8568364fbeb414db3fa5b5bfc436e57b4368c55f
+          committish: 6c937c32233bdf295ab2140dbce97fd00084a5f3
           cache: ${{ env.USE_CACHE }}
 
       - name: Setup Ninja
@@ -183,12 +319,11 @@ jobs:
           # ninja version to download. Default: 1.10.0
           version: 1.10.0
 
-      - name: Prepare Vulkan SDK
-        uses: humbletim/setup-vulkan-sdk@v1.2.0
+      - name: Install Vulkan SDK
+        uses: humbletim/install-vulkan-sdk@v1.1.1
         with:
-          vulkan-query-version: 1.3.204.0
-          vulkan-components: Vulkan-Headers, Vulkan-Loader
-          vulkan-use-cache: false
+          version: 1.3.261.1
+          cache: true
 
       - name: Run scripts
         shell: bash
@@ -199,6 +334,25 @@ jobs:
           ${{ github.workspace }}/vcpkg/vcpkg install --clean-after-build --triplet=x64-osx
           rm -r vcpkg_installed
 
+      - name: Build swiftshader
+        uses: ashutoshvarma/action-cmake-build@master
+        if: steps.swiftshadercache-restore.outputs.cache-hit != 'true'
+        with:
+          build-dir: ${{ github.workspace }}/swiftshader/build
+          source-dir: ${{ github.workspace }}/swiftshader
+          cc: "clang"
+          cxx: "clang++"
+          configure-options:  -G Ninja -DSWIFTSHADER_BUILD_TESTS=OFF -DSWIFTSHADER_ENABLE_ASTC=OFF
+          parallel: 16
+          build-type: MinSizeRel
+
+      - name: Save swiftshader respository cache
+        id: swiftshadercache-save
+        uses: actions/cache/save@v3
+        with:
+          path: ${{ github.workspace }}/swiftshader/build/Darwin
+          key: ${{ steps.swiftshadercache-restore.outputs.cache-primary-key }}
+
       # https://github.com/marketplace/actions/setup-ninja
       - name: Build ${{ matrix.build-type }} configuration with CMake
         uses: ashutoshvarma/action-cmake-build@master
@@ -208,10 +362,36 @@ jobs:
           cc: "clang"
           cxx: "clang++"
           configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
-            -DATLAS_DEMO=ON -G Ninja
+            -DATLAS_DEMO=ON -DATLAS_TESTS=OFF -G Ninja
           parallel: 16
           build-type: ${{ matrix.build-type }}
 
+      - name: Build ${{ matrix.build-type }} test configuration with CMake
+        uses: ashutoshvarma/action-cmake-build@master
+        if: ${{ env.RUN_TESTS }} == true
+        with:
+          build-dir: ${{ github.workspace }}/tests/${{ matrix.build-type }}
+          source-dir: ${{ github.workspace }}
+          cc: "clang"
+          cxx: "clang++"
+          configure-options: -DCMAKE_TOOLCHAIN_FILE='${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake'
+            -DATLAS_TESTS=ON -DATLAS_BINDLESS=OFF -DATLAS_HEADLESS=ON -G Ninja
+          parallel: 16
+          build-type: ${{ matrix.build-type }}
+
+      - name: Run tests
+        shell: bash
+        if: ${{ env.RUN_TESTS }} == true
+        # Add additional scripting steps here
+        run: |
+          cd ${{ github.workspace }}/bin/tests/${{ matrix.build-type }}
+          ./AtlasEngineTests
+        env: 
+          VK_ICD_FILENAMES: ${{ github.workspace }}/swiftshader/build/Darwin/vk_swiftshader_icd.json   
+          VK_ADD_LAYER_PATH: ${{ github.workspace }}/VULKAN_SDK/share/vulkan/explicit_layer.d/
+          DYLD_LIBRARY_PATH: ${{ github.workspace }}/VULKAN_SDK/lib/
+          VK_LOADER_LAYERS_ENABLE: '*validation'
+
       - name: Upload artifact
         if: ${{ matrix.build-type == 'Release' }}
         uses: actions/upload-artifact@v2
@@ -224,4 +404,5 @@ jobs:
             THIRDPARTY.md
             ${{ github.workspace }}/bin/**/AtlasEngineDemo
             ${{ github.workspace }}/bin/**/*.dylib*
+            !${{ github.workspace }}/bin/tests
             !**/CMakeFiles