slang.git/tools/slang-test/slang-test-main.cpp, branch master

Retry file reads in slang-test to handle intermittent I/O errors (#8713)

2025-10-15T00:07:03+00:00

Related #8705

Support tests outside tests directory (#7791) (#8666)

2025-10-13T10:50:32+00:00

As running slang-test in slang root is implicitly assumed (and mentioned in CONTRIBUTING.md), no detailed path checks are done.

Add deterministic shuffling of tests in directory (#8622)

2025-10-08T17:13:47+00:00

Fixes #8621 Add command line options for enable shuffling as well as providing a custom seed. Use Mersenne-Twister engine for a deterministic shuffle.

Adding slang-test option to ignore abort popup message (#8492)

2025-09-24T06:46:31+00:00

With the recent Windows runtime libraries, a new popup window started appearing when `abort()` is called. This was observed when VVL prints a message as a part of WGPU test. Although it can be helpful when we want to debug it, it breaks the behavior of CI scripts when the tests are expected to continue even when they fail. When the test fail, CI script stops in the middle and wait for a user to click on a button on the dialog window, which cannot happen. As a result, when there is a VVL error message, CI run stops in the middle and the testing stops prematurely. This commit adds a new command-line argument, `-ignore-abort-msg`, that ignores the abort message and it wouldn't show the dialog popup window. From the implementation perspective, there are three places that are related. - slang-test itself should turn off the flag. - render-test should turn off the flag after getting the argument from slang-test - test-server should turn off the flag after getting the argument from slang-test When test-server runs render-test, the arguments are already handled by slang-test, so test-server needs to just pass through the arguments.

Add RHI Device Caching and Test Prefix Exclusion (#8448)

2025-09-22T22:46:42+00:00

# Add RHI Device Caching and Test Prefix Exclusion ## Summary This PR introduces two key improvements to the Slang test infrastructure: 1. **RHI Device Caching**: Implements device caching to significantly speed up test execution by reusing graphics devices across tests, **RHI Device Caching reduces slang-test execution time from ~15 minutes to ~5 minutes in Windows release builds** 2. **Test Prefix Exclusion**: Adds `-exclude-prefix` option to skip tests matching specified path prefixes ## Changes ### RHI Device Caching - **New `DeviceCache` class** (`slang-test-device-cache.h/cpp`): Thread-safe device cache with LRU eviction (max 10 devices) - **Cache control option**: `-cache-rhi-device` flag in both `slang-test` and `render-test` - Default: **enabled** in slang-test, **disabled** in render-test when run standalone - Automatically skips caching for CUDA devices (due to driver issues) - **Performance benefit**: Eliminates expensive device creation/destruction cycles, especially beneficial for Vulkan on Tegra platforms ### Test Prefix Exclusion - **New `-exclude-prefix ` option** in slang-test - Allows excluding entire test directories or patterns from execution - Complements existing `-category` and individual test filtering options ### Usage Examples ```bash # Enable device caching (default) slang-test # Disable device caching slang-test -cache-rhi-device false # Exclude tests from specific directories slang-test -exclude-prefix tests/problematic/ slang-test -exclude-prefix tests/slow/ -exclude-prefix tests/experimental/ ``` This change should significantly improve test execution performance, particularly in CI environments with frequent device operations. This is needed for running the GPU test in aarch64, where repeated device creation/destroy is causing driver issues. Needed by: https://github.com/shader-slang/slang/issues/8346 --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Enhances CI reliability and debug logging (#8393)

2025-09-07T17:07:24+00:00

1. Adds retry logic - Implements 3-attempt retry mechanism for intermittent test failures 2. Reduces parallelism for Linux - Changes server counts back to 1 for more stable execution, given the test is short in Linux for now 3. Adds detailed error logging - Enhanced diagnostic information for test parsing failures 4. Add Python 3.10 setup for slangpy tests for github runners 5. Removes submodule checkout for slang and slasngpy tests 6. Adds check-ci job - Implements consolidated CI status for simplified branch protection rule --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Handle slang-test command comments better (#8363)

2025-09-04T03:13:45+00:00

Before this PR only the following was a valid line without any white-space character nor additional `/` character, ``` //TEST: ``` This PR is to allow slang-test to handle the following variants of the test command comments, ``` ///TEST: // TEST: // TEST: ////// TEST: ``` This PR revealed a regression on two tests: - tests/cpp-compiler/c-compile-shared-library.c (cpu) - tests/cpp-compiler/cpp-compile-shared-library.cpp (cpu) They are disabled as a part of this PR. And there is a new github issue to track it later, - https://github.com/shader-slang/slang/issues/8362

render-test: Change D3D12 default to sm_6_5 (#8320)

2025-09-02T23:43:48+00:00

Changes default for render-test to sm_6_5. Since sm_6_5 is the new default, remove the -use-dxil option, add -use-dxcb option Remove -use-dxil option from all test cases. Add -use-dxcb to two tests that needed it. Fixes #7611

Remove the embedded source to avoid self-matching in slang-test (#8305)

2025-08-28T17:45:07+00:00

When `SIMPLE` type test is used with `-g[1-3]` option, the filecheck pattern will most likely to match to the string itself on the embedded source code rather than match to the emitted spirv-asm code. This commit avoids the problem by removing the embedded source code. This commit also provides an option to keep the embedded source code, `-preserve-embedded-source`. The source code removal is happening in two steps: 1. iterate all output lines and find SPIRV-ASM in the following pattern: `%N = OpExtInst %void %M DebugSource %fileId %sourceId`. And then, store the "%sourceId" value to identify which SPIRV instructions are for the embedded source code. 2. iterate all output lines again to find the `%sourceId = OpString "...."` and replace the whole string with the following string, ``` %1 = OpString "// slang-test removed the embedded source // Use `-preserve-embedded-source` to keep it explicitly " ``` This change revealed problems in the existing tests: - tests/bugs/spirv-debug-info.slang : The expected text was missing and it had to be added. The file also had Carrage-Return character on all lines and the pre-commit git hook removed them. - tests/spirv/debug-info.slang : the expected keyword DebugValue had to change to DebugDeclare, because that's what we get with ToT. - tests/spirv/debug-value-dynamic-index.slang : This test is currently failing, and it will pass once DebugLocalVariable instruction missing for parameter of the entry point function #7693 is resolved. --------- Co-authored-by: slangbot

Fail slang-test when VVL printed errors (#8280)

2025-08-26T21:38:57+00:00

fixes https://github.com/shader-slang/slang/issues/8271 This PR does the following, - Fail slang-test when there are VVL error messages. - VVL error for `gfx-unit-test-tool/` were not captured properly by the debug callback. - Set an environment variable, `VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation`, for CI and VisualStudio project setup. - Ignores VVL error about NullHandle is used for the acceleration structure; a fix is at ToT of VVL and not available from release build yet. - Fix VVL error complaining about the varying inputs are not provided for the tests, `gfx-unit-test-tool/linkTimeTypeLayout.internal` and `gfx-unit-test-tool/linkTimeTypeLayoutNested.internal`. --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>