slang.git/tools/slang-test/options.cpp, branch master

Add deterministic shuffling of tests in directory (#8622)

2025-10-08T17:13:47+00:00

Fixes #8621 Add command line options for enable shuffling as well as providing a custom seed. Use Mersenne-Twister engine for a deterministic shuffle.

Adding slang-test option to ignore abort popup message (#8492)

2025-09-24T06:46:31+00:00

With the recent Windows runtime libraries, a new popup window started appearing when `abort()` is called. This was observed when VVL prints a message as a part of WGPU test. Although it can be helpful when we want to debug it, it breaks the behavior of CI scripts when the tests are expected to continue even when they fail. When the test fail, CI script stops in the middle and wait for a user to click on a button on the dialog window, which cannot happen. As a result, when there is a VVL error message, CI run stops in the middle and the testing stops prematurely. This commit adds a new command-line argument, `-ignore-abort-msg`, that ignores the abort message and it wouldn't show the dialog popup window. From the implementation perspective, there are three places that are related. - slang-test itself should turn off the flag. - render-test should turn off the flag after getting the argument from slang-test - test-server should turn off the flag after getting the argument from slang-test When test-server runs render-test, the arguments are already handled by slang-test, so test-server needs to just pass through the arguments.

Add RHI Device Caching and Test Prefix Exclusion (#8448)

2025-09-22T22:46:42+00:00

# Add RHI Device Caching and Test Prefix Exclusion ## Summary This PR introduces two key improvements to the Slang test infrastructure: 1. **RHI Device Caching**: Implements device caching to significantly speed up test execution by reusing graphics devices across tests, **RHI Device Caching reduces slang-test execution time from ~15 minutes to ~5 minutes in Windows release builds** 2. **Test Prefix Exclusion**: Adds `-exclude-prefix` option to skip tests matching specified path prefixes ## Changes ### RHI Device Caching - **New `DeviceCache` class** (`slang-test-device-cache.h/cpp`): Thread-safe device cache with LRU eviction (max 10 devices) - **Cache control option**: `-cache-rhi-device` flag in both `slang-test` and `render-test` - Default: **enabled** in slang-test, **disabled** in render-test when run standalone - Automatically skips caching for CUDA devices (due to driver issues) - **Performance benefit**: Eliminates expensive device creation/destruction cycles, especially beneficial for Vulkan on Tegra platforms ### Test Prefix Exclusion - **New `-exclude-prefix ` option** in slang-test - Allows excluding entire test directories or patterns from execution - Complements existing `-category` and individual test filtering options ### Usage Examples ```bash # Enable device caching (default) slang-test # Disable device caching slang-test -cache-rhi-device false # Exclude tests from specific directories slang-test -exclude-prefix tests/problematic/ slang-test -exclude-prefix tests/slow/ -exclude-prefix tests/experimental/ ``` This change should significantly improve test execution performance, particularly in CI environments with frequent device operations. This is needed for running the GPU test in aarch64, where repeated device creation/destroy is causing driver issues. Needed by: https://github.com/shader-slang/slang/issues/8346 --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Add comment support to expected-failure files in slang-test (#7817)

2025-07-18T20:14:33+00:00

* Initial plan * Add comment support to expected-failure files in slang-test Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

Use stdout for --help text instead of stderr (#7730)

2025-07-11T21:57:38+00:00

* Use stdout for --help text instead of stderr * format code (#13) Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Improve slang-test output verbosity control (#7625)

2025-07-08T18:33:50+00:00

* Improve slang-test output verbosity control This commit improves the existing command-line argument for slang-test, "-v". Previously it printed more information when "-v" was used. This commit adds a new option to silence the information output so that LLM processes less tokens when things are working as expected. * format code (#74) --------- Co-authored-by: slangbot

Add a new slang-test option `-enable-debug-layers` (#7300)

2025-06-02T23:47:16+00:00

* Add a new slang-test option `-enable-debug-layers` A variable `disableDebugLayer` is renamed to `enableDebugLayers`, and a corresponding command-line argument is added, `-enable-debug-layers`. The previous option `-disable-debug-layer` is still available, but it prints a deprecation warning message. The reason why it is added is to make the option available to both Debug and Release. On Debug build, it will be enabled by default, and it will be disabled on Release build. We should be able to not only disable it, but also enable it on Release build. Ideally this option should be enabled all the time, but currently there are too many VUID error messages printed and we are enabling only for Debug build for now. Note that the CI/CD will run with the option disabled until we resolve all of VUID errors.

Enable Windows full debug testsuite in CI (#7085)

2025-05-16T21:51:46+00:00

* Unify Debug Layer Control Logic and Add Disable Option for Debug Builds This PR refactors and unifies the debug layer control logic in slang-test. A new `-disable-debug-layers` option is introduced, allowing debug builds to skip enabling the validation (debug) layer. This is currently needed to ensure stability in the debug test suite. Previously, different toggles such as ENABLE_VALIDATION_LAYER, ENABLE_DEBUG_LAYER, and debugLayerEnabled were used inconsistently across different components of slang-test. This PR standardizes the logic by using a single variable, debugLayerEnabled, to control the enabling/disabling of the debug layer internally. Notes: By default, the debug/validation layer is enabled in debug builds and is not supported in release builds of slang-test. Fixes: #7132 * Disable spirv-opt for the DebugFunctionDefinition issue * Run debug build only in GCP machines * Fix VUID-vkCmdPipelineBarrier-pBufferMemoryBarriers-02818 dstAcessMask can't include VK_ACCESS_TRANSFER_READ_BIT when stage mask has VK_PIPELINE_STAGE_RAY_TRACING_SHADER_BIT_KHR * Set failed retry limit to 32 --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Add a new option "-capability" to slang-test and render-test (#7054)

2025-05-11T08:13:25+00:00

fix(d3d11): correct parameter in VSSetConstantBuffers1 from uavCount … (#6690)

2025-04-04T06:57:08+00:00

* fix(d3d11): correct parameter in VSSetConstantBuffers1 from uavCount to cbvCount (fixes #6531) Root cause - Incorrect parameter passing in slang-rhi 1. slang-rhi #281 - Add the correct cbvCount for setting Constant Buffer 2. Prevent render tests from overwriting reference images * Add missing tests/render/multiple-stage-io-locations.slang.3.expected.png * Add more expected images from texture2d-gather * Add new option: skipReferenceImageGeneration For Github CI we set this to true - So we don't overwrite the expected images --------- Co-authored-by: Jay Kwak <82421531+jkwak-work@users.noreply.github.com>