slang.git/tools/render-test/render-test-main.cpp, branch master

Add RHI Device Caching and Test Prefix Exclusion (#8448)

2025-09-22T22:46:42+00:00

# Add RHI Device Caching and Test Prefix Exclusion ## Summary This PR introduces two key improvements to the Slang test infrastructure: 1. **RHI Device Caching**: Implements device caching to significantly speed up test execution by reusing graphics devices across tests, **RHI Device Caching reduces slang-test execution time from ~15 minutes to ~5 minutes in Windows release builds** 2. **Test Prefix Exclusion**: Adds `-exclude-prefix` option to skip tests matching specified path prefixes ## Changes ### RHI Device Caching - **New `DeviceCache` class** (`slang-test-device-cache.h/cpp`): Thread-safe device cache with LRU eviction (max 10 devices) - **Cache control option**: `-cache-rhi-device` flag in both `slang-test` and `render-test` - Default: **enabled** in slang-test, **disabled** in render-test when run standalone - Automatically skips caching for CUDA devices (due to driver issues) - **Performance benefit**: Eliminates expensive device creation/destruction cycles, especially beneficial for Vulkan on Tegra platforms ### Test Prefix Exclusion - **New `-exclude-prefix ` option** in slang-test - Allows excluding entire test directories or patterns from execution - Complements existing `-category` and individual test filtering options ### Usage Examples ```bash # Enable device caching (default) slang-test # Disable device caching slang-test -cache-rhi-device false # Exclude tests from specific directories slang-test -exclude-prefix tests/problematic/ slang-test -exclude-prefix tests/slow/ -exclude-prefix tests/experimental/ ``` This change should significantly improve test execution performance, particularly in CI environments with frequent device operations. This is needed for running the GPU test in aarch64, where repeated device creation/destroy is causing driver issues. Needed by: https://github.com/shader-slang/slang/issues/8346 --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

render-test: Change D3D12 default to sm_6_5 (#8320)

2025-09-02T23:43:48+00:00

Changes default for render-test to sm_6_5. Since sm_6_5 is the new default, remove the -use-dxil option, add -use-dxcb option Remove -use-dxil option from all test cases. Add -use-dxcb to two tests that needed it. Fixes #7611

Handle debug-layer messages in a separate channel (#7988)

2025-07-31T21:32:02+00:00

* Handle debug-layer messages in a separate channel The Problem (Issue #7343) The issue was that Vulkan Validation Layer error messages were being mixed into regular test output, causing potential false positives or negatives. When using -enable-debug-layers true, validation messages would appear in the same output stream as test results, potentially matching //CHECK: patterns incorrectly. Example Problem: - Test expects: //CHECK: 1 - Validation layer prints: VALIDATION ERROR: 1 invalid buffer binding - Test incorrectly matches the "1" in the error message instead of the actual output Slang Test Communication Architecture Execution Modes Slang has 3 different execution modes controlled by SpawnType: enum class SpawnType { UseSharedLibrary, // In-process execution UseTestServer, // Out-of-process via persistent server UseFullyIsolatedTestServer, // Out-of-process via isolated server UseExe // Direct executable spawn } 1. In-Process Mode (UseSharedLibrary) ┌─────────────────────────────────────────┐ │ slang-test │ │ ┌─────────────┐ ┌─────────────────┐ │ │ │render-test │ │gfx-unit-test │ │ │ │unit-test │ │slangc library │ │ │ └─────────────┘ └─────────────────┘ │ │ │ │ │ │ └───► StdWriters ◄──┘ │ │ (shared) │ └─────────────────────────────────────────┘ - Communication: Direct function calls, shared memory - Debug callbacks: Single callback instance in StdWriters - Used when: Default mode for most tests 2. Out-of-Process Mode (UseTestServer) ┌──────────────────┐ JSON-RPC ┌──────────────────┐ │ slang-test │◄──over pipes────┤ test-server.exe │ │ │ │ │ │ ┌─────────────┐ │ │ ┌─────────────┐ │ │ │StdWriters │ │ │ │render-test │ │ │ │+debug │ │ │ │gfx-unit-test│ │ │ │callback │ │ │ │+debug │ │ │ └─────────────┘ │ │ │callback │ │ └──────────────────┘ │ └─────────────┘ │ └──────────────────┘ - Communication: JSON-RPC over stdin/stdout pipes - Debug callbacks: Separate instances in each process - Used when: CI/CD, multi-threaded testing, crash isolation 3. Direct Executable Mode (UseExe) ┌──────────────────┐ pipes ┌──────────────────┐ │ slang-test │◄───────────────┤ slangc.exe │ │ │ │ other tools │ └──────────────────┘ └──────────────────┘ - Communication: Standard process pipes (stdout/stderr) - Debug callbacks: None (external executables) - Used when: Testing external tools Communication Mechanisms Deep Dive JSON-RPC Protocol Over Pipes The test-server.exe communicates with slang-test using JSON-RPC over stdin/stdout pipes: // Parent process (slang-test) creates child with pipes Process* testServerProcess = /* spawn test-server.exe */; // JSONRPCConnection wraps the pipe communication JSONRPCConnection connection; connection.initWithStdStreams(); // Uses stdin/stdout pipes // Send RPC call TestServerProtocol::ExecutionResult result; connection.sendCall("executeTool", &args, &result); Key Point: The pipes carry structured JSON messages, not raw stdout/stderr. This is what enables clean separation of different data channels. Protocol Structure Your changes extend the ExecutionResult protocol: struct ExecutionResult { String stdOut; // Regular program output String stdError; // Error messages String debugLayer; // NEW: Debug/validation messages int32_t result; int32_t returnCode; }; Your Debug Layer Solution The Challenge Memory pointers cannot cross process boundaries. A debugCallback pointer in the parent process is meaningless in the child process. The Solution: String-Based Serialization You solved this by using string capture and serialization: 1. Debug Callback Interface (slang-std-writers.h) class IDebugCallback { virtual void handleMessage( DebugMessageType type, DebugMessageSource source, const char* message) = 0; }; 2. String-Capturing Implementation (slang-support.h) class CoreDebugCallback : public Slang::IDebugCallback { StringBuilder m_buf; // Captures messages as strings void handleMessage(DebugMessageType type, DebugMessageSource source, const char* message) { if (type == DebugMessageType::Error) { m_buf << message << '\n'; // Serialize to string } } String getString() { return m_buf.toString(); } // Extract accumulated messages }; 3. Bridge Between RHI and Core (slang-support.h) class CoreToRHIDebugBridge : public rhi::IDebugCallback { Slang::IDebugCallback* m_coreCallback; void handleMessage(rhi::DebugMessageType type, rhi::DebugMessageSource source, const char* message) { // Convert RHI types to core types and forward m_coreCallback->handleMessage(convertType(type), convertSource(source), message); } }; Data Flow: Debug Messages End-to-End In-Process Mode Flow GPU Driver → RHI Debug Callback → Core Debug Callback → String Buffer → Test Output Out-of-Process Mode Flow Child Process: GPU Driver → RHI Debug Callback → Core Debug Callback → String Buffer ↓ Parent Process: JSON-RPC Serialization Test Output ← String Processing ← ExecutionResult.debugLayer ←┘ Step-by-Step Example 1. Test Execution Starts // In test-server process CoreDebugCallback debugCallback; CoreToRHIDebugBridge bridge; bridge.setCoreCallback(&debugCallback); // Set up graphics device with debug layers deviceDesc.debugCallback = &bridge; 2. Graphics API Call Triggers Validation Error // Inside Vulkan driver (external code) // Validation layer detects error and calls our callback bridge.handleMessage(RHI_ERROR, RHI_LAYER, "Invalid buffer binding"); 3. Message Capture // In CoreDebugCallback::handleMessage m_buf << "Invalid buffer binding\n"; // Stored in string buffer 4. Test Completion & Serialization // Back in test-server TestServerProtocol::ExecutionResult result; result.debugLayer = debugCallback.getString(); // "Invalid buffer binding\n" result.stdOut = "1"; // Regular test output // Send via JSON-RPC connection.sendResult(&result); 5. Parent Process Receives & Separates Output // In slang-test process String output = buildTestOutput(result); // Results in clean separation: // standard output = { // 1 // } // debug layer = { // Invalid buffer binding // } Why This Solution Works 1. Process Isolation: Each process has its own callback objects, no shared pointers 2. String Serialization: Debug messages converted to strings that can cross process boundaries 3. Protocol Extension: Uses existing JSON-RPC infrastructure, just adds new field 4. Clean Separation: Debug messages never mix with stdout/stderr 5. Backward Compatibility: Existing tests unaffected, debug layer field optional Key Benefits - Eliminates False Positives: Debug messages can't interfere with //CHECK: patterns - Better Debugging: Debug messages clearly separated and labeled - Robust Architecture: Works across all execution modes - Minimal Changes: Leverages existing communication infrastructure This elegant solution transforms a fundamental cross-process communication challenge into a simple string serialization problem, using the existing test server architecture to cleanly separate validation layer messages from test results. * Cover the missing slang-test execution path * format code (#82) Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> --------- Co-authored-by: slangbot Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Fix D3D12 test failure without Windows Developer mode (#7831)

2025-07-19T00:23:06+00:00

* Fix D3D12 test failure without Windows Developer mode Add Windows Developer mode detection for D3D12 experimental features. D3D12 Agility SDK requires Developer mode when using experimental features like D3D12ExperimentalShaderModels. Without this check, device creation fails silently. Changes: - Add isWindowsDeveloperModeEnabled() function to check registry - Add developer mode check before enabling experimental features - Provide clear error message with instructions to enable Developer mode - Return SLANG_E_NOT_AVAILABLE to gracefully fail tests Fixes #6819 Co-authored-by: Gangzheng Tong * Add more regkey checks to cover the developer mode detections in multile Windows versions --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gangzheng Tong

Update slang-rhi (#7303)

2025-06-06T14:39:14+00:00

* update slang-rhi * adapt to new slang-rhi API * enable slang-rhi agility sdk * fix handling empty list * disable failing slang-rhi tests * format code * fix slang-rhi-tests ci step * skip running slang-rhi-tests --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

Add a new slang-test option `-enable-debug-layers` (#7300)

2025-06-02T23:47:16+00:00

* Add a new slang-test option `-enable-debug-layers` A variable `disableDebugLayer` is renamed to `enableDebugLayers`, and a corresponding command-line argument is added, `-enable-debug-layers`. The previous option `-disable-debug-layer` is still available, but it prints a deprecation warning message. The reason why it is added is to make the option available to both Debug and Release. On Debug build, it will be enabled by default, and it will be disabled on Release build. We should be able to not only disable it, but also enable it on Release build. Ideally this option should be enabled all the time, but currently there are too many VUID error messages printed and we are enabling only for Debug build for now. Note that the CI/CD will run with the option disabled until we resolve all of VUID errors.

Cleanups related to RIFF support (#7041)

2025-05-12T17:28:05+00:00

update slang-rhi (#6587)

2025-04-24T08:23:06+00:00

* update slang-rhi submodule * slang-rhi API changes * disable agility sdk * fix texture creation * update formats in tests * Extent3D rename * use 1 mip level for 1D textures for Metal * fix texture upload * update to latest slang-rhi * update slang-rhi * format code * update slang-rhi * do not run texture-intrinsics test on metal * update slang-rhi * deal with failing tests * fix more tests * update slang-rhi --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> Co-authored-by: Simon Kallweit

update slang-rhi (shader object refactor) (#6251)

2025-02-28T01:54:22+00:00

* remove unused resource * define buffer data * add vs2022 build presets * update slang-rhi API usage * update slang-rhi --------- Co-authored-by: Yong He

Fix slang raytracing test (#6404)

2025-02-21T18:31:49+00:00

* Plumb raytracing shaders through render-test-main RenderTestApp is missing handling for RayTracing in a few places. * fixup formatting * enable optix test category by default * Revert "enable optix test category by default" This reverts commit d142068058a827843fd7ebcf89a6ec252fd3c69a. --------- Co-authored-by: cheneym2