Add RHI Device Caching and Test Prefix Exclusion (#8448)

# Add RHI Device Caching and Test Prefix Exclusion

## Summary

This PR introduces two key improvements to the Slang test
infrastructure:

1. **RHI Device Caching**: Implements device caching to significantly
speed up test execution by reusing graphics devices across tests, **RHI
Device Caching reduces slang-test execution time from ~15 minutes to ~5
minutes in Windows release builds**
2. **Test Prefix Exclusion**: Adds `-exclude-prefix` option to skip
tests matching specified path prefixes

## Changes

### RHI Device Caching
- **New `DeviceCache` class** (`slang-test-device-cache.h/cpp`):
Thread-safe device cache with LRU eviction (max 10 devices)
- **Cache control option**: `-cache-rhi-device` flag in both
`slang-test` and `render-test`
- Default: **enabled** in slang-test, **disabled** in render-test when
run standalone
  - Automatically skips caching for CUDA devices (due to driver issues)
- **Performance benefit**: Eliminates expensive device
creation/destruction cycles, especially beneficial for Vulkan on Tegra
platforms

### Test Prefix Exclusion
- **New `-exclude-prefix <prefix>` option** in slang-test
- Allows excluding entire test directories or patterns from execution
- Complements existing `-category` and individual test filtering options

### Usage Examples
```bash
# Enable device caching (default)
slang-test

# Disable device caching
slang-test -cache-rhi-device false

# Exclude tests from specific directories
slang-test -exclude-prefix tests/problematic/
slang-test -exclude-prefix tests/slow/ -exclude-prefix tests/experimental/
```

This change should significantly improve test execution performance,
particularly in CI environments with frequent device operations. This is
needed for running the GPU test in aarch64, where repeated device
creation/destroy is causing driver issues.

Needed by: https://github.com/shader-slang/slang/issues/8346

---------

Co-authored-by: slangbot <ellieh+slangbot@nvidia.com>
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

5 files changed, 92 insertions, 0 deletions

diff --git a/tools/slang-test/options.cpp b/tools/slang-test/options.cpp
index ec30262fc..7a98caec6 100644
--- a/tools/slang-test/options.cpp
+++ b/tools/slang-test/options.cpp
@@ -76,6 +76,7 @@ static bool _isSubCommand(const char* arg)
         "  -verbose-paths                 Use verbose paths in output\n"
         "  -category <name>               Only run tests in specified category\n"
         "  -exclude <name>                Exclude tests in specified category\n"
+        "  -exclude-prefix <prefix>       Exclude tests with specified path prefix\n"
         "  -api <expr>                    Enable specific APIs (e.g., 'vk+dx12' or '+dx11')\n"
         "  -synthesizedTestApi <expr>     Set APIs for synthesized tests\n"
         "  -skip-api-detection            Skip API availability detection\n"
@@ -91,6 +92,7 @@ static bool _isSubCommand(const char* arg)
         "  -capability <name>             Compile with the given capability\n"
         "  -enable-debug-layers [true|false] Enable or disable Validation Layer for Vulkan\n"
         "                                 and Debug Device for DX\n"
+        "  -cache-rhi-device [true|false] Enable or disable RHI device caching (default: true)\n"
 #if _DEBUG
         "  -disable-debug-layers          Disable the debug layers (default enabled in debug "
         "build)\n"
@@ -357,6 +359,18 @@ static bool _isSubCommand(const char* arg)
                 optionsOut->excludeCategories.add(category, category);
             }
         }
+        else if (strcmp(arg, "-exclude-prefix") == 0)
+        {
+            if (argCursor == argEnd)
+            {
+                stdError.print("error: expected operand for '%s'\n", arg);
+                showHelp(stdError);
+                return SLANG_FAIL;
+            }
+            Slang::StringBuilder sb;
+            Slang::Path::simplify(*argCursor++, Slang::Path::SimplifyStyle::NoRoot, sb);
+            optionsOut->excludePrefixes.add(sb);
+        }
         else if (strcmp(arg, "-api") == 0)
         {
             if (argCursor == argEnd)
@@ -488,6 +502,26 @@ static bool _isSubCommand(const char* arg)
                 optionsOut->enableDebugLayers = false;
             }
         }
+        else if (strcmp(arg, "-cache-rhi-device") == 0)
+        {
+            optionsOut->cacheRhiDevice = true;
+
+            if (argCursor == argEnd)
+            {
+                stdError.print("error: expected operand for '%s'\n", arg);
+                showHelp(stdError);
+                return SLANG_FAIL;
+            }
+
+            // Check for false variants
+            const char* value = *argCursor++;
+            if (value[0] == 'f' || value[0] == 'F' || value[0] == 'n' || value[0] == 'N' ||
+                value[0] == '0' ||
+                ((value[0] == 'o' || value[0] == 'O') && (value[1] == 'f' || value[1] == 'F')))
+            {
+                optionsOut->cacheRhiDevice = false;
+            }
+        }
 #if _DEBUG
         else if (strcmp(arg, "-disable-debug-layers") == 0)
         {
diff --git a/tools/slang-test/options.h b/tools/slang-test/options.h
index df2de310a..9223b2eaf 100644
--- a/tools/slang-test/options.h
+++ b/tools/slang-test/options.h
@@ -64,6 +64,9 @@ struct Options
     // only run test cases with names have one of these prefixes.
     Slang::List<Slang::String> testPrefixes;
 
+    // skip test cases with names that have one of these prefixes.
+    Slang::List<Slang::String> excludePrefixes;
+
     // verbosity level for output
     VerbosityLevel verbosity = VerbosityLevel::Info;
 
@@ -133,6 +136,9 @@ struct Options
 
     bool emitSPIRVDirectly = true;
 
+    // Whether to enable RHI device caching in render-test (default: true in slang-test)
+    bool cacheRhiDevice = true;
+
     Slang::HashSet<Slang::String> capabilities;
     Slang::HashSet<Slang::String> expectedFailureList;
 
diff --git a/tools/slang-test/slang-test-main.cpp b/tools/slang-test/slang-test-main.cpp
index 0297227d6..08019f995 100644
--- a/tools/slang-test/slang-test-main.cpp
+++ b/tools/slang-test/slang-test-main.cpp
@@ -3643,6 +3643,11 @@ static void _addRenderTestOptions(const Options& options, CommandLine& ioCmdLine
     {
         ioCmdLine.addArg("-enable-debug-layers");
     }
+
+    if (options.cacheRhiDevice)
+    {
+        ioCmdLine.addArg("-cache-rhi-device");
+    }
 }
 
 static SlangResult _extractProfileTime(const UnownedStringSlice& text, double& timeOut)
@@ -4766,6 +4771,23 @@ static bool shouldRunTest(TestContext* context, String filePath)
     if (!endsWithAllowedExtension(context, filePath))
         return false;
 
+    // Check exclude prefixes first - if any match, skip the test
+    for (auto& excludePrefix : context->options.excludePrefixes)
+    {
+        if (filePath.startsWith(excludePrefix))
+        {
+            if (context->options.verbosity == VerbosityLevel::Verbose)
+            {
+                context->getTestReporter()->messageFormat(
+                    TestMessageType::Info,
+                    "%s file is excluded from the test because it is found from the exclusion "
+                    "list\n",
+                    filePath.getBuffer());
+            }
+            return false;
+        }
+    }
+
     if (!context->options.testPrefixes.getCount())
     {
         return true;
@@ -5129,6 +5151,15 @@ static SlangResult runUnitTestModule(
     return SLANG_OK;
 }
 
+static void cleanupRenderTestDeviceCache(TestContext& context)
+{
+    auto cleanFunc = context.getCleanDeviceCacheFunc("render-test");
+    if (cleanFunc)
+    {
+        cleanFunc();
+    }
+}
+
 SlangResult innerMain(int argc, char** argv)
 {
     auto stdWriters = StdWriters::initDefaultSingleton();
@@ -5420,12 +5451,15 @@ SlangResult innerMain(int argc, char** argv)
         }
 
         reporter.outputSummary();
+
+        cleanupRenderTestDeviceCache(context);
         return reporter.didAllSucceed() ? SLANG_OK : SLANG_FAIL;
     }
 }
 
 int main(int argc, char** argv)
 {
+    // Fallback: run without cleanup if context initialization fails
     SlangResult res = innerMain(argc, argv);
     slang::shutdown();
     Slang::RttiInfo::deallocateAll();
diff --git a/tools/slang-test/test-context.cpp b/tools/slang-test/test-context.cpp
index ede12d0d7..e3655e61d 100644
--- a/tools/slang-test/test-context.cpp
+++ b/tools/slang-test/test-context.cpp
@@ -130,6 +130,8 @@ TestContext::InnerMainFunc TestContext::getInnerMainFunc(const String& dirPath,
             loader->loadPlatformSharedLibrary(path.begin(), tool.m_sharedLibrary.writeRef())))
     {
         tool.m_func = (InnerMainFunc)tool.m_sharedLibrary->findFuncByName("innerMain");
+        tool.m_cleanDeviceCacheFunc =
+            (CleanDeviceCacheFunc)tool.m_sharedLibrary->findFuncByName("cleanDeviceCache");
     }
 
     m_sharedLibTools.add(name, tool);
@@ -152,6 +154,17 @@ void TestContext::setInnerMainFunc(const String& name, InnerMainFunc func)
     }
 }
 
+TestContext::CleanDeviceCacheFunc TestContext::getCleanDeviceCacheFunc(const String& name)
+{
+    SharedLibraryTool* tool = m_sharedLibTools.tryGetValue(name);
+    if (tool)
+    {
+        return tool->m_cleanDeviceCacheFunc;
+    }
+
+    return nullptr;
+}
+
 DownstreamCompilerSet* TestContext::getCompilerSet()
 {
     std::lock_guard<std::mutex> lock(mutex);
diff --git a/tools/slang-test/test-context.h b/tools/slang-test/test-context.h
index e760e8dda..0161637dd 100644
--- a/tools/slang-test/test-context.h
+++ b/tools/slang-test/test-context.h
@@ -90,6 +90,7 @@ class TestContext
 {
 public:
     typedef Slang::TestToolUtil::InnerMainFunc InnerMainFunc;
+    typedef void (*CleanDeviceCacheFunc)();
 
     /// Get the slang session
     SlangSession* getSession() const { return m_session; }
@@ -101,6 +102,9 @@ public:
     /// Set the function for the shared library
     void setInnerMainFunc(const Slang::String& name, InnerMainFunc func);
 
+    /// Get the device cache cleanup function (from shared library)
+    CleanDeviceCacheFunc getCleanDeviceCacheFunc(const Slang::String& name);
+
     void setTestRequirements(TestRequirements* req);
 
     TestRequirements* getTestRequirements() const;
@@ -196,6 +200,7 @@ protected:
     {
         Slang::ComPtr<ISlangSharedLibrary> m_sharedLibrary;
         InnerMainFunc m_func;
+        CleanDeviceCacheFunc m_cleanDeviceCacheFunc;
     };
 
     Slang::List<Slang::RefPtr<Slang::JSONRPCConnection>> m_jsonRpcConnections;