| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Handle debug-layer messages in a separate channel
The Problem (Issue #7343)
The issue was that Vulkan Validation Layer error messages were being mixed into regular test output, causing
potential false positives or negatives. When using -enable-debug-layers true, validation messages would appear in
the same output stream as test results, potentially matching //CHECK: patterns incorrectly.
Example Problem:
- Test expects: //CHECK: 1
- Validation layer prints: VALIDATION ERROR: 1 invalid buffer binding
- Test incorrectly matches the "1" in the error message instead of the actual output
Slang Test Communication Architecture
Execution Modes
Slang has 3 different execution modes controlled by SpawnType:
enum class SpawnType {
UseSharedLibrary, // In-process execution
UseTestServer, // Out-of-process via persistent server
UseFullyIsolatedTestServer, // Out-of-process via isolated server
UseExe // Direct executable spawn
}
1. In-Process Mode (UseSharedLibrary)
┌─────────────────────────────────────────┐
│ slang-test │
│ ┌─────────────┐ ┌─────────────────┐ │
│ │render-test │ │gfx-unit-test │ │
│ │unit-test │ │slangc library │ │
│ └─────────────┘ └─────────────────┘ │
│ │ │ │
│ └───► StdWriters ◄──┘ │
│ (shared) │
└─────────────────────────────────────────┘
- Communication: Direct function calls, shared memory
- Debug callbacks: Single callback instance in StdWriters
- Used when: Default mode for most tests
2. Out-of-Process Mode (UseTestServer)
┌──────────────────┐ JSON-RPC ┌──────────────────┐
│ slang-test │◄──over pipes────┤ test-server.exe │
│ │ │ │
│ ┌─────────────┐ │ │ ┌─────────────┐ │
│ │StdWriters │ │ │ │render-test │ │
│ │+debug │ │ │ │gfx-unit-test│ │
│ │callback │ │ │ │+debug │ │
│ └─────────────┘ │ │ │callback │ │
└──────────────────┘ │ └─────────────┘ │
└──────────────────┘
- Communication: JSON-RPC over stdin/stdout pipes
- Debug callbacks: Separate instances in each process
- Used when: CI/CD, multi-threaded testing, crash isolation
3. Direct Executable Mode (UseExe)
┌──────────────────┐ pipes ┌──────────────────┐
│ slang-test │◄───────────────┤ slangc.exe │
│ │ │ other tools │
└──────────────────┘ └──────────────────┘
- Communication: Standard process pipes (stdout/stderr)
- Debug callbacks: None (external executables)
- Used when: Testing external tools
Communication Mechanisms Deep Dive
JSON-RPC Protocol Over Pipes
The test-server.exe communicates with slang-test using JSON-RPC over stdin/stdout pipes:
// Parent process (slang-test) creates child with pipes
Process* testServerProcess = /* spawn test-server.exe */;
// JSONRPCConnection wraps the pipe communication
JSONRPCConnection connection;
connection.initWithStdStreams(); // Uses stdin/stdout pipes
// Send RPC call
TestServerProtocol::ExecutionResult result;
connection.sendCall("executeTool", &args, &result);
Key Point: The pipes carry structured JSON messages, not raw stdout/stderr. This is what enables clean separation
of different data channels.
Protocol Structure
Your changes extend the ExecutionResult protocol:
struct ExecutionResult {
String stdOut; // Regular program output
String stdError; // Error messages
String debugLayer; // NEW: Debug/validation messages
int32_t result;
int32_t returnCode;
};
Your Debug Layer Solution
The Challenge
Memory pointers cannot cross process boundaries. A debugCallback pointer in the parent process is meaningless in
the child process.
The Solution: String-Based Serialization
You solved this by using string capture and serialization:
1. Debug Callback Interface (slang-std-writers.h)
class IDebugCallback {
virtual void handleMessage(
DebugMessageType type,
DebugMessageSource source,
const char* message) = 0;
};
2. String-Capturing Implementation (slang-support.h)
class CoreDebugCallback : public Slang::IDebugCallback {
StringBuilder m_buf; // Captures messages as strings
void handleMessage(DebugMessageType type, DebugMessageSource source, const char* message) {
if (type == DebugMessageType::Error) {
m_buf << message << '\n'; // Serialize to string
}
}
String getString() { return m_buf.toString(); } // Extract accumulated messages
};
3. Bridge Between RHI and Core (slang-support.h)
class CoreToRHIDebugBridge : public rhi::IDebugCallback {
Slang::IDebugCallback* m_coreCallback;
void handleMessage(rhi::DebugMessageType type, rhi::DebugMessageSource source, const char* message) {
// Convert RHI types to core types and forward
m_coreCallback->handleMessage(convertType(type), convertSource(source), message);
}
};
Data Flow: Debug Messages End-to-End
In-Process Mode Flow
GPU Driver → RHI Debug Callback → Core Debug Callback → String Buffer → Test Output
Out-of-Process Mode Flow
Child Process:
GPU Driver → RHI Debug Callback → Core Debug Callback → String Buffer
↓
Parent Process: JSON-RPC Serialization
Test Output ← String Processing ← ExecutionResult.debugLayer ←┘
Step-by-Step Example
1. Test Execution Starts
// In test-server process
CoreDebugCallback debugCallback;
CoreToRHIDebugBridge bridge;
bridge.setCoreCallback(&debugCallback);
// Set up graphics device with debug layers
deviceDesc.debugCallback = &bridge;
2. Graphics API Call Triggers Validation Error
// Inside Vulkan driver (external code)
// Validation layer detects error and calls our callback
bridge.handleMessage(RHI_ERROR, RHI_LAYER, "Invalid buffer binding");
3. Message Capture
// In CoreDebugCallback::handleMessage
m_buf << "Invalid buffer binding\n"; // Stored in string buffer
4. Test Completion & Serialization
// Back in test-server
TestServerProtocol::ExecutionResult result;
result.debugLayer = debugCallback.getString(); // "Invalid buffer binding\n"
result.stdOut = "1"; // Regular test output
// Send via JSON-RPC
connection.sendResult(&result);
5. Parent Process Receives & Separates Output
// In slang-test process
String output = buildTestOutput(result);
// Results in clean separation:
// standard output = {
// 1
// }
// debug layer = {
// Invalid buffer binding
// }
Why This Solution Works
1. Process Isolation: Each process has its own callback objects, no shared pointers
2. String Serialization: Debug messages converted to strings that can cross process boundaries
3. Protocol Extension: Uses existing JSON-RPC infrastructure, just adds new field
4. Clean Separation: Debug messages never mix with stdout/stderr
5. Backward Compatibility: Existing tests unaffected, debug layer field optional
Key Benefits
- Eliminates False Positives: Debug messages can't interfere with //CHECK: patterns
- Better Debugging: Debug messages clearly separated and labeled
- Robust Architecture: Works across all execution modes
- Minimal Changes: Leverages existing communication infrastructure
This elegant solution transforms a fundamental cross-process communication challenge into a simple string
serialization problem, using the existing test server architecture to cleanly separate validation layer messages
from test results.
* Cover the missing slang-test execution path
* format code (#82)
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
---------
Co-authored-by: slangbot <ellieh+slangbot@nvidia.com>
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add a new slang-test option `-enable-debug-layers`
A variable `disableDebugLayer` is renamed to `enableDebugLayers`,
and a corresponding command-line argument is added,
`-enable-debug-layers`.
The previous option `-disable-debug-layer` is still available, but it
prints a deprecation warning message.
The reason why it is added is to make the option available to both Debug
and Release. On Debug build, it will be enabled by default, and it will
be disabled on Release build. We should be able to not only disable it,
but also enable it on Release build.
Ideally this option should be enabled all the time, but currently there
are too many VUID error messages printed and we are enabling only for
Debug build for now.
Note that the CI/CD will run with the option disabled until we resolve
all of VUID errors.
|
