summaryrefslogtreecommitdiffstats
path: root/source/slang-record-replay
Commit message (Collapse)AuthorAge
* Add record and replay support for IComponentType2 (#8215)jarcherNV2025-08-21
| | | | Add record and replay support for the IComponentType2 struct and its functions getTargetCompileResult and getEntryPointCompileResult.
* Introduce CDataLayout & -fvk-use-c-layout (#8136)Julius Ikkala2025-08-21
| | | | | | | | | | | | | | | | Closes #8112. ~~The issue asks for a "C layout", but in this PR I use the term "CPU layout" because this naming was pre-existing in the codebase as `kCPULayoutRulesImpl_`. The primary purpose of this layout is to match CPU-side struct definitions with the shader side. I'm open to better naming suggestions, though.~~ Edit: switched back to using `CDataLayout` & `-fvk-use-c-layout`, as the CPU target depends on the object layout rules of existing CPU layout rules, but they're incompatible with actual shaders. So a new `kCLayoutRulesImpl_` was needed anyway. --------- Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
* Stable names and backwards compat for serialized IR modules (#7644)Ellie Hermaszewska2025-07-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable names * tests, options and ci for stable names * Add back compat design document * fix warnings * formatting * comment * neaten * regenerate command line reference * consolidate ci scripts * faster ci * remove libreadline * Move new function to end of interface --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
* Remove some cruft/complexity from IR serialization (#7483)Theresa Foley2025-07-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove some cruft/complexity from IR serialization This is a very simple cleanup to unnecessary code paths and remove some flexibility that isn't actually needed, to hopefully simplify the task of more completely overhauling the approach to IR serialization in a later change. The concrete feature that gets removed here is a debug-only feature (which thus shouldn't be affecting any users of Slang) that was added long ago in the life of the compiler as we were working to truly separate the front- and back-ends. At the time there was a lot of code in the compiler back-end that still made use of AST-level data structures, and thus got in the way of our goal to support separate compilation and linking (such that final code generation can only depend on the IR, and not the AST). The option was used to cause the Slang IR to be serialized out and then read back in as part of compilation, to try and enforce that only the wanted constructs could pass through that bottleneck. The idea was only ever half implemented, however, because it made use of a secondary implementation path in IR serialization that supported serializing the "raw" source locations (which are heavily dependent on AST-level information, even down to the number of bytes in source files). This change removes the feature entirely, since it is no longer useful for its intended purpose, and its presence causes there to be entire second code path for source locations in IR serialization that would need to have test coverage if we wanted to be sure it kept working. In addition, our pre-existing infrastructure for module serialization had various options that have either stopped being useful, or were not really useful at the time they were introduced. For example: there are no places in the code today where we attempt to serialize out a module without including both the serialized AST and IR. If that was a feature that we ever supported, the relevant code got removed at some preceding point without breaking any of our tests or (seemingly) upsetting users. Similarly, the options being passed into writing of a serialized module included both a flag to control whether source locations should be serialized *and* a pointer to the `SourceManager` to use in that case... but it was only ever meaningful to set both, or neither. The option has been changed to just be the `SourceManager` pointer, and the name has been updated to reflect its very narrow intended use case. * format code * fixup * regenerate command line reference --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> Co-authored-by: Yong He <yonghe@outlook.com>
* Fix API changes from separate debugging support (#7397)jarcherNV2025-06-12
| | | | | | Recent separate debugging support added two new functions which broke backwards compatibility. This change restores the old API and moves the new functions to an IComponentType2 interface which can be used if separate debug files are needed.
* Add command line option for separate debug info (#7178)jarcherNV2025-06-06
| | | | | | | | | | | | | * Add command line option for separate debug info Add command line arg -separate-debug-info which, if provided, produces both a .spv and a .dbg.spv file. The .dbg.spv file contains full debug info and the .spv file has all debug info stripped out. Also add a DebugBuildIdentifier instruction to store a unique hash in both the output files, so they can be more easily matched together. A matching API is provided to allow using the Slang API to retrieve a base and debug SPIRV as well as the debug build identifier string.
* Make interface types non c-style in Slang2026. (#7260)Yong He2025-06-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make interface types non c-style. * Make Optional<T> work with autodiff and existential types. * Fix. * patch behind slang 2026. * Fix warnings. * cleanup. * Fix tests. * Fix. * Fix com interface lowering. * Add comment to test. * regenerate command line reference * Add test for passing `none` to autodiff function. * Fix recording of `getDynamicObjectRTTIBytes`. * Fix nested Optional types. --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
* Break down record replay to individual tests to avoid timeout (#7340)Jay Kwak2025-06-04
| | | | | | | | | | | | | | | | * Break down RecordReply to individual tests to avoid timeout In Debug build, RecordReplay unit-test was timing out. It was running six tests all in one unit-test, but this commit breaks it down to individual test so that each unit test can be done within the timeout limit. This issue has seen only in Debug build but it has been unnoticed because even when the test failed with test-server, it was still passing on its retry because the time-out applies only when using test-server. * Reduce the retry from 2 times to 1 time * Remove RecordReplay from expected failure
* Language version + tuple syntax. (#7230)Yong He2025-05-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Language version + tuple syntax. * Fix compile error. * regenerate documentation Table of Contents * Fix. * regenerate command line reference * Fix. * Fix. * Fix more test failures. * revert empty line change, * Retrigger CI * #version->#lang * Update source/core/slang-type-text-util.cpp Co-authored-by: ArielG-NV <159081215+ArielG-NV@users.noreply.github.com> * Remove comments. * Fix parsing logic. * Fix parser. * Fix parser. * update test comment * Update options. * regenerate documentation Table of Contents * regenerate command line reference --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> Co-authored-by: ArielG-NV <159081215+ArielG-NV@users.noreply.github.com>
* Fix compiler warning with clang 18.1.8 on windows (#6963)Jay Kwak2025-04-30
|
* Add -dump-module command to slangc (#6638)cheneym22025-03-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add -dump-module command to slangc The new -dump-module command to slangc will load and disassemble a slang module, similar to what would be seen by the -dump-ir command, except that -dump-ir tells slangc to print IR as it performs some compilation command. That is, -dump-ir requires some larger compilation task. -dump-module on the otherhand requires no additional goal and will simply load a module and print its IR to stdout independently from other compilation steps. Its intended purpose is to inspect .slang-module files on disk. It can also be used on .slang files which will be parsed and lowered if slang does not find an associated ".slang-module" version of the module on disk. The compilation API is extended with a new IModule::disassemble() method which retrieves the string representation of the dumped IR. Closes #6599 * format code * Use FileStream not FILE * format code --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com> Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
* Distribute slang-glsl-module.bin in release packages. (#6233)Yong He2025-01-31
| | | | | | | | | * Distribute slang-glsl-module.bin in release packages. * Fix. * fix2. * Build glsl modlue into a dll.
* Cache and reuse glsl module. (#6152)Yong He2025-01-22
| | | | | | | | | | | * Cache and reuse glsl module. * Fix. * Implement record/replay for the new api. * Fix record replay. * Fix test.
* Embed core module in wasm build. (#5569)Yong He2024-11-15
| | | | | | | | | | | * Embed core module in wasm build. * format code * add uintptr_t case. --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
* Move switch statement bodies to their own lines (#5493)Ellie Hermaszewska2024-11-05
| | | | | | | | | * Move switch statement bodies to their own lines * format --------- Co-authored-by: Yong He <yonghe@outlook.com>
* formatEllie Hermaszewska2024-10-29
| | | | | | | * format * Minor test fixes * enable checking cpp format in ci
* Replace stdlib on Slang API with CoreModule (#5405)Jay Kwak2024-10-25
| | | | | | | | | | | | | | | | | | This is a breaking change in a way that the Slang API function names are changed. All of them are commented as "experimental" and we wouldn't provide a back-ward compatibility for them. Following functions are renamed: compileStdLib() -> compileCoreModule() loadStdLib() -> loadCoreModule() saveStdLib() -> saveCoreModule() slang_createGlobalSessionWithoutStdLib() -> slang_createGlobalSessionWithoutCoreModule() slang_getEmbeddedStdLib() -> slang_getEmbeddedCoreModule() hasDeferredStdLib() -> hasDeferredCoreModule() Following command-line arguments are renamed: "-load-stdlib" -> "-load-core-module" "-save-stdlib" -> "-save-core-module" "-save-stdlib-bin-source" -> "-save-core-module-bin-source" "-compile-stdlib" -> "-compile-core-module"
* Move C interface from slang.h to slang-deprecated.h (#5301)Ellie Hermaszewska2024-10-15
| | | | | | | | | | | | | | | | | | | | | | | | | * Squash redundant move warnings * Move C interface from slang.h to slang-deprecated.h spGetBuildTagString remains, because it's useful to have before the global session exists. This C API is used quite pervasively in the C++ helpers (for example slang::UserAttribute. It's not trivial to move these to slang-deprecated.h as they're entangled with some enums which are themselves used elsewhere in the compiler. The fact that these helpers use the C API can be viewed as an implementation detail for now, and this usage moved to slang-deprecated in due course. Closes https://github.com/shader-slang/slang/issues/4758 * Squash warnings for our usage of our deprecated API --------- Co-authored-by: Yong He <yonghe@outlook.com>
* Add interfaces for retrieving separate linkable downstream binaries (#5128)cheneym22024-10-04
| | | | | | | | | | | | | | | | | * Implement separate downstream library interface Create a new com interface to house the methods for precompiling slang modules to target code. Add methods to count dependent modules and scrape them for downstream target binaries such that the downstream target binaries are linkabe outside of slang, e.g. via spirv-link or dxc. Fixes #5147 * Rename to _Experimental Clearly identify this as an interface subject to change.
* Add COM API for querying metadata. (#5168)Yong He2024-09-30
| | | | | | | * Add COM API for querying metadata. * Fix tests. * fix test.
* Report AD checkpoint contexts (#5058)venkataram-nv2024-09-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Transferring source locations when creating phi instructions * Tracking for simple variables * Deriving source locations for loop counters * Printing checkpoint structure breakdown * More readable output format * Special behavior for loop counters * Writing report to file * Add slangc option to enable checkpoint reports * Display types of checkpointed fields * Message in case there are no checkpointing contexts * Catch source locations for function calls * Source cleanup * Fix compilation warnings * Remove stray dump() * Provide the report through diagnostic notes * Add missing path for sourceLoc during unzip pass * Add tests for reporting intermediates * Include more transfer cases for source locations * Fix ordering in address elimination * Fill in more holes with source location transfer * Remove debugging line * Reverting changes to diagnostic sink * Simplify address elimination using source location RAII contexts * Eliminating manual source loc transfers in forward transcription * Fix local var adaptation to use RAII location setter * Simplify primal hoisting logic for source location transfer * Simplify unzipping with RAII location scopes * Simplify transpose logic * Cleaning up for rev.cpp * Reverting spacing changes * Fix mistake with source loc RAII instantiation * Fix formatting issues
* Initial WGSL support (#5006)Anders Leino2024-09-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add WGSL as a target This is required for #4807. * C-like emitter: Allow the function header emission to be overloaded WGSL-style function headers are pretty different from normal C-style headers: Normal C-style headers: ReturnType Func(...) void VoidFunc(...) WGSL-style headers: fn Func(...) -> ReturnType fn VoidFunc(...) This change allows the header style to be overloaded, in order to accomodate WGSL-style headers as required to resolve issue #4807, but retains normal C-style headers as the default implementation. [1] https://www.w3.org/TR/WGSL/#function-declaration-sec * C-like emitter: Allow emission of switch case selectors to be overloaded The C-like emitter will emit code like this: switch(a.x) { case 0: case 1: { ... } break; ... } This is not allowed in WGSL. Instead, selectors for cases that share a body must [1] be separated by commas, like this: switch(a.x) { case 0, 1: { ... } break; ... } To prepare for addressing issue #4807, this patch makes the emission of switch case selectors overloadable. [1] https://www.w3.org/TR/WGSL/#syntax-case_selectors * C-like emitter: Support WGSL-style declarations This patch helps to address issue 4807. C-like languages declare variables like this: i32 a; WGSL declares variables like this: var a : i32 The patch introduces overloads so that the forthcoming WGSL emitter can output WGSL-style declarations, which helps to resolve #4807. * C-like emitter: Support overloading of declarators Unlike C-like languages, WGSL does not support the following types at the syntax level, via declarators: - arrays - pointers - references For this reason, this patch introduces support for overloading the declarator emitter, in order to help address issue #4807. C-like languages: int a[3]; // Array-ness of type is mixed into the "declarator" WGSL: var a : array<int, 3>; // Array-ness of type is part of the... type_specifier! * C-like emitter: Allow struct declaration separator to be overridden C-like languages use ';' as a separator, and languages like e.g. WGSL use ','. This change prepares for addressing issue #4807. * C-like emitter: Allow overriding of whether pointer-like syntax is necessary Things like e.g. structured buffers map to "ptr-to-array" in WGSL, but ptr-typed expressions don't always need C-style pointer-like syntax. Therefore, make it overrideable whether or not such syntax is emitted in various cases in order to address #4807. * C-like emitter: Emit parenthesis to avoid warning about & and + precedence This helps with #4807 because WGSL compilers (e.g. Tint) treat absence of parenthesis as an error. * C-like emitter: Add hook for emitting struct field attributes WGSL requires @align attributes to specify explicit field alignment in certain cases. Thus, this patch prepares for addressing #4807. * C-like emitter: Add hook for emitting global param types Declarations of structured buffers map to global array declarations in WGSL. However, in all other cases such as when structured buffers are used in operands, their types map to *ptr*-to-array. This patch makes it possible for the WGSL back-end to say that structured buffers generally map to "ptr-to-array" types, but still have a special case of just "array" when declaring the global shader parameter. Thus, this patch helps with addressing #4807. * IR lowering: Use std140 for WGSL uniform buffers This patch just cuts out some logic that prevented std140 to be chosen for WGSL uniform buffers. Note that WGSL buffers in the uniform address space is not quite std140, but for now it's close enough to avoid compile issues. Later on, a custom layout should be created for WGSL uniform buffers. When that's done, this change will be revisited, but for now it helps to resolve #4807. * Don't emit line directives in WGSL by default WGSL does not support line directives [1]. The plan currently seems to be to instead support source-map [2]. This is part of addressing issue #4807. [1] https://github.com/gpuweb/gpuweb/issues/606 [2] https://github.com/mozilla/source-map * WGSL IR legalization: Map SV's The implementation closely follows the cooresponding one for Metal. Supported: - DispatchThreadID - GroupID - GroupThreadID - GroupThreadID Unsupported: - GSInstanceID This is not complete, but it helps to address #4807. * WGSL emitter: Add support for basic language constructs A lot of the basics are added in order to generate correct WGSL code for basic Slang language constructs. This addresses issue #4807. This adds support for at least the following: - statments - if statements - ternary operator - while statement - for statements - variable declarations - switch statements - Note: Slang may emit non-constant case expressions, see issue 4834 - literals - integer literals - u?int[16|32|64]_t - float and half literals - bool literals - vector literals and splatting (e.g 1.xxx) - function definitions - assignments - +=, *=, /= - array assignments - vector assignments/updates - swizzles of other vectors - from matrix rows ('m[i]' notation) - from matrix cols (using swizzle notation, e.g 'm._11_12_13') - matrix assignments/updates - to rows ('m[i]' notation) - to cols (using swizzle notation, e.g 'm._11_12_13') - declarations - arrays [1] https://www.w3.org/TR/WGSL/#syntax-switch_body * Add some WGSL capabilities This patch registers some WGSL capabilities required to pass many of the initial compute shader compile tests. Many capabilities still remain to be added -- this is just an initial set to help resolve issue #4807. - asint - min and max - cos and sin - all and any * WGSL and C-like emitters: Add hack to bitcast case expression In WGSL, the switch condition and case types must match. https://www.w3.org/TR/WGSL/#switch-statement Slang currently allows these types to mismatch, as pointed out in #4921. Issue #4921 should eventually be addressed in the front-end by a patch like [1]. However, at the moment that would break Falcor tests. Thus, this patch temporarily works around the issue in the WGSL emitter only in order to help resolve #4807. In the future, the Falcor tests should be fixed, this patch should be dropped and [1] should be merged instead. [1] a32156ef52f43b8503b2c77f2f1d51220ab9bdea
* capture component type (#4967)kaizhangNV2024-08-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactor the IComponentType recording Refactor the `IComponentType` recording by creating a abstract class `IComponentTypeRecorder` to record all the methods of `IComponentType`, so that `ICompositeComponentType`, `IModule`, 'IEntryPoint', 'ITypeConformance' can share the same recording implementation. Capture the out IComponentType from linkWithOptions() link() specialize() renameEntryPoint() * fix bugs * Finish the unimeplemented functions in json consumer Fix the address print to use 64 bit hex. Fix the reference count issue when allocating new recorder object. * Disable few examples using reflection APIs * Add gpu-printing example into slang-test * Replace of using std::unique_ptr with RefPtr
* Migrate 3 more examples to slang-unit-test (#4927)kaizhangNV2024-08-28
| | | | | | | | | * Convert 'ray-tracing-pipeline' example into slang-test * Convert model-view and 'autodiff-texture' examples to slang-test * Add more error message in RecordReplay test * Fix a shader issue in autodiff-texture
* Migrate examples (#4920)kaizhangNV2024-08-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Migrate cpu-hello-world to new slang API Migrate cpu-hello-world to new slang API, and also convert this example as one of the unit test. * Add 'shader-object' to slang-unit-test * Convert ray-tracing example into unit-test Convert ray-tracing example into unit-test * Fix some replay bugs: - Wrong decode type in 'getEntryPointHostCallable'. - Mistakes in computing the output buffer size. - Wrong decode type in array size in specialize() call. - When capture entrypoint, we should increase the reference count for the allocated entrypoint recorder object, because that is allocated by record layer, it should be owned by the layer, user should not be able to free it. - Improve json consumer on the prelude text. * Test verify change: In our test, we add a "callIdx" string at beginning of the hash-code string, as there could be more than one modules in the example, so they could call 'getEntryPointHash' multiple times, in order for the test can identify them, add "callIdx: <number>" as the key word.
* Implement `-fvk-use-dx-layout` (#4912)ArielG-NV2024-08-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implement `-fvk-use-dx-layout` Fixes: #4126 Changes: * Added fvk-use-dx-layout * Modified `HLSLConstantBufferLayoutRulesImpl` for correctness (ex: Array is always 16 byte aligned) * Added kFXCShaderResourceLayoutRulesFamilyImpl and kFXCConstantBufferLayoutRulesFamilyImpl to handle fvk-use-dx-layout * Added `ConstantBufferLayoutRules` to manage constant buffer rules * Added `alignCompositeElementOfNonAggregate`/`alignCompositeElementOfAggregate` to handle forced alignment of composites for ConstantBuffers * `StructuredBuffer` rules are mostly equal to `scalar` layout, not much was needed to be changed to support this behavior. * seperate legacy constant buffer and how Slang does constant-buffer normally * undo an addition * remove accidental test * Address review and fix Address review and remove GLSL support since GLSL requires a seperate legalization (need to linearlize structs like with `legalizeMetalIR` to assign explicit offsets) * comments * remove aggregate and non-aggregate logic We don't need this distinction for the logic --------- Co-authored-by: Yong He <yonghe@outlook.com>
* Feature/capture unit test (#4898)kaizhangNV2024-08-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * record/replay: Add tests Modify the hello-world example to generate the hash code for the entry point spirv code, so that we can compare it with replaying the example. Add the test script to run the example and compare the hash code with replaying it. * Check nullptr for out Diagnostics We need to check whether the output Diagnostics is a nullptr, because it's allowed. * Fix the double free pointers * Add triangle example as the new test for record-replay Change the example base to add the offline rendering path because we don't want to display anything when we're in the test mode. This change involves introducing a TestBase that will parse the command line option. It will decide whether we are in the test mode. Disable all the swapchain and windows related creation, instead we will only create one single framebuffer for the render target. * Address comments TODO: In the follow up patches, I will add more tests and integrate the test flow into slang-unit-test.
* Make precompileForTargets work with Slang API (#4845)cheneym22024-08-15
| | | | | | | | | | | | | | | | | * Make precompileForTargets work with Slang API precompileForTargets, renamed to precompileForTarget, does not need an EndToEndCompileRequest and some objects created from it are not necessary either. Take only a target enum and a diagnostic blob as input and handle everything else internally, such as creating the TargetReq with chosen profile. Fixes #4790 * Update slang-module.cpp * Update slang-module.cpp
* Reduce dependency on std library (#4785)skallweitNV2024-08-07
|
* Feature/replayer (#4750)kaizhangNV2024-07-31
| | | | | | | | | | | | | | | | | | | | | * record/replay: Implement the json consumer Finish the implementation of json consumer. Fix some bug in the block processing as Tailer is not a necessary block so if the Magic bit is "HEAD", we should keep processing. * record/replay: Implement the replayer component Implement the replayer consumer, and also finish the slang-replay standalone app that will run the while replayer. It can take an option "--convert-json | -cj" which will convert the record binary file to a human readable json file. If there is no option provided, it will replay the record file by default. TODO: #4764 is created to remove the std::filesystem usage.
* Feature/capture (#4625)kaizhangNV2024-07-23
* Add decoder * Add a replay executable to consume the decoded content Add file-processor.cpp/h where we implement the logic to process the captured file block by block. Each block is: function header + parameter buffer + function tailer + function output[optional]. After reading one block, the block of data is sent to decoder module to dispatch the corresponding API. Add slang-decoder.cpp/h where we implement the logic to dispatch the slang API according to the input block data. - Rename api_callId.h to capture-format.h - Renmae capture_utility.cpp to capture-utility.cpp - Renmae capture_utility.h to capture-utility.h - Change the #include file name accordingly. * Reorganize source files structure Move all the capture logic code into `capture` directory. - the capture code will be build with slang dll. Move all the replay logic code into `relay` directoy. - the replay code is not part of slang dll, it will be built as a stand alone binary and link against slang dll. Change the #include file names accordingly. Add tools/slang-replay/main.cpp for the slang-replay stand alone binary place holder. Will implement it later. Update premake5.lua accordingly. * Update cmake files Update cmake files to change the build process for capture and relay system. - capture component should be build with slang dll, so we should not include replay component. - replay component should be a separate executable tool, which should not include capture component. - In order to easy use our current cmake infrastructure, move the shared files to a `util` folder - change the header include path * Redesgin the interfaces of consumers Fix some issues in capture Finish implementing all slang-decoder functions * Fix the AppleClang build issue * Address few comments - Fix the weird indent issues. - Correct the function name for CreateGlobalSession() - Rename file-processor to captureFile-processor to be more specific. - Use Slang::List instead of std::vector * record/replay: name refactor change Refactor the naming. Change the name "encoder/capture" to "record".