| Age | Commit message (Collapse) | Author |
|---|
|
generic expressions) (#6787)
|
|
* A new approach to AST serialization
This change completely overhauls the way that AST nodes are being serialized, and the offline source-code generation steps that enable that serialization.
In practice, this ends up being a complete overhaul of the way that *modules* are being serialized (not just the AST part), although things like the serialization format for the Slang IR and for source locations are not affected.
The rest of this commit message is broken down in to sections, in an attempt to help guide anybody looking at the code in how to make sense of all the changes.
The Old C++ Extractor
---------------------
AST serialization used to be driven by information scraped using the `slang-cpp-extractor` tool, which did an ad hoc parse of the C++ declarations of the AST node types and then generated a set of "X macros" that could be for macro-based code generation within the rest of the compiler.
While the existing approach was functional, it wasn't easy to understand or maintain, and it has been getting in the way of forward progress on other features we'd like to work on in the language and compiler.
This change removes the `slang-cpp-extractor` tool entirely.
Marking Up the AST Declarations
-------------------------------
The most notable change that contributors to the compiler may notice is the large number of invocations of a macro `FIDDLE()` on the declarations of the AST node types.
The basic idea is that only declarations (namespaces, types, fields) that are preceded by `FIDDLE()` are visible to the code generator tool.
So if somebody is working with the AST and wondering why a new node type isn't working, or why a field they added isn't being serialized correctly, it is probably because they need to add `FIDDLE()` in front of it.
Generating the Boilerplate Code
-------------------------------
The file `slang-ast-boilerplate.cpp` provides a good example of how the information extracted from the marked-up AST declarations gets used.
In that file, the `FIDDLE TEMPLATE` construct is used to generate type information for each of the AST node types.
Similar logic is used in `slang-ast-forward-declarations.h` to generate the declaration of the `ASTNodeType` enumeration, and forward-declare all the AST node classes.
For many parts of the code, simply including that file replaces the need for the old `slang-generated-*.h` files.
Replacing Visitors and Related Logic
------------------------------------
The old visitor types for the AST used the macros that were generated by `slang-cpp-extractor`, so something new was needed to replace them.
The same goes for the `SLANG_AST_NODE_VIRTUAL_CALL` macros.
The core of the solution implemented here is in `slang-ast-dispatch.h`.
Given a "dispatchable" AST node type (say, `Expr`), a call like:
```
ASTNodeDispatcher<Expr,R>(expr, [&](auto e) { return doSomething(e); })
```
is an expression of type `R`, which does the equivalent of something like:
```
switch(expr->getTag())
{
case ASTNodeType::VarExpr: return doSomething(static_cast<VarExpr*>(expr));
// ...
}
```
The `SLANG_AST_NODE_VIRTUAL_CALL` macro is now implemented in terms of `ASTNodeDispatcher`.
The implementation of the visitor types is more involved.
The code in this change retains some of the macro names from the original version, just to try and make the parallels more clear.
The visitor types are all implemented on top of the `ASTNodeDispatcher` approach, and use `FIDDLE TEMPLATE` to generate all the boilerplate `visit*()` method declarations.
Refactoring of `Linkage` Module Loading
---------------------------------------
Needing to revisit all the places where modules get deserialized made it clear that there is a lot of complexity and apparent duplication in the core routines on the `Linkage` that get used for loading modules.
This change tries to clean up some of that logic, but it is worth noting that there are two legacy features that get in the way of making things as clean as they should be:
* The `LoadedModuleDictionary` type that gets passed around a lot exists entirely to handle the corner case where somebody uses the Slang API to perform a compilation with multiple `TranslationUnitRequest`s in the same `FrontEndCompileRequest`, and one of the translation units `import`s the module defined by another of the translation units.
* There are a lot of special-case behaviors and routines entirely there to support the `ModuleLibrary` feature, although that feature should be considered deprecated (or at least subject to getting entirely re-designed down the line).
The basic idea of the cleanup is that all of the (non-deprecated) ways load a module from a serialized binary, or compile one from source should now bottleneck through `loadModuleImpl`, which then bifurcates into `loadSourceModuleImpl` for the compilation case and `loadBinaryModuleImpl` for the deserialization case.
High-Level Serialization Approach
---------------------------------
The old serialization logic used the [RIFF](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) format to encode the high-level structure of things, and this change retains that usage (and actually doubles down on the RIFF usage).
The old serialization system relied on the idea that for any given type `Foo` that wants to support serialization, there should be something like a `SerialFooData` type in C++, that can represent the state of a `Foo`, and then the actual serialization applied to that `SerialFooData`. This means that in most cases there are four pieces of code written:
* During serialization:
* Copying the data of a `Foo` in memory over to a `SerialFooData` in memory
* Writing the state of a `SerialFooData` into the serialized data stream
* During deserialization:
* Reading the state of a `SerialFooData` from a serialized data stream
* Copying the data of the `SerialFooData` in memory over to a `Foo`
The new logic gets rid of the intermediate `SerialFooData`.
In the serialization direction, we take a `Foo` and write it to the `RIFFContainer` directly, or using some other utilities layered on top of it.
In the deserialization direction, we have additional flexibility. Given a `RIFFContainer::Chunk*` that represents a serialized `Foo`, we often navigate through the in-memory representation of the RIFF data to get to the parts of the serialized value that we actually want/need, without needing to deserialize the entire `Foo`.
To support this kind of operation, this change introduces a few helper types like `ContainerChunkRef` an `ModuleChunkRef`, that are little more than typed wrappers around a `RIFFContainer::Chunk*`.
The Module "Container" Part
---------------------------
A serialized `Module` is encoded as a RIFF chunk, using logic in `slang-serialize-container.cpp` - both before and after this change.
This change reorganizes a lot of the code in that file, to account for the way that eliminating the intermediate `SerialContainerData` type streamlines the overall task of writing out the parts of the module.
In the deserialization logic... there isn't really much to do in `slang-serialize-container.cpp`. Most of the logic in `slang.cpp` and `slang-module-library.cpp` that pertains to deserializing modules uses the `ModuleChunkRef`-based approach, and simply extracts the pieces of the serialized module that it needs.
The Actual Serialization of the AST
-----------------------------------
The actual AST serialization logic is in `slang-serialize-ast.cpp`.
The basic approach in both the writing and reading directions is:
* Use the `FIDDLE TEMPLATE` system to generate a set of functions, one for each AST node type, that recursively invoke the read/write logic on each field of that node (after recursively invoking the case for its direct superclass)
* Use the `ASTNodeDispatcher` system to dispatch out to those functions whene reading or writing anything derived from `NodeBase`
* For now, handle all types *not* derived from `NodeBase` by hand.
There's a lot of room for improvement around that last item: it should be just as easy to generate the serialization and deserialization logic for other types that don't inherit from `NodeBase`, but the current change tries to err on the side of making the logic as explicit and simplistic as possible, rather than trying to get too clever too soon.
The actual serialization *format* used for the AST is almost comically simplistic: the code uses hierarchical RIFF chunks to emulate a JSON-like structure. This is a very wasteful representation (e.g., a `bool` or a null pointer each take up *8 bytes*), but the goal for now is to start with the simplest thing that could possibly work, and only add more cleverness once we are sure it won't get in the way of important future improvements (like lazy/on-demand deserialization or IR and AST, to improve compiler startup times).
The files `slang-serialize.{h,cpp}` have been co-opted to define a new pair of types `Encoder` and `Decoder` that are used for a more-or-less stream-oriented way or reading or writing RIFF chunks for the JSON-like structure.
Almost everything related to the actual AST serialization could do with a cleanup pass, and some time spent on picking good/better names for everything.
Smaller Stuff
-------------
* Cleaned up a lot of code that was using bare `ASTNodeType` or the extractor's `ReflectClassInfo` type to consistently use `SyntaxClass`.
* Fixed an apparent bug in how the destination-driven code genarator was handling `TryExpr`s
* Fixed an apparent bug in how the GLSL legalization pass was handling translation of certain `SV_*` semantics.
* format code
* fixup: template errors caught by non-VS compilers
* format code
* fixup: more template errors
* fixup: more stuff VS didn't catch
* fixup: it's amazing VS doesn't catch these...
* fixup: yet more template stuff VS ignores
* fixup: more VS template nonsense
* fixup: unreachable return macro usage
* fixup: more unreacable returns
* fixup: unused parameter
* fixup: strict aliasing
* fixup: allow missing entry point list chunk
* fixup: wasm build script
* fixup: AST changes since this PR was created
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Use the latest Ubuntu version not specific old version
|
|
Closes https://github.com/shader-slang/slang/issues/5995
|
|
* Add support for Ray Payload Access Qualifiers (PAQs) (#3448)
- Added [raypayload] attribute for struct declarations
- Implemented field validation requiring read/write access qualifiers
- Added diagnostic error for missing qualifiers
- Enabled PAQs in DXC compiler and HLSL emission
- Added new test demonstrating PAQ syntax
- Implemented proper handling of ray payload attributes in IR generation
* format code
* Cleanup: Remove unused vars
* Add check to enablePAQ only for profile >= lib_6_7
* Review Fix - Add PAQ support for DX Raytracing
add enablePAQ flag to DownstreamCompileOpitons, improve PAQ handling
update raypayload-attribute-paq.slang to ensure hlsl and dxil is
validated
* Add diagnostic test for missing paq for lib_6_7
Compile using `-disable-payload-qualifiers` aka lib_6_6 profile
raypayload-attribute-no-struct.slang and
raypayload-attribute.slang
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
|
|
* Fix a bug in default ctor synthesizing
- This is fix for the implementation bug, when a struct has explicit ctor
we should not synthesize the default ctor anymore.
- When invoke the synthesized ctor converted from initializer list, we should
check if the struct is a c-style type if it struct has no synthesized ctor. In this
case we should report error because it's invalid to use initializer list here.
- The only exception is the unsized array, we still have to fall back to use the
legacy initializer list logic to initialize the unsized array until we formalize a
proper solution.
- update test.
|
|
* update hlsl meta
* update test
* use slang syntax in meta file
* improve meta file
* fix pack clamp u8
* remove builtin packed types, use typealias instead
* fix wgsl pack clamp
* fix formatting
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Make capability diagnostic message more friendly.
* Fix.
* Fix.
* Fix.
* Fix test.
* Update expected fail setting for aarch64/linux
* Fix.
|
|
|
|
* Expose value of constant integers in module reflection
This commit adds `VariableReflection::getDefaultValueInt` to get the value of a variable if it is a compile-time constant integer.
TODO: currently it works only if the initializer expression is an integer literal, references to other constant values are not handled.
* Update VarDecl folded constant value during DeclBodyVisitor
Constant folding for integer values is already done internally by _validateCircularVarDefinition, this just reuses the result.
* Address review comments & formatting
* Formatting
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
Improve performance when compiling small shaders.
Avoid copying witness table entries that are not getting used during linking.
Avoid copying auto-diff related decorations and derivative functions during linking, if the user modules doesn't use autodiff.
Cache operator overload resolution results on global session, so each new Session doesn't need to repetitively run through overload resolution from scratch.
|
|
* Simplify implicit cast ctors for vector & matrix.
* Fix formatting.
* Fix tests.
* Fix Falcor test.
* Mark __builtin_cast as internal.
|
|
* Allow LHS of `where` to be any type.
* Register free-form extensions when loading precompiled module.
* Fix test.
* Fix.
* Fix `as<IRType>`.
* try fix precompiled module test.
|
|
Co-authored-by: Yong He <yonghe@outlook.com>
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
* Remove the docs/proposals directory
This directory will get added to the spec repository in the following PR:
https://github.com/shader-slang/spec/pull/6
This closes #6155.
* Remove entry from .github/CODEOWNERS file
* Redirect some proposal references
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Support stage_switch.
* Update proposal status.
* Fix gl_InstanceID.
* Fix.
|
|
* Use two-stage parsing to disambiguate generic app and comparison.
* Typo fix.
* Update doc.
|
|
* SP004: implement initialize list translation to ctor
- We synthesize a member-wise constructor for each struct follow
the rules described in SP004.
- Add logic to translate the initialize list to constructor invoke
- Add cuda-host decoration for the synthesized constructor
- Remove the default constructor when we have a valid member init constructor
- Disable -zero-initialize option, will re-implement it in followup (#6109).
- Fix the overload lookup issue
When creating invoke expression for ctor, we need to call
ResolveInvoke() to find us the best candidates, however
the existing lookup logic could find us the base constructor
for child struct, we should eliminate this case by providing
the LookupOptions::IgnoreInheritance to lookup, this requires
us to create a subcontext on SemanticsVisitor to indicate that
we only want to use this option on looking the constructor.
- Do not implicit initialize a struct that doesn't have explicit default
constructor.
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
|
* Fix ConstantIntVal::toText when the val is a enum.
* Fix comment.
|
|
* Fix the type coerce issue
When synthesize the default ctor, if there is a base type
we will synthesize an InvokeExpr to call base type's default
ctor as well. But we should use the type of the inheritanceDecl
instead of base struct decl.
|
|
* fix non-static methods when trying to synthesize method requirement witness
* add tests
* update test
* improve test
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Properly plumbing layout for global varyings.
* Fix test.
|
|
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Fix cyclic lookups with UnscopedEnums
* Add test with multiple unscoped enums with explicit types
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Inline global constants that contains opaque handles for legalization.
* Add diagnostics on opaque type global variables.
* Fix.
* Fix test.
|
|
differentiable functions (#5866)
* Overhauled the auto-diff system for dynamic dispatch
* More fixes
* remove intermediate dumps
* Update slang-ast-type.h
* More fixes + add a workaround for existential no-diff
* Update reverse-control-flow-3.slang
* remove dumps
* remove more dumps
* Delete working-reverse-control-flow-3.hlsl
* Cleanup comments + unused variables
* More comment cleanup
* Add support for lowering `DiffPairType(TypePack)` & `MakePair(MakeValuePack, MakeValuePack)`
* Fix array of issues in Falcor tests.
* Update slang-ir-autodiff-pairs.cpp
* More fixes for Falcor image tests
* Small fixups.
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Add executable test on matrix-typed vertex input.
* Fix emit logic of matrix layout qualifier.
* Pass fragment shader varying input by constref to allow EvaluateAttributeAtCentroid etc. to be implemented correctly.
|
|
|
|
(#5964)
|
|
inheritance decl. (#5965)
|
|
* Add packed bytes builtin type
* fix test
|
|
|
|
* Add verification logic on push and specialization constants.
* format code
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
|
|
|
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
half. (#5814)
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
* Fix pointer offset logic and add executable tests.
* Fix.
* Fix test.
* Add existential ptr test.
* Allow pointers to existential values.
* Fix.
* Fix.
---------
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
|
|
can't be automatically inferred (#5630)
* [AD] Add support for resolving custom derivatives where generic parameters can't be automatically inferred
* Fix failing tests
* Update custom-derivative-generic.slang
|
|
* Add datalayout for constant buffers.
* Fixes.
* Fix test.
* Fix glsl codegen.
* Update spirv-specific doc.
* Fix test.
* Fix binding in the presense of specialization constants.
* address comments.
* Add a test for constant buffer layout.
|
|
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
* Embed core module in wasm build.
* format code
* add uintptr_t case.
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
|
* Move switch statement bodies to their own lines
* format
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
|
|
* [wasm]: Add function to get string from hash
* Fix bug on the visibility issue
* Formatting
|
|
* format
* Minor test fixes
* enable checking cpp format in ci
|
|
(#5415)
This commit changes the word "stdlib" or "standard library" to "core module" in the source code.
|
|
|
|
* Don't use __assume for SLANG_ASSERT + build fixes.
* Fix.
* build slang-wasm conditionally
* Fix.
* revert retry open file
* revert include.
* another attempt of silencing compiler warnings.
* revert assume change.
|
