| Age | Commit message (Collapse) | Author |
|---|
|
* Add check for the variable requirement
This change adds the capability check for the variables requirement.
With this check, the shader
```
[require(cpp_cuda_glsl_hlsl_metal_spirv)]
Buffer<float> InputTyped;
[require(cpp_cuda_glsl_hlsl_metal_spirv)]
RWBuffer<float> OutputTyped;
```
will issue error if targeting to WSGL
e.g. `.\build\Debug\bin\slangc .\tests\wgsl_no_buffer.slang -o
wgsl_no_buffer.txt -target wgsl -entry Main -stage compute`
.\tests\wgsl_no_buffer.slang(2): error 36108: 'InputTyped' has dependencies that are not compatible on the required target 'wgsl'.
Buffer<float> InputTyped;
^~~~~~~~~~
.\tests\wgsl_no_buffer.slang(4): error 36108: 'OutputTyped' has dependencies that are not compatible on the required target 'wgsl'.
RWBuffer<float> OutputTyped;
^~~~~~~~~~~
Fixes #6304
* Add var capability tests
* Do capability checks for global var only
* Add inferredCapabilityRequirements to var capability check
* Add requirement to the intrinsic types Buffer/RWBuffer
* format code
* Update capabliity test
* use DefaultDataLayout as default data layout
* Use visitMemberExpr to check the capabilities
* Update the cap tests to match the error messages
* update test to use the ScalarDataLayout for hlsl target
* Update tests check condition to use error number only
* Add default push_constant data layout type
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
|
Close #6859
Goal of this PR
We want to support an array whose size can be specialization constant for shared/global variable e.g.
layout (constant_id = 0) const uint BLOCK_SIZE = 64;
shared float buf_a[(BLOCK_SIZE + 5) * 4];
Overview of the solution:
During IndexExpr check, we will loose the restriction to allow SpecConst passing, but the size parameter will not be a constant value because it cannot be folded into a constant, so we will make it follow the same logic as generic parameter value, and the size will be represented by FuncCallIntVal/PolynomialIntVal/DeclRefIntVal.
During IR lowering, we will detect whether there is spec constant in the IntVal, and wrap the IRInst with a SpecConstRateType, and propagate the type though the lowering logic, such that the IntVal representing the array size will have SpecConstRateType.
During spirv emit stage, if we detect that a IRInst has SpecConstRateType, we will emit it as SpecConstantOp.
We have to implement new logic to emit OpSpecConstantOp, the existing emit logic doesn't support emitting OpSpecConstantOp, especially this op can embed arithmetic operation at global scope, where we can only emit arithmetic instruct at local. But there are only few instructs we need to support.
Overview of the solution:
This PR doesn't support generic, and we will create a separate PR to extend that, tracked in #6840.
|
|
* A new approach to AST serialization
This change completely overhauls the way that AST nodes are being serialized, and the offline source-code generation steps that enable that serialization.
In practice, this ends up being a complete overhaul of the way that *modules* are being serialized (not just the AST part), although things like the serialization format for the Slang IR and for source locations are not affected.
The rest of this commit message is broken down in to sections, in an attempt to help guide anybody looking at the code in how to make sense of all the changes.
The Old C++ Extractor
---------------------
AST serialization used to be driven by information scraped using the `slang-cpp-extractor` tool, which did an ad hoc parse of the C++ declarations of the AST node types and then generated a set of "X macros" that could be for macro-based code generation within the rest of the compiler.
While the existing approach was functional, it wasn't easy to understand or maintain, and it has been getting in the way of forward progress on other features we'd like to work on in the language and compiler.
This change removes the `slang-cpp-extractor` tool entirely.
Marking Up the AST Declarations
-------------------------------
The most notable change that contributors to the compiler may notice is the large number of invocations of a macro `FIDDLE()` on the declarations of the AST node types.
The basic idea is that only declarations (namespaces, types, fields) that are preceded by `FIDDLE()` are visible to the code generator tool.
So if somebody is working with the AST and wondering why a new node type isn't working, or why a field they added isn't being serialized correctly, it is probably because they need to add `FIDDLE()` in front of it.
Generating the Boilerplate Code
-------------------------------
The file `slang-ast-boilerplate.cpp` provides a good example of how the information extracted from the marked-up AST declarations gets used.
In that file, the `FIDDLE TEMPLATE` construct is used to generate type information for each of the AST node types.
Similar logic is used in `slang-ast-forward-declarations.h` to generate the declaration of the `ASTNodeType` enumeration, and forward-declare all the AST node classes.
For many parts of the code, simply including that file replaces the need for the old `slang-generated-*.h` files.
Replacing Visitors and Related Logic
------------------------------------
The old visitor types for the AST used the macros that were generated by `slang-cpp-extractor`, so something new was needed to replace them.
The same goes for the `SLANG_AST_NODE_VIRTUAL_CALL` macros.
The core of the solution implemented here is in `slang-ast-dispatch.h`.
Given a "dispatchable" AST node type (say, `Expr`), a call like:
```
ASTNodeDispatcher<Expr,R>(expr, [&](auto e) { return doSomething(e); })
```
is an expression of type `R`, which does the equivalent of something like:
```
switch(expr->getTag())
{
case ASTNodeType::VarExpr: return doSomething(static_cast<VarExpr*>(expr));
// ...
}
```
The `SLANG_AST_NODE_VIRTUAL_CALL` macro is now implemented in terms of `ASTNodeDispatcher`.
The implementation of the visitor types is more involved.
The code in this change retains some of the macro names from the original version, just to try and make the parallels more clear.
The visitor types are all implemented on top of the `ASTNodeDispatcher` approach, and use `FIDDLE TEMPLATE` to generate all the boilerplate `visit*()` method declarations.
Refactoring of `Linkage` Module Loading
---------------------------------------
Needing to revisit all the places where modules get deserialized made it clear that there is a lot of complexity and apparent duplication in the core routines on the `Linkage` that get used for loading modules.
This change tries to clean up some of that logic, but it is worth noting that there are two legacy features that get in the way of making things as clean as they should be:
* The `LoadedModuleDictionary` type that gets passed around a lot exists entirely to handle the corner case where somebody uses the Slang API to perform a compilation with multiple `TranslationUnitRequest`s in the same `FrontEndCompileRequest`, and one of the translation units `import`s the module defined by another of the translation units.
* There are a lot of special-case behaviors and routines entirely there to support the `ModuleLibrary` feature, although that feature should be considered deprecated (or at least subject to getting entirely re-designed down the line).
The basic idea of the cleanup is that all of the (non-deprecated) ways load a module from a serialized binary, or compile one from source should now bottleneck through `loadModuleImpl`, which then bifurcates into `loadSourceModuleImpl` for the compilation case and `loadBinaryModuleImpl` for the deserialization case.
High-Level Serialization Approach
---------------------------------
The old serialization logic used the [RIFF](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) format to encode the high-level structure of things, and this change retains that usage (and actually doubles down on the RIFF usage).
The old serialization system relied on the idea that for any given type `Foo` that wants to support serialization, there should be something like a `SerialFooData` type in C++, that can represent the state of a `Foo`, and then the actual serialization applied to that `SerialFooData`. This means that in most cases there are four pieces of code written:
* During serialization:
* Copying the data of a `Foo` in memory over to a `SerialFooData` in memory
* Writing the state of a `SerialFooData` into the serialized data stream
* During deserialization:
* Reading the state of a `SerialFooData` from a serialized data stream
* Copying the data of the `SerialFooData` in memory over to a `Foo`
The new logic gets rid of the intermediate `SerialFooData`.
In the serialization direction, we take a `Foo` and write it to the `RIFFContainer` directly, or using some other utilities layered on top of it.
In the deserialization direction, we have additional flexibility. Given a `RIFFContainer::Chunk*` that represents a serialized `Foo`, we often navigate through the in-memory representation of the RIFF data to get to the parts of the serialized value that we actually want/need, without needing to deserialize the entire `Foo`.
To support this kind of operation, this change introduces a few helper types like `ContainerChunkRef` an `ModuleChunkRef`, that are little more than typed wrappers around a `RIFFContainer::Chunk*`.
The Module "Container" Part
---------------------------
A serialized `Module` is encoded as a RIFF chunk, using logic in `slang-serialize-container.cpp` - both before and after this change.
This change reorganizes a lot of the code in that file, to account for the way that eliminating the intermediate `SerialContainerData` type streamlines the overall task of writing out the parts of the module.
In the deserialization logic... there isn't really much to do in `slang-serialize-container.cpp`. Most of the logic in `slang.cpp` and `slang-module-library.cpp` that pertains to deserializing modules uses the `ModuleChunkRef`-based approach, and simply extracts the pieces of the serialized module that it needs.
The Actual Serialization of the AST
-----------------------------------
The actual AST serialization logic is in `slang-serialize-ast.cpp`.
The basic approach in both the writing and reading directions is:
* Use the `FIDDLE TEMPLATE` system to generate a set of functions, one for each AST node type, that recursively invoke the read/write logic on each field of that node (after recursively invoking the case for its direct superclass)
* Use the `ASTNodeDispatcher` system to dispatch out to those functions whene reading or writing anything derived from `NodeBase`
* For now, handle all types *not* derived from `NodeBase` by hand.
There's a lot of room for improvement around that last item: it should be just as easy to generate the serialization and deserialization logic for other types that don't inherit from `NodeBase`, but the current change tries to err on the side of making the logic as explicit and simplistic as possible, rather than trying to get too clever too soon.
The actual serialization *format* used for the AST is almost comically simplistic: the code uses hierarchical RIFF chunks to emulate a JSON-like structure. This is a very wasteful representation (e.g., a `bool` or a null pointer each take up *8 bytes*), but the goal for now is to start with the simplest thing that could possibly work, and only add more cleverness once we are sure it won't get in the way of important future improvements (like lazy/on-demand deserialization or IR and AST, to improve compiler startup times).
The files `slang-serialize.{h,cpp}` have been co-opted to define a new pair of types `Encoder` and `Decoder` that are used for a more-or-less stream-oriented way or reading or writing RIFF chunks for the JSON-like structure.
Almost everything related to the actual AST serialization could do with a cleanup pass, and some time spent on picking good/better names for everything.
Smaller Stuff
-------------
* Cleaned up a lot of code that was using bare `ASTNodeType` or the extractor's `ReflectClassInfo` type to consistently use `SyntaxClass`.
* Fixed an apparent bug in how the destination-driven code genarator was handling `TryExpr`s
* Fixed an apparent bug in how the GLSL legalization pass was handling translation of certain `SV_*` semantics.
* format code
* fixup: template errors caught by non-VS compilers
* format code
* fixup: more template errors
* fixup: more stuff VS didn't catch
* fixup: it's amazing VS doesn't catch these...
* fixup: yet more template stuff VS ignores
* fixup: more VS template nonsense
* fixup: unreachable return macro usage
* fixup: more unreacable returns
* fixup: unused parameter
* fixup: strict aliasing
* fixup: allow missing entry point list chunk
* fixup: wasm build script
* fixup: AST changes since this PR was created
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Simplify implicit cast ctors for vector & matrix.
* Fix formatting.
* Fix tests.
* Fix Falcor test.
* Mark __builtin_cast as internal.
|
|
* Add datalayout for constant buffers.
* Fixes.
* Fix test.
* Fix glsl codegen.
* Update spirv-specific doc.
* Fix test.
* Fix binding in the presense of specialization constants.
* address comments.
* Add a test for constant buffer layout.
|
|
* format
* Minor test fixes
* enable checking cpp format in ci
|
|
(#5415)
This commit changes the word "stdlib" or "standard library" to "core module" in the source code.
|
|
* initial diff-ref-type interface
* Initial support for `IDifferentiablePtrType`
* Fix unused vars
* More tests + fix switch case fallthrough.
* Update slang-ir-autodiff.cpp
* Update diff-ptr-type-loop.slang
* Add optimization to allow more complex pair types
* Update slang-ir-autodiff-primal-hoist.cpp
* Update diff-ptr-type-loop.slang
* Update slang-ir-autodiff-primal-hoist.cpp
* More fixes to address reviews
* Update slang-check-expr.cpp
* Optimizations + rename `differentiableRefInterfaceType` -> `differentiablePtrInterfaceType`
* Move pair logic to ir-builder, unify the type dictionaries.
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
|
|
* Tuple swizzling and element access.
* Update proposal status.
* Cleanup.
* Fix merrge error.
* Address review.
|
|
* Variadic Generics Part 2: IR lowering and specialization.
* Update design doc status.
* Update design doc.
* Resolve review comments.
|
|
|
|
* Overhaul IR lowering of pointer types.
* Propagate address space in IRBuilder.
* Fixup.
* Fix.
* Fix.
* Change how Ptr type is printed to text.
* Fix.
|
|
* Default (zero'd) values with `-zero-initialize` flag
Adds `-zero-initialize` flag to set values to a __default() expression if they are missing a initExpr.
* address review and ensure __default calls ctor + zero's fields.
1. We must keep zero-initialize in SemanticsDeclHeaderVisitor. This is done because else a ctor will be initialized before we can set struct fields to `__default`.
2. IRDefaultCtorDecoration was added to track default ctor's with parent struct.
3. ParentAggTypeModifier was added to track ChildOfStruct->IRType for sharing data such as with functions. This is required to ensure we associate a lowered function with a lowered struct type
* Removed decoration to track defaultCtor in favor of field.
This was done since decorations are checked for IR objects, storing auxillary info does not work here as a result if usable object.
* address some review comments
Since `IDefaultInitializable` is taking a considerabley larger amount of time than anticipated I am pushing some of the other fixes requested. I did not remove the "IRStruct storing a default Ctor" hack yet.
mostly renamed/adjusted tests to work as intended
added test to ensure we don't synthisize a junk `= 0` when not in `zero initialize` mode
removed member in favor of sharedContext+dictionary.
* a working but incorrect impl
* default init without any IR hacks (fully working aside from generic/containored-types)
* Finish zero init code
1. IDefaultInitializer interface was added. If conforming, your type may be zero-initialized. To Conform a `__init()` is required
2. `[OnlyAutoInitIfForced]` was added. This attribute states that a default initializer should only be implicitly called if forced by the compiler (`zero-initialize` for example). This allows types which implicitly/explicitly conform to IDefaultInitialize to have optional auto-init behavior (which is Slang's default for user structs) to be disabled.
* note about `[OnlyAutoInitIfForced]`. This is required for std-lib to not automatically resolve init-expressions for std-lib, but it has the added benifit of allowing user made structs/classes to control the default behavior of initializing
* fix ErrType assumption
* testing why dx12 fails local but passes CI
* push vector changes to generic test
* push syntax adjustment, still figuring out what is wrong with cuda.
* remove debug changes & adjust style
* fix field-init expressions with structs initializers
don't init a static in a ctor. This would be illegal code and wrong code (init list in lower-to-ir)
* minor adjustments temporarily while the rest of the issue is discussed
* fix
* implement IDefaultInitializable
* remove a unneeded whitespace change
* fix type checking error
should be checking if a valid type is `Type`, not `BasicExpressionType`
* needs to be DeclRefType, not Type
* fix langguage server error
* change findinheritance for correctness + cleanup
* remove return false
verified the issue was `findInheritance`
* push attempt at language server fix
* still trying to fix inheritance
* added extension support, remove redundant code
Did not address all review comments yet, want to see if CI also passes my changes
* undo a change which caused CI to fail
* change logic + DefaultConstructExpr
setup code to use defaultConstructExpr when possible to construct a default without overhead of invoke/related
also changed code so parent's defaultInitializable propegates to derived member
* 1. fix error in `isSubtype` 2. add flag to isSubtype
`subtypeInheritanceIsNotFullyResolved` was added since we may not be done the lookup stage but still require `isSubtype` checking to verify usage of inheritance while working with inheritance. In This case we will just skip `ensureLookup` and "caching" (since we don't have a cache invalidation system, nor need)
* fix bug in logic + add test to better catch the bug
* address comment + isSubTypeOption + wrapper type test,
* fix wrong code adjustment
I checked on the CI and realized I caused a failure, mistake was made not negating some code
* syntax, class naming capital
* remove stdlib default initialize changes, replace with `__default()` for init
* remove redundant code + fix defaultConstruct emitting
previously defaultConstruct emitting was crashing due to having generics unresolved. By not resolving the default construct immediately, everything works.
* remove a coment
* add test to ensure static variables dont `init` inside a struct's `__init`
* fix Ptr members breaking struct use
* address review and add -zero-initialize test
`-zero-initialize` test was added to be sure debug pointers are not broken with default init values
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
|
|
* More tests for append structured buffer
* Append and Consume structured buffer tests for DX12
* neaten
* test wobble
* Add counter layout information to append/consume structured buffers
* add getRWStructuredBufferType
* Correct definition of get size for append/consume structured buffers
* tweak append structured buffer test
* Allow initializing counter buffer in render test
* vulkan test for consume structured buffer
* Handle null counterVarLayout in getExplicitCounterBindingRangeOffset
* remove dead code
* Implement atomic counter increment/decrement for spirv
* explicit spirv test
* Add missing check on result
* Hold on to counter resources
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Support `constref` parameters passing.
* Fix.
* Fix.
* Add test and diagnostic on mix use of __constref and no_diff.
* check for [constref] on differentiable member method.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Fix for threading issues around global session & epoch ids.
* Make m_epochId atomic for thread visibility.
|
|
* Correct namespace for getClockFrequency
* missing const
* Add missing assignment operator
* Remove unused variables
* Return correct modified variable
* Use stable hash code for file system identity
* terse static_assert
* Structured binding for map iteration
* Make (==) and getHashCode const on many structs
* Add ConstIterator for LinkedList
* Replace uses of ItemProxy::getValue with Dictionary::at
* Extract list of loads from gradientsMap before updating it
* Const correctness in type layout
* Add unordered_dense hashmap submodule
* Use wyhash or getHashCode in slang-hash.h
* refactor slang-hash.h
* Use ankerl/unordered_dense as a hashmap implementation
Notable changes:
- The subscript operator returns a reference directly to the value,
rather than a lazy ItemProxy (pair of dict pointer and key)
slang-profile time (95% over 10 runs):
- Before: 6.3913906 (±0.0746)
- After: 5.9276123 (±0.0964)
* 64 bit hash for strings
So they have the same hash as char buffers with the same contents
* Narrowing warnings for gcc to match msvc
* revert back to c++17
* Correct c++ version for msvc
* Use path to unordered_dense which keeps tests happy
* Do not assign to and read from map in same expression
* Remove redundant map operations in primal-hoist
* Split out stable hash functions into slang-stable-hash.h
* 64 bit hash by default
* regenerate vs projects
* Correct return type from HashSetBase::getCount()
* correct width for call to Dictionary::reserve
* Use stable hash for obfuscated module ids
* Signed int for reserve
* clearer variable naming
* Parameterize Dictionary on hash and equality functors
* Allow heterogenous lookup for Dictionary
* missing const
* Use set over operator[] in some places
* Remove unused function
* s/at/getValue
|
|
* Support per field matrix layout
* Fix warnings.
* Fix.
* Fix tests.
* Fix spiv gen.
* Fix.
* More test fixes.
* Fix.
* Run only GPU tests on self-hosted servers.
* Remove -use-glsl-matrix-layout-modifier.
* Fix.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Clean up and improve Val deuplication performance.
* Fix.
* Fix.
* Fix.
* Fix.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Redesign DeclRef + Deduplicate Val.
* Update project files
* Fix warning.
* Fix.
* Fix.
* Remove `Val::_equalsImplOverride`.
* Rmove `Val::_getHashCodeOverride`.
* Remove `semanticVisitor` param from `resolve`.
* Cleanups.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Simplify lookup.
* Various bug fixes.
* Report type dictionary size in perf benchmark.
* Remove type duplication.
* increase initial dict size.
* Bug fix.
* Fix bugs.
* Fixup.
* Revert type legalization looping.
* Fix specialization pass.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Create and cache flattened inheritance lists
The basic change here is to have a cached lookup that can map a `Type`,
or a `DeclRef` that might refer to a type or `extension`, to a list of
the *facets* that comprise it.
The notion of a *facet* here is similar to what the C++ standard calls
"sub-objects".
A declared type like a `struct` has:
* a facet for its own direct members
* one facet for each of its (transitive) base `struct` types
* one facet for each `interface` it conforms to
* one facet for each `extension` that applies to that type
The set of facets for a type is de-duplicated (so that "diamond"
inheritance patterns don't cause issues) and deterministically ordered,
using a variation of the C3 linearization algorithm.
The creation of a linearized list of facets should help the compiler
implementation in two key places:
* Testing if a type implements an interface (or inherits from a base
type) should now only take time linear in the number of (transitive)
bases of that type. We can simply scan the linearized facet list to
see if it contains a facet corresponding to the given base.
* Looking up the members of a type (or a value of a given type) should
be greatly simplified, since all of the members can be found in a
single linear scan of the facet list. In addition, those facets will
be ordered so that facets for "more derived" types will precede those
for "less derived" types, so that shadowing in the case of overrides
should be easier to implement.
This change only implements the first of these two improvements, since
there is already a *lot* of churn involved.
Notes and caveats:
* The handling of conjunction types (e.g., `IFoo & IBar`) complicates
the implementation, both because the simple approach to subtype
testing alluded to above is no longer complete, and also because
we need to be more careful about what forms of subtype witnesses
we construct, so that we can maintain the currently-required invariant
that two witnesses are only equal if they have matching structure.
* We don't implement the full/"proper" C3 algorithm here because it has
some failure cases that we'd still like to support. In particular if
we have both `IX : IA, IB` and `IY : IB, IA`, the C3 algorithm says it
is illegal to have `IZ : IX, IY` because the two bases it inherits
from disagree on the relative ordering of `IA` and `IB` in their
own linearizations. Handling such cases may make our implementation
less efficient, and it will also require testing of those corner
caes.
* When it comes time to revamp the implementation of lookup, we will
need to deal with the fact that a single linear list (seemingly)
cannot give us sufficient information to decide which of two members
of the same name should shadow the other, or if there is an ambiguity.
Or rather, it *can* give us that information if we are willing to
accept some very user-unfriendly behavior and simply say that
declarations earlier in the linearization always shadow later
declarations, even if the facets involved are not related by an
inheritance relationship of any kind.
* In order to remove one kind of vicious circularity from the approach,
the linearization that we are computing for `extension` declarations
will not be sufficient for lookups in the body of such an `extension`.
A future change may need to have support for creating and caching
two distinct linearizations for each `extension`: one that is to be
used when that `extension` is pulled into the linearization for a
type that it applies to, and another for when lookup will be performed
in the context of the `extension` itself.
* This change does *not* include the simple expedient of adding a direct
cache for subtype tests to the `SharedSemanticsContext`, although
adding such a cache would be a simple matter.
* This change introduces more deduplication for subtype witnesses,
which should enable more deduplication for other `Val`s (including
`Type`s), but it does not introduce any assumptions that equal
`Val`s or `Type`s must have identical pointer representations.
* Eventually we may find that, similar to the situation with `Type`s,
we will want to have a split between surface-level and canonicalized
versions of other `Val`s, including subtype witnesses.
* Fix clang error.
* remove debugging code.
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Use scratchData on `IRInst` to replace HashSets.
* Update test results.
* Initialize scratchData.
* Update autodiff documentation.
* Use enum instead of bool.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Make DeclRefBase a Val, and DeclRef<T> a helper class.
* Fixes.
* Workaround gcc parser issue.
* Revert NodeOperand change.
* Fix.
* Fix clang incomplete class complains.
* Fix code review.
* Small cleanups and improvements.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Bottleneck DeclRef creation through ASTBuilder.
* Fix clang error.
* Fix.
* Fix.
* More fix.
* Rebase on top of tree.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* MVP for higher order functions
* Add shader subgroup partitioned glsl intrinsics
* Implement parsing and checking for tuple types
Currently there is no way to do anything useful with them from the source language however
* neaten
* Correct precedence of function type parsing
* neaten
* higher order function tests
* function types of any arity
* Inference for higher order functions
* Add second test for unsynchronized params
* regenerate vs projects
* dx11 -> dx12 for saturated cooperations tests
* Disable saturated cooperation tests on vulkan
They fail on release builds in CI, not essential for the higher order function work however
* remove saturated-cooperation tests
* Remove unnecessary assert and clarify control flow in AddDeclRefOverloadCandidates
* Add Tuple type name mangling
* Use functype keyword to introduce function types
* Add more inference tests for hof
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP lowerCamel Dictionary.
* WIP more lowerCamel fixes for Dictionary.
* Add/Remove/Clear
* GetValue/Contains
* Fix tabs in dictionary.
Count -> getCount
* Fix fields with caps.
* Key -> key
Value -> value
Use m_ for members where appropriate.
Use lowerCamel in linked list.
* Some small fixes/improvements to Dictionary.
* Kick CI.
|
|
stdlib. (#2615)
* Allow array parameters in forward diff.
* Use type canonicalization instead of coersion.
* Reimplement array type.
* Fix.
* Update test case.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add gdb generated files to .gitignore
* Switch to c++17
TODO: Ellie update coding style doc
* WIP mesh shaders
* Add MeshOutputType and mesh output decorations
* Lift array type layout creation out of _createTypeLayout
in preparation for sharing it elsewhere
* Initial pass at GLSL legalization for mesh shaders
* Create output types for builtin mesh outputs
This should be rendered as an out paramter block
* Handle writes to member fields in mesh shader output
* Per primitive output from mesh shaders
* Add mesh shader tests
* Redeclare mesh output builtins
* Remove unused instruction
* Emit explicit mesh output max max size
* Add unimplemented warning for array members in mesh output
* Implement mesh output splitting for GLSL in terms of getSubscriptVal
* Allow HLSL syntax for mesh output modifiers
* Improve error messages for mesh output
* Add test for HLSL style mesh output syntax
* Emit explicit mesh output indices max size
* HLSL generation support for mesh shaders
* Better errors for mesh shader misuse
* Neaten comments
* Regenerate vs2019 project files
* Fix build on vs2019
* Retreat on c++17
Will make the change in a separate PR
* slang-glslang binary dep 11.10.0 -> 11.12.0-32
* Fixes for msvc compiler
* Update msvc project
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
* Modified the new type system to support generic differentiable types and added support for differentiating overloaded functions.
* Changed a few asserts to release asserts to avoid unreferenced variable errors
* Fixed a naming issue with TypeWitnessBreadcumb::Flavor::Decl
* Added logic to avoid tracking differentiable types if the module does not use auto-diff or define differentiable types.
* Moved the auto-diff passes to after the specialization step, added a more complex generics test
* Added a generics stress test and fixed AST-side logic. IR side needs some more work
* Added differential getter and setter logic, fixed multiple issues with DifferentiableTypeDictionary, added support for loops and conditions
* Changed differential getters to use pointer types, added getter type checking
* Fixed some bugs related to diff type registration and differential getters
* Removed some superfluous code
* Removed some more unused code.
* Fixed an issue with witness substitution
* Minor fix
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* wip: dedup AST type nodes and cache lookup.
* Fix.
* Remove profiling.
* Fixes.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add `none` literal that is convertible to `Optional`.
* Fix cpu code gen.
* Include vk and cpu test for is-as operator test.
* Inline comparison operators.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* `is` and `as` operator and `Optional<T>`.
* Fix.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Merge slang-ir-diff-jvp.cpp
* Added support and tests for other float vector types
* Added swizzle test and code to handle it (tests failing currently)
* Fixed one test, the other is still pending
* Fixed instruction cloning logic to avoid modifying original function
* Fixed an issue with custom 'pow_jvp' and added support for vector contructor
* Minor update to comments
* Fixed support for division
* Fixed an issue with uninitialized diagnostic sink
* Moved derivative processing to after mandatory inlining.
Skip instructions that don't have side-effects and aren't used by anything.
* WIP: Handling unconditional control flow and multi-block functions
* Support for unconditional multi-block functions
* Added a dead code elimination step to the derivative pass
* Changed name of 'hasNoSideEffects()'
* Refactored variable names
* Added initial IR defs for new type system
* Added necessary logic for semantic checking
* Overhauled type system to use builtin pair types and conform to the IDifferentiable interface
* Automatically replace IRDifferentiablePairType to a custom IRStructType
* Added generics handling by expanding the conformance context functionality and allowing for type parameters
* Minor fix: early return in processPairTypes()
* Minor fixes to differentiable resolution on generic types
* Added new instructions for differential pairs. Basic tests work now.
Looking into generic types.
* Adjusted most tests to the new type system. OutType and InOutType are still not properly working.
* Updated __jvp to produce both primal and differential output
* Moved autodiff related declarations to diff.meta.slang
* Refactored variable names
* Added initial IR defs for new type system
* Added necessary logic for semantic checking
* Overhauled type system to use builtin pair types and conform to the IDifferentiable interface
* Automatically replace IRDifferentiablePairType to a custom IRStructType
* Added generics handling by expanding the conformance context functionality and allowing for type parameters
* Minor fix: early return in processPairTypes()
* Minor fixes to differentiable resolution on generic types
* Added new instructions for differential pairs. Basic tests work now.
Looking into generic types.
* Adjusted most tests to the new type system. OutType and InOutType are still not properly working.
* Updated __jvp to produce both primal and differential output
* Moved autodiff related declarations to diff.meta.slang
* Removed external changes
* Cleanup the transcription logic: each case returns a pair of insts for the primal and differential computation.
|
|
* Clean up `IRReturnVoid`.
* Update gitignore.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Use TerminatedUnownedStringSlice for literals in output C++.
* Remove Escape/Unescape functions used in slang-token-reader.cpp
Add target type of 'host-cpp' etc to map to the target types.
* Fix some corner cases around string encoding.
* Added unit test for string escaping.
Fixed some assorted escaping bugs.
* Updated test output.
* Added decode test.
* Stop using hex output, to get around 'greedy' aspect. Use octal instead.
|
|
* Support `[DllImport]`
* Fix.
* Fix.
* Fix array type emit in cpp.
* Fix.
* Fix.
* Fix
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
Read/write resource types (what D3D/HLSL often refer to as UAVs) can be broadly categorized based on whether they require an underlying format (e.g., a `DXGI_FORMAT`) for reads, or not. D3D refers to the ones that require a format as "typed" UAVs (even though a `RWStructuredBuffer<MyData>` is clearly "typed" at the HLSL level). Vulkan refers to these cases as "storage images" and "storage texel buffers."
Under the D3D model, an application does not have to specify the exact format for a formatted/"typed" UAV in order for loads to work, but it *does* need to specify if an HLSL resource with a declared `float` or vector-of-`float` element type will be backed by data with a `*_UNORM` or `*_SNORM` format. This is where the `unorm` and `snorm` type modifiers come in.
Superficially, it might seem that adding this feature to the Slang compiler is "just" a matter of adding the two modifiers, which is easily done with a pair of one-line `syntax` declarations in `core.meta.slang` plus the corresponding AST node types.
Unfortunately the superficial view misses the detail that, to date, Slang has not had any support for *type modifiers* at all, and has only supported *declaration modifiers*. The distinction has so far not mattered, even with modifiers like `const` because, e.g., the difference between a "`const` array of `float`" and an "array of `const float`" doesn't really matter.
So, adding these two modifiers required introducing a lot of infrastructure along the way. Let's walk through what needed to happen:
* As described above, the actual `syntax` was added easily in the Slang stdlib
* I added a new subclass of `Modifier` for `TypeModifier`s in the AST, and added the AST nodes for `unorm` and `snorm` as subclasses of that.
* In order to syntactically support modifiers applied to types (e.g., `unorm float`), I needed to add a `ModifiedTypeExpr` subclass of `Expr` that represents a base type expression with one or more modifiers applied
* The parser needed some subtle new logic. There are two main cases where type modifiers will come up:
1. In contexts where we might be parsing a declaration (e.g., `const unorm float a`), we need to support a list of modifiers that might freely mix type modifiers and "declaration modifiers" which are not intended to apply to types. In this case we need to split the lis tof modifiers into the type-related ones and the declaration-related ones, and attach each subset to the appropriate place. This is very important for features like C-style pointers, where in `static const float* a;`, the `static` modifier applies to the entire declaration of `a`, but the `const` modifier *only* applies to the `float` type specifier, and *not* to the outer pointer type (the actual type of `a`).
2. In contexts where we are not parsing a declaration (e.g., a generic type argument), we need to support a list of modifiers and appy them *all* to the type specifier being parsed, even if some of them might not be appropriate.
* While working in the parser I implemented a certain amount of unrelated cleanup for code that was using raw `Modifier*`s to represent lists of modifiers, instead of the purpose-built `Modifiers` type.
* The `_parseGenericArg` case needed specific work, because it is an important case in the grammar where we need to parse *either* a type expression or a value exprssion, but cannot easily predict which we will see. The fix implemented for now is to always try to parse modifiers and, if we see any, to assume we are in the type case. Because of the rules for how modifiers in a C-like language inhere to the type specifier (and not necessarily the entire type), we need to refactor some of the type expression parsing routines to support parsing a "suffix" of a type expression.
* Note: I decided to be conservative and only make these changes in `_parseGenericArg` because that is place that is *needed* in order for user code with `unorm`/`snorm` to work, but in practice a user could still confuse our parser by using type modifiers as part of a cast (e.g., `x = (unorm float)y;`). While there is currently no reason why a user should want to do this, it *does* suggest that we need to be prepared to see type modifiers in other ambiguous "expression or type?" contexts. We have so far preferred to avoid looking up built-in syntax declarations like modifiers in expression contexts, because we want to allow users to create variable names that might conflict with some of the more surprising modifier keywords in HLSL (e.g., both `triangle` and `sample` are modifier keyword). A nuanced strategy may be required when we get around to closing this gap (which will be needed around when we want full pointer support, since a cast like `(const SomeType*)somePtr` is pretty common).
* In semantic checking, we now need a `visitModifiedTypeExpr`, which visits the base expression to produce a `Type` and then checks each of the `Modifier`s attached to it. During this process we need to translate the AST-level `Modifier`s into something that can exist properly in the universe of `Type`s. We introduce a `ModifiedType` subclass of `Type`, distinct from the `ModifiedTypeExpr` subclass of `Expr`. Furthermore, we introduce a `ModifierVal` subclass of `Val`, distinct from `Modifier`/`TypeModifier`.
* One unfortunate thing here is that it means we have both, e.g., `UNormModifier` to represent the parsed syntax, and `UNormModifierVal` to represent the `Type`/`Val`-level representation of the same concept. It is quite likely that we are near the point where we can/should consider having two distinct AST representations: one for freshly-parsed ASTs and one for semantically-checked ASTs. The `Type`/`Val` hierarchy clearly belongs to the latter.
* No actual semantic checking is currently being applied to the `unorm` and `snorm` modifiers, although we should in principle check that they are only being applied to `float` and vector-of-`float` types.
* In an attempt to simplify some of the creation logic and build a tiny bit of reusable infrastructure, I went ahead and added the skeleton of a dedupe-caching system in `ASTBuilder` so that we can easily ensure only a single `UNormModifierVal` and a single `SNormModifierVal` ever get created inside the scope of a single builder.
* TODO: Thinking about this, I'm now worried the deduplication does not mean I can make the simplifications I currently do in semantic checking by assuming that any two `UNormModifierVal`s will be pointer-identical. This is because we do not currently (IIRC) have the required "bottleneck" in the compiler where all ASTs get serialized after initial checking, and then deserialized when `import`ed into a downstream module, so that every AST node during a checking step comes from a single `ASTBuilder`. Hmm...
* If we can rely on deduplication to do its thing, then the `Val` and `Type` implementations of modifiers can be relatively simple.
* TODO: One issue here is that the equality comparison for `ModifiedType` currently checks for the same base type and the same modifiers in the same order. This works for now when we only have a small number of type modifiers and any given type will hae at most one, but in the longer run it relies on us to implement some kind of canonicalization scheme, which would both ensure that between `Modified(T, {A, B})` and `Modified(T, {B, A})` only one is allowed (that is, a canonical ordering on modifiers), and that we do not allow `Modified(Modified(T, {A}), {B})`.
* TODO: One other issues is that the `ModifiedType` case does not currently interact correctly with the `as()`-based casting for types (whereas that operation *does* interact in a semantically-correct fashion with `typedef`s). Fixing this issue in a robust way really depends on us re-architecting the `Type` system so that *any* `Type` can have modifiers attached, with modifiers affecting type identity/deduplication.
* The key place where `ModifiedType` creates a complication in semantic checking is type conversion/coercion. A user is likely to declare a `RWTexture2D<unorm float>`, fetch from it (producing a value of type `unorm float`) and then assign the result to a `float` variable, prompting for a conversion from `unorm float` to `float` (because they are distinct `Type`s).
* We handle this case in the core `_coerce()` operation by checking if either `toType` or `fromType` is a `ModifiedType`. If *either* one is a modified type, we apply logic to check for modifiers that are present on one and not the other. Basically we check which modifiers need to be "dropped" and which need to be "added" during conversion, and validate that these modifiers *can* be dropped/added without creating a semantic error. The only type modifiers we support right now *can* be dropped/added like this, so we are fine.
* TODO: When we add more complete pointer support, we could need logic here to validate when casts between, e.g., `const int*` and `int*` should/shouldn't be allowed.
* Note: Even opening the door to type modifiers at all creates the same kind of challenges for user-defined generic types (and functions!) since `MyType<int>` and `MyType<const int>` are distinct instantiations in a future where we support `const` as a type modifier. We *may* need to plan to restrict where modified types can be used, so that certain built-in generic types support modified types as arguments, but user-defined types don't (or at least might need to opt-in to get support).
* The result of a `_coerce()` that drops/adds modifiers is a `ModifierCastExpr`, which is a kind of no-op AST node that merely expresses that the conversion is allowed and valid.
* In IR lowering we currently do the simple thing and translate a `ModifiedType` to a distinct IR node called `AttributedType`.
* The change in terminology from "modifier" to "attribute" is to follow the way that these kinds of modifiers best map to the `IRAttr` case in the IR (rather than the `IRDecoration` case). We probably ought to do a careful terminology scrub here, because having this terminology mismatch between IR and AST could be a source of confusion.
* TODO: In principle, using `IRAttributedType` creates the same basic problems as using `ModifiedType`: code that is usin `as()` or similar operations to check for a specific subclass of `IRType` may not see the case they were looking for due to use of `IRAttributedType`.
* Initially I had hoped to avoid the problem by having the `IRAttr`s be attached directly as operands to an otherwise-ordinary `IRType`. E.g., a lowered `unorm float4` would be an `IRVectorType` with an "extra" operand that is an `IRUNormAttr`, something like: `Vector<Float, 4, UNorm>`. This sounds great (and looks great!), but runs into the problem that it is incompatible with the way we currently represent things like generic type parameters. A generic type parameter `T` is represented as an `IRParam`, and it does *not* make sense to have an additional `IRParam` to represent `const T` or `unorm T`, etc.
* The Right Way to solve this stuff at both the AST and IR levels is to avoid passing around bare `Type*` or `IRType*` in general, and instead use a value type that implements the needed policy more directly: something like a `TypeHolder` or `IRTypeHolder` (placeholder name). The `*Holder` type would abstract over the various "wrapper" nodes required to store all the additional data like attributes but, importantly, would *not* allow that extra information to be dropped or lost during operations like casting (e.g., note how the current `Type` implementation of `as()` loses information on `typedef` names, making our error messages slightly worse). This is actually quite similar to how we currently use the `DeclRef<T>` system to allow working with what is *usually* a `T*` under the hood, but in a way that ensures we don't lose track of any generic substitution information.
* During C-like code emit we have a process that turns an `IRType` into a chain of declarators as needed to emit a C-like declaration with pointers, arrays, etc. The `IRAttributedType` case needs to get folded into this logic. Basically, when we see an `IRAttributedType` we immediately emit any modifiers that are required to be in a prefix position, then recursively emit the underlying type with an extra layer of declarator that tracks the modifiers, so that we can emit any modifiers that should be placed in a postfix position *after* the type. As a specific example, our C/C++ back-end would want to use the postifx option to handle `const`, because then it can properly emit stuff like `int const * const *` and not the incorrect `const const int**`.
* The HLSL emit logic overrides the prefix case for handling type attributes, and uses it to emit `unorm` and `snorm` where they occur.
* One unfortunate detail is that (apparently) some downstream HLSL compilers do not allow the `unorm`/`snorm` modifiers to apply to `vector<float, *>` types, even though that should be semantically valid. Instead, they only support `float`, `float2`, `float3`, and `float4` explicitly. To work around this issue, we go ahead and change our HLSL emit logic so that when we encountered 1-to-4 component vectors of `float`, `int`, or `uint` we emit the type name using the typical HLSL shorthand. This is actually a signficicant change in our HLSL output, but it both seemed like a good fix to have anyway, and was also the only obvious way to address the downstream parser shortcomings without a massive kludge.
* As a result of this change the `half-texture.slang` test broke, since it was using raw HLSL as the expected output. I changed the test to do a DXIL comparison instead, which is our preferred way of testing cross-compilation behavior (since it is more robust in the face of small changes to our source output).
|
|
* Properly fill `declref` in `Linkage::getContainerType`.
* Fix timestamp query on cpu
* Fix typo.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
The basic feature here is the ability to use the `&` operator to produce the conjunction/intersection of two interfaces. That is, you can have interfaces:
interface IFirst { int getFirst(); }
interface ISecond { int getSecoond(); }
and if you need a generic function where the type parameter `T` must conform to *both* of these interfaces, you express that by constraining the parameter to the intersection of the interfaces:
void someFunction<T : IFirst & ISecond>(T value) { ... }
Without this feature, the main alternative an application would have is to define an intermediate interface, like:
interface IBoth : IFirst, ISecond {}
Forcing users to deal with an intermediate interface creates more work for type authors (they need to remember to inherit from the right combined interface(s)), or for `extension` authors (when you add `ISecond` to a type that used to just support `IFirst`, you had better also add `IBoth`). In the worst case, a family of N related "leaf" interfaces would give rise to an exponential number of intermediate interfaces to represnt the possible combinations.
A conjunction like `IFirst & ISecond` is officially its own type, and can be used to declare a type alias:
typealias IBoth = IFirst & ISecond;
This change only includes the first pass of work on this feature, so there are several caveats to be aware of:
* Using a conjunction as part of an inheritance clause is not yet supported (e.g., `struct X : IFirst & ISecond`). This is true even if the conjunction was introduced by an intermediate `typealias`
* The `&` syntax introduced here is only parsed in places where only a type (not an expression) is possible. This means you cannot do things like cast to a conjunction with `(IFirst & ISecond)(someValue)`.
* This work *should* apply to conjunctions of more than two interfaces (like `IA & IB & IC`) but that has not yet been tested
* In the long run it may be sensible to allow conjunctions that use concrete types, but we really ought to have the semantic checking logic rule that out for now.
* During testing, I encountered compiler crashes when trying to use this feature together with `property` declarations. Further investigation and debugging is called for.
* The handling of conjunction types is currently incomplete, in that there are many equivalences the compiler does not yet understand. For example, it is clear that `IA & IB` is equivalent to `IB & IA`, but the compiler currently does not understand this and will treat them as different types. A deeper implementation approach is called for.
* Conjunctions are currently only supported for generic type parameter constraints, when performing full specialization. Use of conjunctions for existential-type value parameters or with dynamic dispatch is not yet supported.
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Fix handling of access modifiers inside type definition.
* Fix access problem for AST node.
Make dumping produce a single function with switch, to potentially make available without Dump specific access.
* WIP on serialization design doc.
* Remove project references to previously generated files.
* More docs on serialization design.
* Improve serialization documentation.
Remove unused function from IRSerialReader.
* Small fixes around naming. Remove long comment from slang-serialize.h - as covered in serialization.md
* Remove long comment in slang-serialize.h as covered in serialization.md
* More information about doing replacements on read for AST and problems surrounding.
* Typo fix.
* Spelling fixes.
* Value serialize.
* Value types with inheritence.
* Use value reflection serial conversion for more AST types
* Use automatic serialization on more of AST.
* Get the types via decltype, simplifies what the extractor has to do.
* Update the serialization.md for the value serialization.
* Small doc improvements.
* Update project.
* Remove ImportExternalDecl type
Added addImportSymbol and ImportSymbol type
Fixed bug in container which meant it wouldn't read back AST module
* Because of change of how imports and handled, store objects as SerialPointers.
* First pass symbol lookup from mangled names.
* Cache current module looked up from mangled name.
* Fix SourceLoc bug.
Improve comments.
* Added diagnostic on mangled symbol not being found
* Fix typo.
* WIP serializing stdlib.
* WIP serializing stdlib in.
* Fix problem serializing arrays that hold data that is already serialized.
* Remove clash of names in MagicTypeModifier.
* Make conversion from char to String explicit.
Fix reference count issue with SerialReader.
* Add code to save/load stdlib.
* Use return code to avoid warning - SerialContainerUtil::write(module, options, &stream))
* Make all String numeric ctors explicit.
Added isChar to UnownedStringSlice.
Added operator== for UnownedStringSlice to String to avoid need to convert to String and allocate.
* Add error check to readAllText.
* tabs -> spaces on String.h
* tab -> spaces String.cpp
* Remove msg for StringBuilder, just build inplace for exceptions.
* Check SerialClasses - for name clashes.
Renamed Modifier::name as Modifier::keywordName
* Handling of extensions when deserializing AST - updating the moduleDecl->mapTypeToCandidateExtensions
Co-authored-by: Tim Foley <tim.foley.is@gmail.com>
|
|
* First pass at generalizing serializer.
* Split out ReflectClassInfo
* Use the general ReflectClassInfo
* Fix some typos in debug generalized serialization.
* Add calculation of classIds.
Make distinct addCopy/add on SerialClasses.
* Write up of more generalized serialization
* WIP to transition from ASTSerialReader/Writer etc to generalized SerialReader/Writer and associated types.
* Improvements to SerialExtraObjects.
Keep RefObjects in scope in factory
* Compiles with Serial refactor - doesn't quite work yet.
* First pass serialization appears to work with refector.
* Split out type info for general slang types.
* Split out slang-serialize-misc-type-info.h
* DebugSerialData -> SerialSourecLocData
DebugSerialReader -> SerialSourceLocReader
DebugSerialWriter -> SerialSourceLocWriter
* Remove unused template that only compiles on VS.
* Fix warning around unused function on non-VS.
|
