| Age | Commit message (Collapse) | Author |
|---|
|
* Split bwd_diff op into separate ops for primal and propagate func.
* Fix.
* Download swiftshader with github actions instead of curl on linux.
* Fix github action.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Fix issues around dynamic generic function and autodiff.
* Fix return type issue.
* Fix type unification for generic `inout` parameter.
* Fix.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Autodiff through simple dynamic dispatch.
* Revert changes.
* Fix.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
functions (#2505)
* Added non-differentiable call test
* Extended testing for nondifferentiable calls
* Fixed subtle issue with extensions on generic types not applying the correct substitutions, leading to unspecialized generics at the emit stage
* More fixes. inout struct params now work fine
* Update inout-struct-parameters-jvp.slang
* Update slang-ir.cpp
* Fixed hoisting lookup_interface_method
* Fixed non-diff call return value
* Fixed issue with phi nodes
* Fixed problem with IRSpecialize preventing hoisitng of DifferentialPairType
* Fixed non-diff call test to conform to the new 'no_diff' system
|
|
|
|
* Clean up type checking of higher order expressions.
* Replace `goto` with `break` to pacify clang.
* Fix.
* Fixes.
* Fix more tests.
* Fix lowerWitnessTable parameter error.
* Exclude attributes from ast printing.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add gdb generated files to .gitignore
* Switch to c++17
TODO: Ellie update coding style doc
* WIP mesh shaders
* Add MeshOutputType and mesh output decorations
* Lift array type layout creation out of _createTypeLayout
in preparation for sharing it elsewhere
* Initial pass at GLSL legalization for mesh shaders
* Create output types for builtin mesh outputs
This should be rendered as an out paramter block
* Handle writes to member fields in mesh shader output
* Per primitive output from mesh shaders
* Add mesh shader tests
* Redeclare mesh output builtins
* Remove unused instruction
* Emit explicit mesh output max max size
* Add unimplemented warning for array members in mesh output
* Implement mesh output splitting for GLSL in terms of getSubscriptVal
* Allow HLSL syntax for mesh output modifiers
* Improve error messages for mesh output
* Add test for HLSL style mesh output syntax
* Emit explicit mesh output indices max size
* HLSL generation support for mesh shaders
* Better errors for mesh shader misuse
* Neaten comments
* Regenerate vs2019 project files
* Fix build on vs2019
* Retreat on c++17
Will make the change in a separate PR
* slang-glslang binary dep 11.10.0 -> 11.12.0-32
* Fixes for msvc compiler
* Update msvc project
|
|
* Add [ForwardDerivativeOf] attribute.
* Fix handling around phi nodes.
* Fixes.
* Remove IR opcode for ForwardDerivativeOfDecoration.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Rework differential conformance dictionary checking.
* Revert space changes.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Auto synthesis of IDifferntial interface methods.
* Add comments.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
`[DerivativeMember(DiffType.field)]` (#2460)
* wip: remove auto-diff for member access, add diff through property accessors.
* Fix getter-setter test.
* Fix getter-setter-multi test.
* Fix nested-jvp test.
* Use [DerivativeMember] attribute to differentiate through member access.
* Clean up.
* More cleanup.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Modified the new type system to support generic differentiable types and added support for differentiating overloaded functions.
* Changed a few asserts to release asserts to avoid unreferenced variable errors
* Fixed a naming issue with TypeWitnessBreadcumb::Flavor::Decl
* Added logic to avoid tracking differentiable types if the module does not use auto-diff or define differentiable types.
* Moved the auto-diff passes to after the specialization step, added a more complex generics test
* Added a generics stress test and fixed AST-side logic. IR side needs some more work
* Added differential getter and setter logic, fixed multiple issues with DifferentiableTypeDictionary, added support for loops and conditions
* Changed differential getters to use pointer types, added getter type checking
* Fixed some bugs related to diff type registration and differential getters
* Removed some superfluous code
* Removed some more unused code.
* Fixed an issue with witness substitution
* Minor fix
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
A commonly requested feature is to be able to supply only some
of the arguments to a generic explicitly, while allowing the rest
to be inferred. A common example is a function that performs some
kind of conversion:
To convert<To, From>( From fromValue ) { .... }
A user would like to be able to call this operation like:
int i = convert<int>( 1.0f );
but the current Slang type checker requires all or none of the generic
arguments be supplied. Supplying all of the arguments is tedious:
int i = convert<int, float>( 1.0f );
In this case, the `float` type argument is redundant and could be
inferred from context. However, if the user tries to omit the generic
argument list:
int i = convert( 1.0f );
The current type-checker cannot infer the `int` type argument (even if
one might claim it *should* infer based on the desired result type).
This change adds support for the `convert<int>(...)` case, by allowing
a generic to be applied to a prefix of its explicit arguments, and then
inferring the remaining arguments from contextual information when that
"partially applied" generic is applied to value-level arguments.
Most of the changes are just plumbing: adding the notion of a partially
applied generic and then supporting them during overload resolution.
A single test case is included that covers the `convert`-style use case.
It is likely that more testing is needed to cover failure modes of this
feature.
|
|
* Language feature: pointer sized int types.
* Fix.
* small change to test.
* Fix stdlib.
* Fix.
* Fix.
* Add typedef for `size_t` in stdlib.
* Fix test.
* Add `intptr_t::size` constant.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* wip: dedup AST type nodes and cache lookup.
* Fix.
* Remove profiling.
* Fixes.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
(#2388)
|
|
|
|
* Warning on bool to float conversion.
* Fix test cases.
* Improve.
* LanguageServer: don't show constant value for non constant variables.
* Fix tests.
* Fix warnings in tests.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add gfx interface definition in Slang.
- add gfx interface definitons in Slang.
- fix slang compiler to correctly type-check `out` interface argument.
- modify gfx interface to be fully COM compatible
- add convenient ShaderProgram creation methods to gfx.
* Fix compile errors and warnings.
* Update project files
* Fix cuda.
* Properly implement queryInterface in command encoder impls.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Support `class` types.
* Ignore class-keyword test
* Fix codereview comments and warnings.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
|
|
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Use TerminatedUnownedStringSlice for literals in output C++.
* Remove Escape/Unescape functions used in slang-token-reader.cpp
Add target type of 'host-cpp' etc to map to the target types.
* Fix some corner cases around string encoding.
* Added unit test for string escaping.
Fixed some assorted escaping bugs.
* Updated test output.
* Added decode test.
* Stop using hex output, to get around 'greedy' aspect. Use octal instead.
* Added HostHostCallable
Small changes to use ArtifactDesc/Info instead of large switches.
* Fix C++ emit to handle arbitrary function export.
* Add options handling for callable without an output being specified.
* Can compile with COM interface. Added example using com interface.
* Use the IR Ptr type instead of hack in C++ emit for interfaces.
* Fix issue with outputting the COM call when ptr is used.
* Fix crash issue on compilation failure.
* Add support for __global.
* Added `ActualGlobalRate`
Added special handling around globals and COM interfaces.
Tested out in cpu-com-example.
* Fix typo in NodeBase.
* Support for accessing globals by name working.
* Check that actual global initialization is working.
* Refactor the com replacement such that it doesn't need a cache or do anything special with GlobalVar.
* Remove context.
Only create replacement if needed.
* Split out COM host-callable into a unit-test.
* host-callable com testing on C++and llvm.
* Comment around the COM ptr replacement.
* Disable com test on vs 32 bit.
Fix C++ prelude
* Disable 32 bit targets testing com host-callable.
* Use JSON parsing to locate VS version.
* Need platform detection in C++prelude.
* Fix com host callable test for LLVM.
* Work around for not being able to include "targetConditionals.h"
|
|
* Major language server features.
* Include slangd in binary release.
* Fix compiler issues.
* Fix compiler error.
* Completion resolve.
* Various improvements.
* Update diagnostic test expected output.
* Bug fix for source locations.
* Adjust diagnostic update frequency.
* Update github actions to store artifacts.
* Fix infinite parser loop.
* Fix parser recovery.
* Fix parser recovery.
* Update test.
* Fix test.
* Disable IR gen for language server.
* Allow commit characters in auto completion.
* Fix lookup for invoke exprs.
* More parser robustness fixes.
* update solution file
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Clean up `IRReturnVoid`.
* Update gitignore.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* New language feature: basic error handling.
* Fix.
* Fix `tryCall` encoding according to code review.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
The problematic case is when an `interface` has a `[mutating]` method:
interface ICounter
{
[mutating] void increment();
}
and code tries to invoke that method on a value of existential type:
ICounter c = ...;
c.increment();
We know that the existential value `c` is conceptually a tuple of:
* A concrete type `X`
* A witness that `X : ICounter`
* A value `v` of type `X`
We simply want to invoke `increment()` on the `v` part, using the `X : ICounter` witness table.
The catch that the compiler faces is that the variable `c` is mutable, so we need to be careful that we "snapshot" its value (the tuple `X, X:ICounter, v`) at a single point.
The snapshotting behavior is important when invoking a method that involves `This` or associated types in its signature, so we cannot get rid of it.
The snapshotting we do relies on the idea of a `LetExpr` AST node, which cannot be written in the input syntax.
A `LetExpr` introduces a variable binding (with an initial-value expression) and then evaluates a body expression in the context of that binding.
For a call site like `c.increment()` the front-end makes an intermediate copy of `c` and then "opens" that immutable value to get at the elements of the tuple `X`, `X : ICounter`, `v`.
The resulting AST after checking looks something like:
ICounter c = ...;
(let tmp = c in extractExistentialValue(tmp)).increment();
In that form it is more clear why the attempt to call `increment()` fails:
1. The binding `tmp` sure looks immutable
2. There is no logic in the compiler to make `extractExistentialValue(x)` be an l-value if `x` is
3. There is seemingly no logic to write back from `tmp` to `c` when the operation completes
Let us walk through those problems in order.
Item (1) turns out to be a bit of a non-issue.
Despite the way that I've written out `let` expressions above, the logic in `moveTemp()` in the compiler actually introduces a *mutable* binding.
Item (2) can be fixed for the purposes of semantic checking by modifying `openExistential()`.
Simplistically, we make the overall expression be an l-value if the operand is.
Item (3) is handled at the level of AST->IR lowering. Each kind of expression that can form an l-value needs to have a way to represent the "location" of that l-value in the `LoweredValInfo` type.
This change adds a case to handle the `extractExistentialVal` operation, by tracking both the extract value (of concrete type) and the underlying l-value (of existential type).
Where all of this comes crashing against reality a bit is that the scoping I've drawn for the `let` expressions above kind of doesn't work once we look at types.
The basic problem is that the *type* of the `(let tmp = c in ...)` expression is the concrete type `X` that was extracted from the existential.
That type can conceptually be written as `ExtractExistentialType(tmp)` which, notably, references `tmp`.
That means that we end up with AST expression nodes that reference the variable `tmp` *outside* of its scope.
Furthermore, those references to `tmp` can end up being lowered to IR *before* we have lowered the `let ...` expression itself.
Fixing the scoping issue turns out to be a major undertaking.
The first (and more obvious) issue is needing to address the scoping problem.
The solution I implemented includes a bit of refactoring to make all the `SemanticsVisitor` types better able to pass around the contextual scope-dependent state that might be needed during semantic checking, but really only adds a single piece of state.
The semantic-checking state used for checking expressions is bottlenecked so that there will (or at least *should*) always be an explicit representation of a "scope" that surrounds a complete expression (as opposed to a sub-expression).
When a `LetExpr` needs to be introduced, it is added to a pending list on the active scope, rather than being added locally.
Once the complete expression is checked, the resulting expression is wrapped up in the pending `LetExpr`s so that their scope is as broad as possible.
Technically this solution doesn't cover all cases. For example:
interface ICell { associatedtype Content; Content getContent(); }
...
ICell cell = ...;
let content = cell.getContent();
In this case the type of `content` refers to the binding introduced by a `LetExpr` in the initial-value expression.
I am leaving such issues as a piece of future work, in the hopes that we can get at least a partial fix for the problem in place.
A future fix probably nees to extend the scoping even wider (e.g., by unwrapping the `LetExpr`s from the initial-value expression and turning them into distinct temporaries).
The second piece of the fix is that we need a way for the modified value of the extracted existential to be "written back" to the original location.
Well...
We are actually being a little slippery here, based on some logic in the compiler codebase that I guess Just Works.
When AST->IR lowering encounters a `LetExpr` that binds an l-value to a name, it actually ends up binding that name more or less as a *reference* to that l-value.
At this point the `let`-ness of `LetExpr` is very much in doubt: the binding can be mutable, and it can even be an *alias* of some location?!?
In any case, the result is that the AST->IR codegen logic implicitly handles the "write-back" because the `let`-bound temporary is actually an alias for the original location.
A more complete future fix might need to introduce a distinct case in `LoweredValInfo` to handle the case of copy of a mutable temporary.
|
|
Co-authored-by: Yong He <yhe@nvidia.com>
Co-authored-by: Theresa Foley <10618364+tangent-vector@users.noreply.github.com>
|
|
* Various vulkan/glsl fixes.
* Fix.
* Fix.
* Canonicalize type constraints for name mangling.
Co-authored-by: Yong He <yhe@nvidia.com>
Co-authored-by: Theresa Foley <10618364+tangent-vector@users.noreply.github.com>
|
|
* Make translation unitts in the same CompileReq visible to `import`.
* Fix code review comments.
Co-authored-by: Yong He <yhe@nvidia.com>
Co-authored-by: Theresa Foley <10618364+tangent-vector@users.noreply.github.com>
|
|
* Add debug symbols for release build.
* Hack to try and capture failing compilation.
* Typo fix for capture hack.
* Specify return type on lambdas.
* Added const.
* Try breakpoint.
* Up count
* Let's capture everything so we can valgrind.
* Disable always writing repros.
* Make Scope non RefCounted.
* Fix issue with not serializing Scope.
* More comments around changes to Scope.
Remove Scope* from serialization.
* Remove code used for testing original issue.
|
|
The `OverloadedExpr` type didn't provide a default value for its field:
Name* name;
This led to a null-pointer crash in the logic that deals with synthesizing interface requirements because it creates an `OverloadedExpr` but doesn't initialize the field.
This change makes two fixes:
1. The logic in the synthesis path actually initializes `name` so that it can feed into any downstream error messages
2. The `OverloadedExpr` declaration now includes an initial value for `name` so that it will at least be null instead of garbage if we slip up again
|
|
This change originally started with the simple goal of allowing generic functions with default argument values on their parameters to work:
```
void someFunction<T>(T value, int optional = 0);
```
The core problem there was that the compiler code was (correctly) anticipate the case where the default argument value for a parameter depends on a generic parameter, such as:
```
interface IDefaultable { static This getDefault(); }
void anotherFunction<T : IDefaultable>(T first, T second = T.getDefault());
```
Supporting this latter case requires some kind of ability to apply subsitutions to an `Expr`, but our compiler logic simply errored out in that case. The first major fix that went into this change was to add a new `SubstExpr<T>` type that behaves a lot like `DeclRef<T>` in that it stores a `T*` plus a set of substititions that need to be applied to it.
In addition, it was found that even if `anotherFunction<ConcreteType>(...)` might work, when generic argument inference was used for just `anotherFunction(...)` would fail because it includes a strict match on the number of arguments/parameters in the call expression.
The next problem that arose was that the test I'd created used an interace with an `__init` requirement, and it appeared that our code generation didn't work for that case:
```
interface IStuff { __init(int val); }
void f<T : IStuff>(T x = T(0));
```
In this case, the `T(0)` initialization would get compiled to `(ConcreteType) 0` in the output rather than calling the function generated for the `__init` inside `ConcreteType`. The basic problem there was a bit of crufty old logic we have in place to work around the large number of `__init` declarations in the stdlib that don't have proper `__intrinsic_op` modifiers on them. We really need to fix the underlying problem there, but I worked around it by having the IR lowering pass only do its workaround magic on stdlib declarations.
The next problem down this line was that my test had two different `__init` declarations in the concrete type and the logic for checking interface conformance was picking the wrong one to satisfying an interface requirement despite it being obviously wrong (not even the right number of parameter).
This last problem led me down the rabbit-hole of trying to actually get our semantic checking for interface requirements right. There were a few pieces to that work:
* Actually checking that the parameter and result types for two callables match is the simple part. If that was all that would be required we would have implement this logic a long time ago.
* Next we have to deal with functions that make use of the `This` type, associated types, etc. We have to know that when the interface uses `This`, we want to treat that as equivalent to `ConcreteType`, and similarly for associated types. Getting that working is mostly a matter of setting up a this-type subsitution for the interface member being checked.
* Finally, when comparing generic declarations like `IBase::doThing<T>` and `Derived::doThing<U>` we need to deal with the way that `T` and `U` represent the "same" logical type parameter, but are distinct `Decl`s. This is handled by specializing the base declaration to the parameters of the derived one (e.g., forming `IBase::doThing<U>` using the `U` from `Derived::doThing`).
The result seems to be passing our tests, but there are still a few gotchas lurking, I'm sure.
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP: First pass in supporting output of line error information.
* Add support for lexing to better be able to indicate SourceLocation information.
* Fix lexer usage in DiagnosticSink in C++ extractor.
* Update diagnostics tests to have line location info.
* Fixed test expected output that now have source location information in them.
* Better handling of tab.
* Fix test expected results for tabbing change.
* DiagnosticLexer -> DiagnosticSink::SourceLocationLexer
Added line continuation tests.
* Fix typo.
* Added String::appendRepeatedChar
* Change to rerun tests.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
|
* Fix constant folding in attributes
* remove unnecessary change
* remove unnecessary change
* remove unnecessary change
* Fixed circular checking issue.
* cleanup
* more cleanup
* minimize diff
* minimize diff
* minimize diff
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Fix handling of access modifiers inside type definition.
* Fix access problem for AST node.
Make dumping produce a single function with switch, to potentially make available without Dump specific access.
* WIP on serialization design doc.
* Remove project references to previously generated files.
* More docs on serialization design.
* Improve serialization documentation.
Remove unused function from IRSerialReader.
* Small fixes around naming. Remove long comment from slang-serialize.h - as covered in serialization.md
* Remove long comment in slang-serialize.h as covered in serialization.md
* More information about doing replacements on read for AST and problems surrounding.
* Typo fix.
* Spelling fixes.
* Value serialize.
* Value types with inheritence.
* Use value reflection serial conversion for more AST types
* Use automatic serialization on more of AST.
* Get the types via decltype, simplifies what the extractor has to do.
* Update the serialization.md for the value serialization.
* Small doc improvements.
* Update project.
* Remove ImportExternalDecl type
Added addImportSymbol and ImportSymbol type
Fixed bug in container which meant it wouldn't read back AST module
* Because of change of how imports and handled, store objects as SerialPointers.
* First pass symbol lookup from mangled names.
* Cache current module looked up from mangled name.
* Fix SourceLoc bug.
Improve comments.
* Added diagnostic on mangled symbol not being found
* Fix typo.
* WIP serializing stdlib.
* WIP serializing stdlib in.
* Fix problem serializing arrays that hold data that is already serialized.
* Remove clash of names in MagicTypeModifier.
* Make conversion from char to String explicit.
Fix reference count issue with SerialReader.
* Add code to save/load stdlib.
* Use return code to avoid warning - SerialContainerUtil::write(module, options, &stream))
* Make all String numeric ctors explicit.
Added isChar to UnownedStringSlice.
Added operator== for UnownedStringSlice to String to avoid need to convert to String and allocate.
* Add error check to readAllText.
* tabs -> spaces on String.h
* tab -> spaces String.cpp
* Remove msg for StringBuilder, just build inplace for exceptions.
* Check SerialClasses - for name clashes.
Renamed Modifier::name as Modifier::keywordName
* Handling of extensions when deserializing AST - updating the moduleDecl->mapTypeToCandidateExtensions
Co-authored-by: Tim Foley <tim.foley.is@gmail.com>
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Fix handling of access modifiers inside type definition.
* Fix access problem for AST node.
Make dumping produce a single function with switch, to potentially make available without Dump specific access.
* WIP on serialization design doc.
* Remove project references to previously generated files.
* More docs on serialization design.
* Improve serialization documentation.
Remove unused function from IRSerialReader.
* Small fixes around naming. Remove long comment from slang-serialize.h - as covered in serialization.md
* Remove long comment in slang-serialize.h as covered in serialization.md
* More information about doing replacements on read for AST and problems surrounding.
* Typo fix.
* Spelling fixes.
* Value serialize.
* Value types with inheritence.
* Use value reflection serial conversion for more AST types
* Use automatic serialization on more of AST.
* Get the types via decltype, simplifies what the extractor has to do.
* Update the serialization.md for the value serialization.
* Small doc improvements.
* Update project.
* Remove ImportExternalDecl type
Added addImportSymbol and ImportSymbol type
Fixed bug in container which meant it wouldn't read back AST module
* Because of change of how imports and handled, store objects as SerialPointers.
* First pass symbol lookup from mangled names.
* Cache current module looked up from mangled name.
* Fix SourceLoc bug.
Improve comments.
* Added diagnostic on mangled symbol not being found
* Fix typo.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
|
over (#1580)
* #include an absolute path didn't work - because paths were taken to always be relative.
* Access the members iteration in _ensureAllDeclsRec via indices to avoid a change in the array invalidating the list.
* Fix another iterator of members in SemanticVisitor
* Slight improvements to comments - main purpose is to kick a new build.
|
|
In some cases, functionality is available as either a GLSL extension for Vulkan/SPIR-V, or through the NVAPI system for D3D. This situation creates complications because while GLSL extensions are generally all supported by the open-source glslang compiler (which we can bundle and ship), NVAPI operations are exposed through a specific header (`nvHLSLExtns.h`) that ships as part of the NVAPI SDK.
When a user wants to explicitly use NVAPI-provided operations in their shader code, there are no major complications for Slang; the user sets up their include paths, `#include`s the relevant header, calls functions in it, and lets Slang deal with the details of compilation.
The challenge for Slang arises when we want to provide a cross-platform interface in our standard library (e.g., the `RWByteAddressBuffer.InterlockedAddF32` method that was recently added) that uses either a GLSL extension (when compiling for Vulkan/SPIR-V) or an NVAPI (when compiling to DXBC or DXIL). In that case, the code *generated* by Slang now has a dependency on NVAPI, and we need to somehow emit a `#include` directive that pulls it in when invoking fxc or dxc. Because we do not (and seemingly cannot) bundle the NVAPI header with the compiler, we have to rely on ther user to have it available and to somehow communicate to Slang where it is.
Exposing portable routines that sometimes use NVAPI currently creates two main challenges:
1. The user is forced to interact with the "prelude" mechanism in the compiler, which allows the programmer to define code in a given target language that gets prepended to the Slang-generated code. While the prelude mechanism is powerful, it is also hard for users to integrate into their workflow, and our experience so far is that users want something that Just Works.
2. If the user writes code that uses some of our abstract operations that layer on NVAPI *and* they also want to use NVAPI explicitly, they end up with two copies of the NVAPI header (one included by the Slang front-end, and another included by the downstream fxc/dxc compiler). This puts the user in the situation of (a) having to ensure that they set the defines like `NV_SHADER_EXTN_SLOT` consistently both when invoking Slang and when adding their prelude, and (b) even if they do make the definitions consistent, they run into the problem that fxc/dxc complain about overlapping register bindings on the two copies of the `g_NvidiaExt` global shader paraemter that the NVAPI header declares.
This change attempts to resolve both issues by adding a lot of "do what I mean" logic to the compiler to try to ease things in the common case. In particular:
1. The user no longer needs to use the "prelude" mechanism when using NVAPI. The compiler now embeds a default prelude for HLSL output, which will `#include` the NVAPI header if and only if the generated code needs NVAPI access because of portable standard library routines that were used.
2. The user can mix-and-match explicit NVAPI use and stdlib functions that compile to use NVAPI. The register/space to be used by NVAPI when included via prelude is now set based on whatever the user set via the preprocessor so that it should automatically be consistent between both cases. Furthermore, the code we emit for the declaration of `g_NvidiaExt` when compiling explicit NVAPI use is set up to be conditional, so that it is skipped in the case where the prelude will pull in its own declaration of that parameter.
The way all this is achieved involves a lot of moving pieces:
* We now have an HLSL prelude, which mostly just serves to `#include "nvHLSLExtns.h"` in the case where NVAPI support is needed downstream.
* Standard library operations that require NVAPI for their implementation on HLSL include a new `[__requiresNVAPI]` attribute.
* The preprocessor has been extended so that after tokenizing an input file it looks up the NVAPI-relevant macros in the resulting environment, and if they are set it attached a modifier (`NVAPISlotModifier1) to the AST `ModuleDecl` that is based on their values. Logic is added to detect if multiple input files specify values for the macros in ways that conflict.
* The semantic checking step is extended so that it detects the "magic" NVAPI declarations (the `g_NvidiaExt` paramter and the `NvShaderExtnStruct` type that it uses) and attaches a modifier to them so that they can be identified as such in later steps.
* Parameter binding is extended to collect a list of the AST modifiers that reflect NVAPI binding, and to reserve the relevant register(s) so that ordinary user-defined parameters cannot conflict with them.
* IR lowering translates the three new AST modifiers related to NVAPI over to IR equivalents.
* IR linking is extended to make sure that it clones any `IRNVAPISlotDecoration`s attached to the input modules. The pass intentionally does not care where the modifiers came from; it just collects them all and leaves it to downstream code to sort out what they mean.
* Emit logic is extended to have a notion of "prelude directives" which are preprocessor directives that should come *before* the prelude in the generated code, because they can impact the way that the prelude compiles. This is done so that we don't have to introduce ad hoc logic for each downstream compiler to set any relevant `-D` flags (e.g., both fxc and dxc would need to duplicate such logic for NVAPI support).
* The HLSL source emitter is extended to track whether it emits any operations that require NVAPI support.
* The HLSL source emitter is extended to emit prelude directives based on whether NVAPI is needed and, if it is, to also set the register and space that NVAPI should use based on what was stored in the decoration(s) on the IR module.
* The HLSL source emitter is extended so that it detects global instructions that represent "magic" NVAPI constructs , and emit them as conditional definitions so that they are skipped when NVAPI is included via the prelude.
* The handling of requires capabilities during emit logic was cleaned up a bit so that more logic is shared across targets, and also so that the same logic is used both when emitting a function declaration/definition and when emitting a call to an instrinsic function (which won't get declared/defined).
|
|
* First pass at filter for AST serial writing.
* Serialization of AST for modules.
* Removed some commented out source.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
|
Most people agree that it is a Good Thing when compilers are deterministic: the exact same input bits produce the exact same output bits every time the compiler is run. Bonus points are awarded if the results are independent of the platform the compiler was compiled for and run on.
One of the easiest kinds of nondeterminism to have sneak into a compiler is for it to produce the "same" code inside functions, but sometimes emits functions or other global symbols in a different order from run to run. Right now, the Slang compiler has some of this kind of nondeterminism.
The main way (but not necessarily the only way) that a compiler ends up producing output with a different ordering across runs is by iterating over the contents of a hash-based container (in our codebase, a `Dictionary` or `HashSet`), where the keys make use of pointers. Most operating systems intentionally try to randomize the address space of processes across runs (as a security feature), so that exact pointer values are not stable across runs, and thus hash value are not stable across runs, and thus the ordering of entries is not stable across runs.
This change identifies a few cases of iterating over dictionaries or sets that could have produced output non-determinism:
* The `HLSLIntrinsicSet` was using a `Dictionary` to store intrinsics that had been referenced, and would later produce a linear list of those intrinsics based on their order in the dictionary.
* The `WitnessTable`s produced by the front-end stored a `Dictionary` or requirements, and lowering from AST->IR was iterating over that dictionary to ensure that everythign got emitted.
* The `SharedSemanticsContext` was tracking a `HashSet` of modules that were imported into scope (so that their `extension`s should be visible), and an iteration over that list was used when producing candidate extensions during lookup. This case is unlikely to cause any nondeterminism in final output, but could lead to nondeterministic ordering in diagnostic messages for ambiguous reference/overload cases.
* The IR linker maintains a `Dictionary` of symbols based on their mangled names, and iterates over it in code that clones all witness tables into the linked IR whether or not they are referenced.
For most of these cases the fix is simple:
* Keep both a `Dictionary`/`HashSet` and a `List` of the appropriate type
* Whenever adding to the hash-based container also add to the list
* Whenever iterating, iterate over the list
In the final case of the IR linker, the relevant code was marked with a `TODO` comment noting that it shouldn't actually be needed, so I simply dropped it and the change doesn't seem to break any of our tests. I've been fairly confident that code wasn't needed for a while.
This change isn't exactly elegant, and a better long term solution might be to introduce two new types, `OrderedDictionary` and `OrderedSet`, which are similar to `Dictionary` and `HashSet` except that they guarantee a deterministic order of enumeration of their contents, based on insertion order.
(Note that a `SortedDictionary` and/or `SortedSet` that use something like a binary tree to produce a "determinsitc" sorted order wouldn't actually help here, because sorting entries by pointer values wouldn't solve the underlying problem that the pointer values aren't stable across runs)
I've chosen to avoid adding new types to `core` in the interest of making the change as small as possible. If we all agree that new types are warranted, it should be easy to clean up these use cases.
Testing this change is difficult, because we can't produce a reliable test to rule out nondeterminism. I have done best-effort testing by hand by crafting shaders that show output nondeterminism, and then compiling them both with and without these changes.
|
