| Age | Commit message (Collapse) | Author |
|---|
|
* WIP on serialize/save state.
* Relative string encoding.
* Added RelativeContainer unit test.
Split out RelativeContainer into core.
* Fix bug in RelativeString encoding.
* More work around relative container.
* Fix checks.
* Use RelativeBase for safe access.
Use malloc/free/realloc instead of List.
* Add natvis support for relative types.
* Setting up of state (not includes) writing of repro state.
* Capture after spCompile.
* Writing SourceFile and file system files.
Added -dump-repo
* First pass at loading state.
* First pass at reading repro.
* Small optimization around Safe32Ptr
* Refactor how repro data is stored - to make saving off the files more simple, by having all all backed by 'files'.
Make file loading always set up PathInfo so we get uniqueIdentifier info.
* Generate unique file names.
* Added RelativeFileSystem
Added saveFile to ISlangFileSystemExt and implemented for interfaces
Added mechanism to save of files (and manifest)
* Added ability to replace files in repo with directory holding their contents.
* Add support for entry points.
* Fix problem compiling on linux.
* Added SIMPLE_EX option, where everything on command line must be specified.
* Fix typo in unit test for relative container.
* Fix another typo in unit test for RelativeContainer.
* Fix small bugs.
* Fix release unused variable issue in slang-state-serialize.cpp
* Fix checking for SIMPLE_EX in testing, else broke COMMAND_LINE_SIMPLE.
* Fix warnings on 32 bit debug build.
* Added import-subdir-search-path-repro.slang test. Although disabled for now as writes to root of slang project.
* Remove wrong version of import-subdir-search-path-repro.slang
* Added import-subdir-search-path-repro.slang
|
|
* Initial work on representing layout at IR level
This change starts the process of making the back-end of the compiler independent of the AST-level layout information (`TypeLayout`, `VarLayout`, etc.) so that it instead only relies on layout information that is embedded into IR modules. This brings us incrementally closer to a world in which the back-end could be run without the AST-level structures even existing (e.g., for an application that just wants to ship IR without any AST information for IP protection, while still supporting some amount of linking and specialization).
The main parts of the change are:
* There is a bunch of incidental churn related to specifying entry points by index instead of the `EntryPoint` object for certain operations. This ends up being a better choice because we can use the index to look up side-band information about the entry point that might not be stored on the `EntryPoint` object itself. In particular...
* We expand the `ComponentType` interface to support looking up the mangled name of an entry point by index. In common cases (no generic/interface specialization) this would be the same as asking the `EntryPoint` for its mangled name, but in cases where we have specialized a generic entry point, the mangled name would include speicalization arguments that are only available on the `SpecializedComponentType` that wraps the entry point. This part of the change isn't ideal and there might be a better solution waiting to be invented. Note that we store mangled entry point names as strings rather than using `DeclRef`s because that ensures that the information could be serialized and deserialized without a dependence on the AST.
* The `TargetProgram` type (which represents binding a specific `ComponentType` for a shader program to a specific `TargetRequest` that represents the target platform) is expanded to include an `IRModule` that represents layout information, in addition to the AST-level `ProgramLayout` it already contained. We create both of these objects at the same time (on-demand) to simplify the overall flow (so that any code that triggers creation of the AST-level layout will also ensure that the IR-level layout exists).
* A bunch of code in the emit passes that was passing down layout-related objects has been eliminated. It appears that most of those objects weren't actually being used, so this is just a cleanup, but it helps ensure that the back-end steps are "clean" and don't depend on the AST-level information. The one big exception here is that the emit logic needs to know the stage for the entry point being emitted (to deal with one wrinkle in translating DXR to VKRT).
* A big change (actually introduced by @jsmall-nvidia in a branch that this change copied and then built from) is to introduce some more explicit IR instructions to represent layout information, notably an `IRTypeLayout` and an `IRVarLayout`. For now these objects still reference their AST equivalents, but the separation gives us an incremental path to move information from the AST-level objects over to the IR ones. This work includes logic in `IRBuilder` to construct the IR-level layout objects from the AST-level ones on-demand, so that the existing code paths that try to attach AST-level layout will continue to work for now.
* Because layout information is now embedded in the IR, the `slang-ir-link.cpp` logic loses a lot of cases that used to deal with attaching AST-level layout objects to IR-level instructions during the linking process. Instead, the linker now assumes that one (or more) of the input IR modules will have layout information associated with it, and the linker makes sure to copy layout decorations (and the instructions they reference) from the input IR module(s) to the output using its more ordinary mechanisms.
* Inside `slang-lower-to-ir.cpp`, we add logic to construct an IR module in a `TargetProgram` that simply references the global shader parameters, entry points, etc. and attaches IR layout decorations to them. This is akin to the existing pass in the same file that constructs IR to represent specialization information, and both of these passes share infrastructure with the main AST->IR lowering pass. Eventually, it is expected that this pass will encompass more of the logic for copying AST-level layout information over to IR-level equivalents.
* One small wrinkle with this change was that the output for an HLSL generation test case changed some of its `#line` directives. The old code was actually more inaccurate than the new, so this change just updated the baseline. It also added some logic in the linker to make sure that when an IR instruction has multiple definitions, we try to pick up a source location from any of them, in case the "main" one somehow didn't get a location.
* Another small fix was that the key/value map in `StructTypeLayout` for mapping fields/members to their layouts was keyed on `Decl*` when it really should have been `VarDeclBase*`.
This change should in principle be a pure refactoring with no functionality changes, so no new tests were added. It is unfortunately also a change that has a high probability of breaking at least *some* client code, so we may want to be defensive and mark this with a new major version number (well, a new *minor* version number since we are pre-`1.0`) to give us some room for releasing hotfixes to the old version if needed.
* fixup: infinite recursion bug detected by clang
* fixup: remove commented-out code
|
|
* Simple testing of unbounded array of array on GPU.
* Fix problem on CPU targets around NonUniformResourceIndex
Use the unbounded-array-of-array-syntax test for CPU and GPU tests.
|
|
* WIP: Unsized arrays on CPU.
* unbounded-array-of-array working on CPU.
* Test that has an unbounded array of array directly (ie without wrapping with ParameterBlock). Test works on CPU.
* Remove some left over comments.
* Added documention on unsized array usage on CPU targets.
|
|
* WIP: Unsized arrays on CPU.
* unbounded-array-of-array working on CPU.
* Remove some left over comments.
|
|
* First pass support for performance profiling
* Test across all elements
* Fix bug - sourceContents is not used, should use rawSource.
* * Add ability to get prelude from API.
* Allow specifying source language for render-test
* Made it possible to compile a test input file as C++
* Special handling for reflection
* Added C++ impl to performance-profile.slang
* Remove some clang warnings.
* Output profile timings on appveyor and other TC.
* Remove passing around of StdWriters (can use global).
Small comment improvements.
|
|
results. (#1061)
|
|
Work on #1059
The `%` operator in the Slang implementation had several issues, and this change tries to address some of them:
* Renamed most occurences of "mod" describing this operator to be "rem" for "remainder" to better match its semantics in HLSL
* Split the operator into distinct integer and floating-point variants (`IRem` and `FRem`) to simplify having different codegen for the two
* Added floating-point variants of `operator%` and `operator%=` to the stdlib.
* Added custom C++ codegen for `kIROp_FRem` such that it maps to the standard C/C++ `remainder()` function
* Added custom GLSL codegen so that `kIROp_FRem` maps to the GLSL `mod()` function (which isn't correct...)
* Added a test case to confirm that D3D11, D3D12, and CPU targets all agree on the definition of floating-point `%`
* Fixed `render-test-tool` to allow a negative integer in a `data=...` specification. This didn't end up being used in the final test, but still seems like a good fix.
* Added a customized baseline for the Vulkan flavor of that test to confirm that we are *not* compiling correctly to SPIR-V just yet
Addressing the correctness of the output for GLSL/SPIR-V will have to come as a later change given that the operation we want is not exposed directly by unextended GLSL.
|
|
* WIP: Improving CPU performance/ABI
* Optionally output code on CPU for groupThreadID and groupID.
* Added ability to set compute dispatch size on command line for render-test.
Dispatch compute tests taking into account dispatch size.
Added test for semantics are working.
* Test using GroupRange.
* Fix problem with adding \n for externa diagnostic - to do it if there isn't a \n at the end. Change the ouput order (put result before) so last value is diagnostic string.
|
|
* First pass of render-test refactor.
* Make window construction a function that can choose an implementation.
* Remove OpenGL as currently has windows dependency.
* Disable Vulkan as Renderer impl has dependency on windows.
* Pass Window in as parameter of 'update'.
* Add win-window.cpp as was missing.
* Fix warning on windows about signs during comparison.
* * Added mechanism to add random arrays as buffer inputs and select type
* Improved RenderGenerator to generate more types, and to be more careful around int32 ranges.
* Added support for security checks (for Visual Studio C++)
* Disable Execption handling being on by default when compiling kernels
* Added a 'Group' version of the entry point that will evaluate all threads in a group in a single call. In test code use this method if available.
* Added -compile-arg to be able to pass arguments to the compile within render-test
* Add documention for the _Group execution feature.
* Fix some typos in cpu-target.md
|
|
* Updated docs to reflect ParameterBlock support
* Fixed CPU binding to handle ParameterBlocks
* Updated parameter-block.slang to be able to work as a CPU test
|
|
* * Made entry point parameters a separate entry point
* Made CPUMemoryBinding work with entry point parameters/initialize constant buffers
* Added isCPUOnly to bindings, because entry point parameters do not layout like constant buffer
* entry-point-uniform.slang works on CPU
* EntryPointParams -> UniformEntryPointParams
Updated CPU documentation.
* Update cpu-target.md to removed completed issues.
* Only allocate CPU buffers if the size is > 0.
Small update to cpu-target doc.
|
|
* WIP: Memory binding.
* WIP for binding.
* Fix handling of writing to constant buffer.
* Fix bug in handling indices.
|
|
* Add support for '=' when defining a name in test.
* Add support for double intrinsics.
* Add support for asdouble
Add findOrAddInst - used instead of findOrEmitHoistableInst, for nominal instructions.
Support cloning of string literals.
C++ working on more compute tests.
* Constant buffer support in reflection.
Fixed debugging into source for generated C++.
buffer-layout.slang works.
* Added cpu test result.
* Remove some commented out code.
Comment on next fixes.
* Improvements to reflection CPU code.
* C++ working with ByteAddressBuffer.
* Enabled more compute tests for CPU.
* Enabled more compute tests on CPU.
Added support for [] style access to a vector.
* Enabled more CPU compute tests.
* Handling of buffer-type-splitting.slang
Named buffers can be paths to resources
* Fix some warnings, remove some dead code.
* Fix problem with verification of number of operands for asuint/asint as they can have 1 or 3 operands. asdouble takes 2.
* Fix handling in MemoryArena around aligned allocations. That _allocateAlignedFromNewBlock assumed the block allocated has the aligment that was requested and so did not correct the start address.
|
|
* Added setDownstreamCompilerPrelude
Renamed setPassThroughPath to setDownstreamCompilerPath.
Fixed tests.
Added prelude directory & code to TestToolUtil to setup default preludes for testing/command line apis.
* Fix merge problem
* Remove hacks to make prelude work by adding a search path as no longer needed with 'user prelude'.
* Split up prelude into scalar intrinsics, and types.
Use slang.h for main header.
slang-cpp-prelude.h can now just include what it needs (relative to prelude directory) and define the few remaining things/work arounds.
* Fix typo.
|
|
The set of supported operations in front-end constant folding was very limited: `+`, `-`, `*`, `/`, and `%`.
This meant that enum declarations like:
```
enum MyBits
{
A = 1 << 0,
B = 1 << 1,
C = A | C,
}
```
would fail to compile, with a claim that the expressions like `1 << 0` aren't compile-time constants.
This change adds `<<`, `>>`, `&`, `|`, and `^` to the list of integer operations we will cosntant-fold in the front-end. It also changes one of the declarations in the existing test case for `enum`s to use the added functionality.
Note that this change does *not* address the more deep-seated problems with our approach to constant-folding in the front-end. It does not change the constant folding to rely on IR machinery, or to allow for more general `constexpr` functions, and it does not address the fact that constant-folding is currently applied without paying attention to the type (and thus precision) of the original expression.
|
|
* * Simplify some of test code around CPPCompiler
* Test using 'callable' with pass-through
* Small cpu doc improvements
* Improvements to Clang output parsing.
* Remove temporary file (base filename) .
* Improve handling of external errors - handle severity.
* On error dumping out to 'actual' file for runCPPCompilerCompile.
* Small fixes.
Set the source language type correctly for pass thru.
* Remove warning for test for clang backend c
* Preliminary work around making render-test compute potentiall work with CPU.
Made ShaderCompiler -> a stateless ShaderCompilerUtil.
Means we don't require a Renderer interface to do shader compilation.
* Refactor such that CPU test can take place in without Window or Renderer.
* Hack to look for prelude in source file directory.
Fix bug returning the SharedLibrary for HostCallable.
* Compute test running on CPU.
* Need the prelude currently in same directly as test.
* Hack to remove warning - that then produces an error on appveyor build.
Disable running render CPU test on non-windows.
* Improve handling of disabling CPU tests on linux.
* Added bit-cast.slang working on CPU.
|
|
If the user writes code like this:
MyStruct s = (MyStruct) 0;
then we will interpret it as if they had written:
MyStruct s = {};
That is, the "cast from zero" idiom will be taken as a legacy syntax for default construction (using an empty initializer list). This will be semantically equivalent to zero-initialization for all existing HLSL code (where `struct` fields can't have default initialization expressions defined), and is the easiest option for us to support in Slang (since we already support default-initialization using empty initializer lists).
The implementation of this feature is narrowly scoped:
* It only targets explicit cast expressions like `(MyStruct) 0` and not "constructor" syntax like `MyStruct(0)`
* It only applies when there is a single argument that is exactly an integer literal with a zero value (not a reference to a `static const int` that happens to be zero).
This change adds a test case to make sure that the feature works as expected. Because it relies on our existing initializer-list handling, the "cast from zero" idiom should work for any user-defined type where an initializer list would work.
|
|
Before this change, global and function-scope `static const` declarations were represented as instructions of type `IRGlobalConstant`, which was represented similarly to an `IRGlobalVar`: with a "body" block of instructions that compute/return the initial value.
This representation inhibited optimizations (because a reference to a global constant would not in general be replaced with a reference to its value), and also caused problems for resource type legalization because the logic for type legalization did not (and still does not) handle initializers on globals (so global *variables* that contain resource types are still unsupported).
The change here is simple at the high level: we get rid of `IRGlobalConstant` and instead handle global-scope constants as "ordinary" instructions at the global scope. E.g., if we have a declaration like:
static const int a[] = { ... }
that will be represented in the IR as a `makeArray` instruction at the global scope, referencing other global-scope instructions that represent the values in the array.
This simple choice addresses both of the main limitations. A `static const` variable of integer/float/whatever type is now represented as just a reference to the given IR value and thus enables all the same optimizations. When a `static const` variable uses a type with resources, the existing legalization logic (which can handle most of the "ordinary" instructions already) applies.
Another secondary benefit of this approach is that the hacky `IREmitMode` enumeration is no longer needed to help us special-case source code emit for `static const` variables.
Beyond just removing `IRGlobalConstant`, and updating the lowering logic to use the initializer direclty, the main change here is to the emit logic to make it properly handle "ordinary" instructions that might appear at global scope.
One open issue with this change, that could be addressed in a follow-up change, is that "extern" global constants that need to be imported from another module (but which might not have a known value when the current module is compiled) aren't supported - we don't have a way to put a linkage decoration on them. A future change might re-introduce global constants as a distinct IR instruction type that just references the value as an operand (if it is available). We would then need to replace references to an IR constant with references to its value right after linking.
|
|
* Start exposing a new COM-lite API
This change is mostly about exposing a new API to the Slang compiler that allows more fine-grained control over the compilation flow. The basic concepts in the new API are:
* An `IGlobalSession` is the granularity at which we load/parse the Slang stdlib, and therefore gives applications a way to amortize startup cost for the library across multiple compiles. This is a concept that might be able to go away in a future version of Slang.
* An `ISession` owns all the code that gets loaded/compiled/generated. Any `import`ed modules are shared across everything in a session (we don't re-parse/-check the code when we see another `import` for the same module). Any generic- or interface-based code in the session can be specialized using types from the same session (but not necessarily across sessions).
* An `IModule` is the unit of code loading and scoping. It doesn't expose any API in this change, but would be the right scope for looking up types or entry points by name.
* An `IProgram` is a "linked" combination of modules and entry points from which code can be generated and reflection information queried.
This change re-uses the existing reflection API types, rather than introduce a new API that duplicates that functionality. That will probably change in a future revision.
There are two major pieces of functionality added here that aren't related to the new API:
* We now have an API concept of "entry point groups" which are one or more entry points that are intended to be used together so that they need to have non-overlapping parameters. For now this is being used to handle "hit groups" and local root signatures for ray tracing, but I'm not sure this is a concept we will keep in the long run.
* We have a very special-case (client-application-specific) flag that ascribes special meaning to the `shared` keyword, so that it can be attached to global parameters to indicate that they are actually to be part of the local root signature rather than the global one for DXR.
None of the API design (including naming) here is finalized; the only reason to check in the changes at this point to avoid having a long-running branch that leads to merge pain. Clients should *not* try to depend on the new API just yet, since it is still a work in progress.
* fixup: clang warning
* fixup: try to detect clang C++11 support
* fixup
* fixup
* fixup
* fixup
* fixup: review feedback
|
|
* Small changes based on review
* Remove the explicit 'nominal' tests
* Made isValueEqual and isEqual on on IRConstant take a pointer
* Small improvements to comments, and clarity of using 'nominal'
* Simplify comparison by just using isTypeOperandEqual as basis for isTypeEqual
* Use cross compile to test half-texture.slang on glsl
* Don't need half-texture.slang.expected
* Fix handling of nominal comparison based on review, ensuring that for nominal insts, they can only be compared by pointer.
|
|
* Specify glsl semantic format - such that conversions are possible from hlsl sematics.
* Comment improvements. Give appropriate type in glsl for sv_tessfactor. Note that sv_tessfactor is not functional though.
* Work in progress for comparison of types.
* * Fix type comparison issues around the hash.
* Fix tests whos output changed with use of isTypeEqual
|
|
Previously, interface types were allowed to be used directly as function parameters, local variables, and global shader parameters.
Using an interface type as a field of a `struct` type or a `cbuffer` declaration was not implemented.
This change adds that support, and fixes several unrelated issues that caused problems in doing so.
* The most important work here was adding a case for `IRStructType` to `maybeSpecializeBindExistentialsType` that creates a specialized variant of a `struct` type on-demand based on specialization operands. This logic loops over the fields of the original struct, and creates new fields by binding the existentials/interfaces in the type of each field. Caching is used to ensure that the same `struct` type specialized to the same operands should yield the same result.
* To allow subsequent specialization to occur when a `struct` with interface-type fields is used, it was also necessary to specialize field-address and field-extract instructions in cases where the value that the field is being extracted from is a `wrapExistential`.
* Similarly, we neede to make sure that the logic for specializing called functions based on the concrete types for interfaces in the argument list would also take into account `struct` types with existential-type fields inside of them.
* Doing the above changes revealed some serious flaws in how the `ir-specialize.cpp` logic was tracking which instructions still needed to be processed. It had previously been assuming that it could assume any relevant instructions were on its work list, and when the work list went empty it could exit. This runs into two problems: (1) sometimes we create new instructions when specializing, and it may be impossible to ensure that all the new instructions (e.g., those created by utility routines in other files) get added to the work list, and (2) sometimes the instruction(s) that need to be re-visited when we specialize something aren't its direct users, but instead somethign that transitively depends on the instruction.
These issues were fixed by two changes to the pass: (1) we now maintain a list of known "clean" instructions instead of implicitly using the work-list as a list of "dirty" instructions (so that implicitly any new instruction is dirty), and periodically iterating over all instructions to add the non-clean ones to the work list for processing, and (2) when an instruction is specialized/replaced we mark everything that transitively depends on it "dirty" (by removing it from the "clean" list).
* Added some logic to "fix up" the type of an IR function after changes that might modify its parameter list. Failing to have this logic meant that certain types were still live (because they were referenced by a function type) that couldn't actually be emitted as legal HLSL/GLSL.
* Added some special cases to IR instruction creation for `wrapExistential` and `BindExistentialsType` so that they act as no-ops when there are no "slots" providing specialization information. This helps avoid some special cases when specializing structure fields (since some fields specialization and others don't, so in general there are zero or more operands specific to each field).
* Added a test case that uses an interface type in a `cbuffer`, as well as an interface type in a `struct` passed as an entry-point `uniform` parameter.
* Fixed up some parts of the `.natvis` files to reflect naming changes from a previous PR and thus restore some of the useful Visual Studio debugging experience for Slang.
|
|
* A few changes required for application adoption of interface-type parameters
There are a few small changes here that are all related in that they arose from trying to integrate support for specialization via global interface-type shader parameters into a real application.
Allow querying the "pending" layout via reflection API
------------------------------------------------------
The naming here isn't ideal, and could probably use a round of "bikeshedding" to arrive at something better, but the basic idea is that when you have a type like:
```
struct MyStuff
{
int a;
IFoo foo;
int b;
}
```
the fields `a` and `b` get allocated space directly in the "primary" layout for `MyStuff` (at offsets 0 and 4, with `sizeof(MyStuff) == 8`), but the `foo` field can't be allocated space until we know what concrete type will get plugged in there.
If we have a concrete type in mind:
```
struct Bar : IFoo { int bar; }
```
then we can know how much space the `foo` field will take up, but we still can't allocate it space directly in `MyStuff`, because we already decided that `sizeof(MyStuff) == 8`.
Now imagine we place some `MyStuff` values into constant buffers:
```
cbuffer X {
MyStuff x;
}
cbuffer Y {
MyStuff y;
float4 z;
}
```
In each case we know that we want to place the `MyStuff::foo` field at the end of the containing constant buffer so that it doesn't disrupt the layout of the existing fields. But that means that the offset of `MyStuff::foo` relative to the start of the `MyStuff` isn't fixed, because of unrelated fields like `z` that need to get in between.
In our layout code, we handle this by having a notion of a "pending" layout. Once we know how `MyStuff::foo` will be specialized, we can compute both a "primary" and a "pending" layout for `MyStuff`, which basically treats it as if it were two distinct types:
```
struct MyStuff_Primary
{
int a;
int b;
}
struct MyStuff_Pending
{
Bar foo;
}
```
Layout for an aggregate type like the `X` or `Y` constant buffer then proceeds by computing an aggregate primary layout and an aggregate pending layout, and then finally a constant buffer or parameter block "flushes" all or part of the pending data by appending it to the primary data to get the final layout.
What all this means is that a type like `MyStuff` will have two different layouts (a default one for the primary data and a "pending" one for any specialized interface-type fields), and a variable like `Y::y` will also have two variable layouts that specify offsets (one set of offsets for its primary part, and one set of offsets for its pending part).
In order to handle interface-type fields with these layout rules, an application needs a way to query the "pending" part of a type or variable layout, which luckily gives it back just another type/variable layout. The API change here is minimal, although actually exploiting the new API correctly in application code could prove challenging.
Allow creating of explicitly specialized types
----------------------------------------------
This feature isn't actually implemented all the way through the compiler (I just needed enough to make the API calls go through), but I've added support for specializing a type that has interface-type fields through the reflection API. This maps to an `ExistentialSpecializedType` in the AST, and I'm lowering it to the IR as a `BindExistentialsType`, although that isn't 100% correct for the future.
This feature will require a future PR to actually flesh out the implementation work, but I'll wait until that is the sticking point on the application side before I do that.
Introduce a tiny `Hasher` abstraction
-------------------------------------
While implementing all the boilerplate for a new `Type` subclass (we really need to reduce that work...), I got fed up with how we do hash-code computation and introduced a small utility `Hasher` type that is intended to wrap up the idiom of combining hashes. For now this isn't a major change, but in the future I'd like to expand on the design a bit to clean up some of the warts around how we handle hashing:
* The `Hasher` implementation can and should switch from maintaining a single `HashCode` as its state to something that contains a more complete state (larger than the hash code) and just hashes new bytes into that state as it goes. This should make it possible to implement a `Hasher` for more serious hash functions, whether MD5, CityHash, or whatever we decide is good default.
* Things that are hashable shouldn't have a `getHashCode()` method, but instead should have something like a `hashInto(Hasher&)` method. This change would have the dual benefits that (1) a composite type can easily hash all the fields that contribute to its identity into the hasher with minimal fuss/boilerplate, and (2) the hashes for composite types will be of higher quality because they can exploit all the bits of the hasher's state to combine the fields, instead of restricting each sub-field to just the bits in a hash code.
We should be able to incrementally improve the quality of our design there over future changes, but for now it probably isn't a critical priority.
Fixes for legalization of existential types
-------------------------------------------
There were some missing cases in the handling of type legalization, such that a global interface-type shader parameter that got specialized to a type that contains *only* resource-type fields would cause a crash in the legalization step.
I added a test for this case, and then made `ir-legalize-types.cpp` account for this case (the code to handle it ias a bit of a kludge, and shows that the `declareVars()` routine there is getting to a level of complexity that is worrying.
* fixup: review feedback
|
|
generation. (#947)
|
|
* Update glslang
This moves to a version of glslang that is hosted with the slang project and that includes a patch for a high-priority fix that hasn't been upstreamed into the main glslang repository yet.
* Change a GLSL extension name
The glslang codebase changed the extension name required to enable certain features from `GL_KHX_shader_explicit_arithmetic_types` to `GL_EXT_shader_explicit_arithmetic_types`.
|
|
* * Added $c macro - that will do casting to target type. Used here to cast texture reads back to half. Works in tandem with $z which will close parens.
* half-texture.slang test
* Make binding failing if TextureView fails
* Simplify logic around parens.
* Improve comment around $c macro.
* Test against hlsl output to avoid error on CI.
|
|
* Overhaul the core routines for implicit conversion
The main user-visible change is that we have fixed the bug where conversions that should only be allowed explicitly were being allowed implicitly. This might be seen as a regression by users, so we'll have to be careful when rolling out the fix.
The core of that fix involves checking whether an `init` declaration that will be invoked as an implicit conversion actually supports implicit conversions.
The main visible change in the code is some renamings to try and help make the core type-coercion routines better fit our naming conventions.
The main cleanup is to enforce the invariant that any of the implicit-conversion core routines will always emit a diagnostic (or have a subroutine it calls do so) when conversion fails and the `outToExpr` parameter is non-null. This is a small change, but should improve the user experience if an implicit conversion fails in the context of a single element of an initializer list (the error should point at the line in question, and not at the whole list).
The big thing that is impacted by removing the ability to use explicit conversions implicitly is conversion of `enum` types to integers. This was intended to be explicit (a la `enum class` in C++), but the bug made it so that implicit conversion was allowed.
Closing up that gap meant that some of the checking around user-defined attributes got wonky, because we attempt to check that the attribute argument is an integer constant expression, but an `enum` case can't possible be an integer constant - it is a value of the `enum` type. I added code to work around that issue by having a parallel path for checking compile-time-constant expressions of `enum` type, but it is clear that a more general solution is needed eventually.
* fixup: test case needs explicit cast
|
|
* Allow plugging in types with resources for interface parameters
The key feature enabled by this change is that you can take a shader declared with interface-type parameters:
```hlsl
ConstantBuffer<ILight> gLight;
float4 myShader(IMaterial material, ...)
{ ... }
```
and specialize its interface-type parameters to concrete type that can contain resources like textures, samplers, etc.
The hard part of doing this layout is that we need to support signatures that include a mix of interface and non-interface types. Imagine this contrived example:
```hlsl
float4 myShader(
Texture2D diffuseMap,
ILight light,
Texture2D specularMap)
{ ... }
```
We end up wanting `diffuseMap` to get `register(t0)` and `specularMap` to get `register(t1)`, so that they have the same location no matter what we plug in for `light`.
But if we plug in a concrete type for `light` that needs a texture register, we need to allocate it *somewhere*.
We handle this by having the `TypeLayout` for `light` come back with a "primary" type layout that doesn't have any texture registers, but with a "pending" type layout that includes the texture register requirements of whatever concrete type we plug in.
This split between "primary" and "pending" layout then needs to work its way up the hierarchy, so that an aggregate `struct` type with a mix of interface and non-interface fields (recursively), needs to compute an aggregate "primary type layout" and an aggregate "pending type layout," and then each field needs to be able to compute its offset in the primary/pending layout of the aggregate.
A large chunk of the work in this PR is then just implementing the split between primary and pending data, and ensuring that layouts are computed appropriately.
The next catch is that when a "parameter group" (either a parameter block or constant buffer) contains one or more values of interface type, then we can allow the parameter group to "mask" some of the resource usage of the concrete types we plug in, but others "bleed through."
For example, if we have:
```hlsl
struct MyStuff { float3 color; ILight light; }
ConstantBuffer<MyStuff> myStuff;
struct SpotLight { float3 position; Texture2D shadowMap; }
``
If we plug in the `SpotLight` type for `myStuff.light`, then the `float3` data for the light can be "masked" by the fact that we have a constant buffer (we can just allocate the `float3` `position` right after `color`), but the `Texture2D` needed for `shadowMap` needs to "bleed through" and become "pending" data for the `myStuff` shader parameter.
Adding support for that detail more or less required a full rewrite of the logic for allocating parameter group type layouts.
The next detail is that when we go to legalize a declaration like the `myStuff` buffer, we will end up with something like:
```hlsl
struct MyStuff_stripped { float3 color; }
struct Wrapped
{
MyStuff_stripped primary;
SpotLight pending;
}
ConstantBuffer<Wrapped> myStuff;
```
This "wrapped" version of the buffer type more accurately reflects the layout we need/want for the uniform/ordinary data, but in order to further legalize it and pull out the resource-type fields like `shadowMap` we need to have accurate layout information, and the problem is that layout information for the original buffer can't apply to this new "wrapped" buffer.
The last major piece of this change is logic that runs during existential type legalization to compute new layouts for "wrapped" buffers like these that embeds correct offset/binding/register information for any resources nested inside them. A key challenge in that code is that existential legalization needs to erase any "pending" data from the program entirely, so that offset information that used to be relatie to the "pending" part of a surrounding type now needs to be relative to the primary part.
The work here may not be 100% complete for all scenarios, but it does well enough on the new and existing tests that I want to checkpoint it. Note that a few other tests have had their output changed, but in all cases I've reviewed the diffs and determined that the change in observable behavior is consistent with what we intened Slang's behavior to be.
Note that there is still one major piece of support for interface-type parameters that is missing here, and which might force us to revisit some of the decisions in this code: we don't properly support user-defined `struct` types with interface-type fields.
* fixup: typos
|
|
An `enum` case currently lowers to the IR as the value of its tag expression, so if we have:
```hlsl
enum E
{
X = 99,
}
```
then `X` will lower as IR for the expression `99` which is just a literal.
If instead we have:
```hlsl
enum E
{
X = 99u,
}
```
then after type checking the expression is `int(99u)`, which will get emitted to the global (module) scope as a cast/conversion instruction, that then gets referenced at the use sites of `E.X`.
The emit logic wasn't set up to handle the case of referencing something directly from the global scope, so this change makes it so that side-effect-free global instructions are treated just like literal constants. This simple handling is fine for now since it will only apply to `enum` tag values (true global constants declared with `static const` have different handling).
|
|
* Disable Dx12 half tests. The half-calc test runs, but is not actually doing any half maths. If the code is changed such that it is, the device fails when the shader is used. This can be seen by looking at dxil-asm.
* Fix using software driver for dx12 even when hardware is requested.
* * Refactor Dx12 _createAdapter such that it doesn't have side effects and stores desc information
* Disable half on dx12 software renderer because it crashes
* * Disable erroneous warnings from dx12
* Test for adapter creation
* Identify warp specifically
* Structured buffer test now works on dx12.
* Fix intemittent crash on dx12.
Due to if a resource was initialized with data, the actual resource constructed might be larger, for alignment issues. This led to a memcpy potentially copying from after the allocated source data and therefore a crash. Now only copies the non aligned amount of data.
* * Rename the test to use - style
* Disable TextureCube lookup in tests, as does not produce the correct result in dx12 (will fix in future PR)
* Updated hlsl.meta.slang.h that has rcp for glsl.
* * Fix bug where the SRV description was incorrect for cubemaps on dx12
* Re-enable cube map access in dx12 tests
* Slightly re-organize texture upload on dx12 to not repeatidly create and destroy upload resource for array/cube scenarios
|
|
* Disable Dx12 half tests. The half-calc test runs, but is not actually doing any half maths. If the code is changed such that it is, the device fails when the shader is used. This can be seen by looking at dxil-asm.
* Fix using software driver for dx12 even when hardware is requested.
* * Refactor Dx12 _createAdapter such that it doesn't have side effects and stores desc information
* Disable half on dx12 software renderer because it crashes
* * Disable erroneous warnings from dx12
* Test for adapter creation
* Identify warp specifically
* Structured buffer test now works on dx12.
* Fix intemittent crash on dx12.
Due to if a resource was initialized with data, the actual resource constructed might be larger, for alignment issues. This led to a memcpy potentially copying from after the allocated source data and therefore a crash. Now only copies the non aligned amount of data.
* * Rename the test to use - style
* Disable TextureCube lookup in tests, as does not produce the correct result in dx12 (will fix in future PR)
* Updated hlsl.meta.slang.h that has rcp for glsl.
|
|
any half maths. If the code is changed such that it is, the device fails when the shader is used. This can be seen by looking at dxil-asm. (#914)
|
|
* Look at getting half to work on vk.
* Alter half test so can always produce consistent test results.
* First pass working half on vk.
* Improve comments for vulkan extensions around half.
* Upgraded vulkan headers to v1.1.103
https://github.com/KhronosGroup/Vulkan-Headers
* * Add getFeatures on Render interface
* Vulkan renderer determines at startup if it can support half
* Parse render-features on render-test
* Small changes to half-calc.slang test.
* Structured buffer half access works as expected for Vk, but isn't for dx12, so disable for now.
* Require the half feature for renderers for the half-structured-buffer.slang test.
* * Added ToolReturnCode to be more rigerous about how a return code is passed back from a tool
* Added support for a tool being able to pass back an 'ignored' result.
* Used enum codes to indicate meanings
* Made spawnAndWait return a ToolReturnCode
* Ignore tests that don't have required render-feature
* Fix macro line continuation usage.
* Check dx12 has half support.
* Checking for half on dx12 - if CheckFeatureSupport fails, don't fail renderer initialization.
* Fix typo.
|
|
* * leftSide and rightSide set op to nullptr, before was just uninitialized
* Added support for GLSL for vector/scalar comparisons
* Added test
* * Remove unneeded precedence code.
* Simplify function to _maybeEmitGLSLCast
* * Take into account precedence & closing of brackets in same way as function call, if function call used for vector comparison (as on GLSL)
|
|
* Improve support for interfaces as shader parameters
This change adds two main things over the existing support:
1. It is now possible to plug in concrete types that actually contain (uniform/ordinary) fields for the existential type parameters introduced by interface-type shader parameters. The `interface-shader-param2.slang` test shows that this works.
2. There is a limited amount of support for doing correct layout computation and generating output code that matches that layout, so that interface and ordinary-type fields can be interleaved to a limited extent. The `interface-shader-param3.slang` test confirms this behavior.
There are several moving pieces in the change.
* When it comes to terminology, we try to draw a more clear distinction between existial type parameters/arguments and existential/object value parametes/arguments. A simple way to look at it is that an `IFoo[3]` shader parameter introduces a single existential type parameter (so that a concrete type argument like `SomeThing` can be plugged in for the `IFoo`) but introduces three existential object/value parameters (to represent the concrete values for the array elements).
* At the IR level, we support a few new operations. A `BindExistentialsType` can take a type that is not itself an interface/existential type but which depends on interfaces/existentials (e.g., `ConstantBuffer<IFoo>`) and plug in the concrete types to be used for its existential type slots.
* Then a `wrapExistentials` instruction can take a type with all the existentials plugged in (possibly by `BindExistentialsType`) and wrap it into a value of the existential-using type (e.g., turn `ConstantBuffer<SomeThing>` into a `ConstantBuffer<IFoo>`).
* The IR passes for doing generic/existential specialization have been updated to be able to desugar uses of these new operations just enough so that a `ConstantBuffer<IFoo>` can be used.
* When we specialize an IR parameter of an interface type like `IFoo` based on a concrete type `SomeThing`, we turn the parameter into an `ExistentialBox<SomeThing>` to reflect the fact that we are conceptually referring to `SomeThing` indirectly (it shouldn't be factored into the layout of its surrounding type).
* Parameter binding was updated so that it passes along the bound existential type arguments in a `Program` or `EntryPoint` to type layout, so that we can take them into account. The type layout code needs to do a little work to pass the appropriate range of arguments along to sub-fields when computing layout for aggregate types.
* Type layout was updated to have a notion of "pending" items, which represent the concrete types of data that are logically being referenced by existential value slots. The basic idea is that these values aren't included in the layout of a type by default, but then they get "flushed" to come after all the non-existential-related data in a constant buffer, parameter block, etc.
* The logic for computing a parameter group (`ConstantBuffer` or `ParameterBlock`) layout was updated to always "flush" the pending items on the element type of the group, so that the resource usage of specialized existential slots would be taken into account.
* The type legalization pass has been adapted so that we can derive two different passes from it. One does resource-type legalization (which is all that the original pass did). The new pass uses the same basic machinery to legalize `ExistentialBox<T>` types by moving them out of their containing type(s), and then turning them into ordinary variables/parameters of type `T`.
Big things missing from this change include:
- Nothing is making sure that "pending" items at the global or entry-point level will get proper registers/bindings allocated to them. For the uniform case, all that matters in the current compiler is that we declare them in the right order in the output HLSL/GLSL, but for resources to be supported we will need to compute this layout information and start associating it with the existential/interface-type fields.
- Nothing is being done to support `BindExistentials<S, ...>` where `S` is a `struct` type that might have existential-type fields (or nested fields...). Eventually we need to desugar a type like this into a fresh `struct` type that has the same field keys as `S`, but with fields replaced by suitable `BindExistentials` as needed. (The hard part of this would seem to be computing which slots go to which fields). As a practial matter, this missing feature means that interface-type members of `cbuffer` declarations won't work.
The current tests carefully avoid both of these problems. They don't declare any buffer/texture fields in the concrete types, and they don't make use of `cbuffer` declarations or `ConstantBuffer`s over structure types with interface-type fields.
* fixup: add override to methods
* fixup: typos
|
|
* Fix up handling of NumElements for D3D buffer views
For everything but structured buffers we'd been setting this to the size in bytes, but that isn't really valid at all.
The `NumElements` member in the view descs is supposed to be the number of buffer elements, so it would be capped at the byte size divided by the element size.
This change fixes the computation of `NumElements` to take the size of the format into account for non-structured views.
For the "raw" case, we use a size of 4 bytes since that matches the `DXGI_FORMAT_R32_TYPELESS` format we use (which seems to be required for raw buffers).
I also added support for the raw case for SRVs where it didn't seem to be supported before (not that any of our tests cover it).
* Fix handling of size padding for D3D11 buffers
The existing code was enforcing a 256-byte-aligned size for all buffers, but this can cause problems for a structured buffer.
A structured buffer must have a size that is a multiple of the stride, so a structured buffer with a 48-byte stride and a 96-byte size would get rounded up to 256 bytes, which is not an integer multiple of 48.
This change makes it so that we only apply the padding to constant buffers.
According to MSDN, constant buffers only require padding to a 16-byte aligned size, and no other restrictions are listed for D3D11, but it is difficult to know whether those constrains are exhaustive.
I've left in the 256-byte padding for now (rather than switch to 16-byte), even though I suspect that was only needed as a band-aid for the `NumElements` issue fixed by another commit.
* Fix an IR generation bug when indxing into a strutured buffer element
The problem here arises when we have a structured buffer of matrices (an array type would likely trigger it too):
```hlsl
RWStructuredBuffer<float3x4> gMatrices;
```
and then we index into it directly, rather than copying to a temporary:
```hlsl
// CRASH:
float v = gMatrices[i][j][k];
// OKAY:
float3x4 m = gMatrices[i];
float v = m[j][k];
```
The underlying issue is that our IR lowering pass tries to defer the decision about whether to use a `get` vs. `set` vs. `ref` accessor for a subscript until as late as possible (this is to deal with the fact that sometimes D3D can provide a `ref` accessor where GLSL can only provide a `get` or `set`).
We probably need to overhaul that aspect of IR codegen sooner or later, but this change uses some of the existing machinery to try to force the `gMatrices[i]` subexpression to take the form of a pointer when doing sub-indexing like this. This fixes the present case, and hopefully shouldn't break anything else that used to work (because the subroutines I'm using to coerce the `gMatrices[i]` expression should be idempotent on the cases that were already implemented).
* Add a test case to confirm fxc/dxc layout for structured buffers of matrices
Even when row-major layout is requested globally, fxc and dxc seem to lay out a `StructuredBuffer<float3x4>` with column-major layout on the elements.
This commit adds a test that confirms that behavior.
This commit does not try to implement a fix for the issue (either fixing Slang's layout reflection information to be correct for what fxc/dxc do in practice, or fixing Slang's HLSL output to work around the fxc/dxc behavior), but just documents the status quo.
If/when we decide how we'd like to handle the issue long-term, this test can/should be updated to match the decision we make.
* fixup: build breakage on clang/gcc
This is one of those cases where the Microsoft compiler is letting through some stuff that isn't technically valid C++ ("delayed template parsing").
Fixed by just moving some declarations to earlier in the file.
|
|
* Added test for scope operator
|
|
and vector (#864)
* Added identity bit casts for matrix (cos no op). We don't support matrix asint on glsl targets
* Added tests in bit-cast.slang
|
|
* First steps toward supporting interface-type parameters on shaders
What's New
----------
From the perspective of a user, the main thing this change adds is the ability to declare top-level shader parameters (either at global scope, or in an entry-point parameter list) with interface types. For example, the following becomes possible:
```hlsl
// Define an interface to modify values
interface IModifier { float4 modify(float4 val); }
// Define some concrete implementations
struct Doubler : IModifier
{
float4 modify(float4 val) { return val + val; }
}
struct Squarer : IModifier { ... }
// Define a global shader parameter of interface type
IModifier gGlobalModifier;
// Define an entry point with an interface-type `uniform` parameter
void myShader(
unifrom IModifier entryPointModifier,
float4 inColor : COLOR,
out float4 outColor : SV_Target)
{
// Use the interface-type parameters to compute things
float4 color = inColor;
color = gGlobalModifier.modify(color);
color = entryPointModifier.modify(color);
outColor = color;
}
```
The user can specialize that shader by specifying the concrete types to use for global and entry-point parameters of interface types (e.g., plugging in `Doubler` for `gGlobalModifier` and `Squarer` for `entryPointModifier`).
The "plugging in" process is done in terms of a concept of both global and local "existential slots" which are a new `LayoutResourceKind` that represents the holes where concrete types need to be plugged in for existential/interface types.
In simple cases like the above, each interface-type parameter will yield a single existential slot in either the global or entry-point parameter layout. Users can query the start slot and number of slots for each shader parameter, just like they would for any other resource that a parameter can consume. Before generating specialized code, the user plugs in the name of the concrete type they would like to use for each slot using `spSetTypeNameForGlobalExistentialSlot` and/or `spSetTypeNameForEntryPointExistentialSlot`.
There are some major limitations to the implementation in this first change:
* Parameters must be of interface type (e.g., `IFoo`) and not an array (`IFoo[3]`), or buffer (`ConstantBuffer<IFoo>`) over an interface type. Similarly, `struct` types with interface-type fields still don't work.
* The work on interface-type function parameters still doesn't include support for `out` or `inout` parameters, nor for functions that return interface types (that isn't technically related to this change, but affects its usefullness).
* No work is being done to correctly lay out shader parameters once the concrete types for existential slots are known, so that this change really only works when the concrete type that gets plugged in is empty.
These limitations are severe enough that this feature isn't really usable as implemented in this change, and this merely represents a stepping stone toward a more complete implementation.
Implementation
--------------
The API side of thing largely mirrors what was already done to support passing strings for the type names to use for global/entry-point generic arguments, so there should be no major surprises there.
The logic in `check.cpp` computes the list of existential slots when creating unspecialized `Program`s and `EntryPoint`s (this is logically the "front end" of the compiler), and then checks the supplied argument types against what is expected in each slot when creating specialized `Program`s and `EntryPoint`s. This again mirrors how generic arguments are handled.
Type layout was extended to compute the number of existential slots that a type consumes, and will thus automatically assign ranges of slots to top-level and entry-point shader parameters in the same way it already allocates `register`s and `binding`s. The big missing feature is the ability to specialize a layout to account for the concrete types plugged into the existential-type slots.
IR generation for specialized programs and entry points was slightly extended so that it attaches information about the concrete types plugged into the existential slots, and the witness tables that show how they conform to the interface for that slot. The linking step needed some small tweaks to make sure that information gets copied over to the target-specific program when we start code generation.
The meat of the IR-level work is in `ir-bind-existentials.cpp`, which takes the information that was placed in the IR module by the generation/linking steps and uses it to rewrite shader parameters. For example, if there is a shader parameter `p` of type `IModifier`, and the corresponding existential slot has the type `Doubler` in it, we will rewrite the parameter to have type `Doubler`, and rewrite any uses of `p` to instead use `makeExistential(p, /*witness that Doubler conforms to IModifier*/)`.
Once the replacement is done on the parameters, the existing work for specializing existential-based code when the input type(s) are known kicks in and does the rest.
Testing
-------
A single compute test is added to validate that this feature works. It is narrowly tailored to not require any of the features not supported by the initial implementation (e.g., all of the concrete types used have no members).
The test case *does* include use of an associated type through one of these existential-type parameters, which has exposed a subtle bug in how "opening" of existential values is implemented in the front-end. Rather than fix the underlying problem, I cleaned up the code in the front-end to special case when the existential value being opened is a variable bound with `let`, to directly use a reference to that variable rather than introduce a temporary. Similarly, in the IR generation step, I added an optimization to make variables declared with `let` skip introducing an IR-level variable and just use the SSA value of their initializer directly instead.
* fixup: missing files
* fixup: incorrect type for unreachable return
* fixup: actually comment ir-bind-existentials.cpp
|
|
* Split front- and back-ends
This change is a major refactor of several of the types that provide the behind-the-scenes implementation of the public C API.
The goal of this refactor is primarily to allow for future API services that let the user operate both the front- and back-ends of the compiler in a more complex fashion.
For example, as user should be able to compile a bunch of source code into modules, look up types, functions, etc. in those modules, specialize generic types/functions to the types they've looked up, and then finally request target code to be gernerated for specialized entry points.
The back-end code generation they trigger should re-use the front-end compilation work (parsing, semantic checking, IR generation) that was already performed.
The most visible change is that `CompileRequest` has been split up into several smaller types that take responsibility for parts of what it did:
* The `Linkage` type owns the storage for `import`ed modules, and well as the `TargetRequest`s that represent code-generation targets. The intention is that an application could use a single `Linkage` for the duration of its runtime (so long as it was okay with the memory usage), so that each `import`ed module only gets loaded once. For now, this type needs to manage the search paths, file system, and source manager, because of its responsibility for loading files.
* A `FrontEndCompileRequest` owns the stuff related to parsing, semantic checking, and initial IR generation. This most notably includes the `TranslationUnitRequest`s and the `FrontEndEntryPointRequest`s (which used to be just `EntryPointRequest`s). It's main job is to produce AST and IR modules for each translation unit, and to find and validate the entry points. The front-end request does *not* interact with generic arguments for global or entry-point generic parameters.
* The main output of both `import` operations and front-end translation units is the `Module` type, which is just a simple container for both the AST module (to service the reflection/layout APIs, and also for semantic checking of code that `import`s the module) and the IR module (for linking and code generation). This type captures the commonalities between the old `LoadedModule` (which is now just an alias for `Module`) and `TranslationUnitRequest` (which now owns a `Module`).
* The secondary output of front-end compilation is a `Program`, which comprises a list of referenced `Module`s and validated `EntryPoint`s that will be used together. Layout and code generation both need a `Program` to tell them what modules and entry points will be used together (we don't want to just code-gen everythin that has ever been loaded into the linakge). The `Program`s created by the front-end do not include generic arguments, so they may provide incomplete layout information and/or be unsuitable for code generation.
* A `BackEndCompileRequest` owns stuff related to turning a `Program` into output kernels for the targets of a `Linkage`. Most of the data it owns beyond the `Program` to be compiled is minor, so this is a good candidate for demotion from a heap-allocated object to just a `struct` of options that gets passed around.
* The `CompileRequestBase` type is an attempt to wrap up the common functionality of both front-end and back-end compile requests. Most of it is just exposing the availability of a linkage and `DiagnosticSink`, so this type is a good candidate for subsequent removal. The main interesting thing it has is the flags related to dumping and validation of IR, so there is probably a good refactoring still to be made around deciding how options should be handled going forward.
* Behind the scenes, the `Program` type is set up to handle some level of on-line compilation and layout work. The `Program` knows the `Linkage` it belongs to, and allows for a `TargetProgram` to be looked up based on a specific `TargetRequest`. A `TargetProgram` then allows layout information and compiled kernel code to be asked for on-demand, in order to support eventual "live" compilation scenarios.
* The `EndToEndCompileRequest` type is a composition/coordination type that replaces the old `CompileRequest` in a way that uses the services of the various other types. It owns a few pieces of state that only make sense in the context of an end-to-end compile (e.g., there is really no way to "pass through" code when the front- and back-ends are run separately) or a command-line compile (everything to do with specifying output paths for files is really just for the benefit of `slangc`, and might even be moved there over time).
* One important detail is that the `EndToEndCompilRequest` owns all of the string-based generic arguments for both global and entry-point generic parameters. The logic in `check.cpp` for dealing with those arguments has been heavily refactored to separate out the parsings steps that are specific to end-to-end compilation with string-based type arguments, and the semantic checking steps that result in a specialized `Program` (which can be exposed through new APIs that aren't tied to end-to-end compilation).
It is perhaps not surprising that this change had a lot of consequences, so I'll briefly run over some of the main categories of changes required:
* I changed the way that global generic arguments are passed via API (use `spSetGlobalGenericArgs` instead of the generic arguments for `spAddEntryPointEx`, which are not just for entry-point generics), which has been a change that we've needed for a long time. This is technically a breaking API change, although we should have very few client applications that care about it.
* A bunch of places that used to take "big" objects like `CompileRequest` now just take the sub-pieces they care about (e.g., a function might have only needed a `Linkage` and a `DiagnosticSink`). This makes many subroutines or "context" struct types more generally useful, at the cost of taking more parameters.
* In a few cases the conceptually clean separation of the layers breaks down (often for edge-case or compatibility features), and so we may pass along additional objects that are allowed to be null, but are used when present. A big example of this is how the back-end code generation routines accept an `EndToEndCompileRequest` that is optional, and only used to check whether "pass through" compilation is needed. We should probably look into cleaning this kind of logic up over time so that we don't need to violate the apparent separation of phases of compilation.
* In cases where separation of layers was being broken for the sake of GLSL features, I went ahead and ripped them out, since all of that should be dead code anyway.
* In many cases I increased the encapsulation of data in the core types to help track down use sites and make sure they are following invariants better.
* In cases where code was doing, e.g., `context->shared->compileRequest->session->getThing()` I have tried to introduce convenience routines so that the usage site is just `context->getThing()` to improve encapsulation and allow changes to be made more easily going forward.
* The `noteInternalErrorLoc` functionality was moved off of the compile request and into `DiagnosticSink`, since that is the one type you can rely on having around when you want to note an internal error. We may consider going forward if (and how) it should reset the counter used for noting locations on internal errors.
* A few APIs now take `DiagnosticSink*` arguments where they didn't before, and as a result some public APIs need to create `DiagnosticSink`s to pass in, before going ahead and ignoring the messages. In the future there should be variations of these APIs that accept an `ISlangBlob**` parameter for the output.
* fixup: missing include for compilers with accurate template checking (non-VS)
* fixup: review feedback
|
|
values being cast between valid floats. (#832)
* Typo fix
|
|
|
|
|
|
|
|
* Allow entry points to have explicit generic parameters
Prior to this change, the Slang implementation required users to use global `type_param` declarations in order to specialize a full shader. For example:
```hlsl
type_param L : ILight;
ParameterBlock<L> gLight;
[shader("fragment")]
float4 fs(...)
{ ... gLight.doSomething() ... }
```
With this change we can rewrite code like the above using explicit generics, plus the ability to have `uniform` entry-point parameters:
```hlsl
[shader("fragment")]
float4 fs<L : ILight>(
uniform ParameterBlock<L> light,
...)
{ ... light.doSomething() ... }
```
Having this support in place should make it possible for us to eliminate global generic type parameters and the complications they cause (both at a conceptual and implementation level).
The most central and visible piece of the change is that `EntryPointRequest` now holds a `DeclRef<FuncDecl>` instead of just ` RefPtr<FuncDecl>`, which allows it to refer to a specialization of a generic function.
Various places in the code that refer to the `EntryPointRequest::decl` member now use a `getFuncDecl()` or `getFuncDeclRef()` method as appropriate (see `compiler.h`).
In order to fill in the new data, the `findAndValidateEntryPoint` function has been greaterly overhauled.
The changes to its operation include:
* The by-name lookup step for the entry point function has been adapted to accept either a function or a generic function.
* The generic argument strings provided by API or command line are no longer parsed all the way to `Type`s, but instead just to `Expr`s in the first pass.
* There are now two cases for checking the global generic arguments against their matching parameters. The first case is the new one, where we plug the generic argument `Expr`s into the explicit generic parameters of an entry point (that case re-uses existing semantic checking logic). The second case is the pre-existing code for dealing with global generic type arguments.
The `lower-to-ir.cpp` logic for hadling entry points then had to be extended. Making it deal with a full `DeclRef` instead of just a `Decl` was the easy part (just call `emitDeclRef` instead of `ensureDecl`).
The more interesting bits were:
* We need to carefully add the `IREntryPointDecoration` to the nested function and not the generic in the case where we have a generic entry point. There is a handy `getResolvedInstForDecorations` that can extract the return value for an IR generic so that we can decorate the right hting.
* We need to make sure that in the case where we emit a `specialize` instruction (which normally wouldn't get a linkage decoration), we attach an `[export(...)]` decoration to it with the mangled name of the decl-ref, so that it can be found during the linking step.
The IR linking step is then slightly more complicated because the mangled entry point name could either refer directly to an `IRFunc` or to a `specialize` instruction for a generic entry point. The logic was refactored to first clone the entry point symbol without concern for which case it is (the old code was specific to functions), and then *if* the result is a `specialize` instruction, we attempt to run generic specialization on-demand.
That on-demand specialization is a bit of a kludge, but it deals with the fact that all the downstream passing only expect to see an `IRFunc`. A future cleanup might try to split out that specialization step into its own pass, which ends up being a limited form of the specialization pass.
Since I was already having to touch a lot of the code around IR linking, I went ahead and refactored the signature of the operations. I eliminated the need for the caller to create, pass in, and then destroy an `IRSpecializationState` (really an IR *linking* state), and replaced it with a structure local to the pass (that data structure was a remnant of an older approach in the compiler), and then also renamed the main operation to `linkIR` to reflect what it is doing in our conceptual flow.
Smaller changes made along the way include:
* Refactored `visitGenericAppExpr` to create a subroutine `checkGenericAppWithCheckedArgs` so that it can be used by the entry-point validation logic described above).
* Refactored the declarations around the IR passes in `emitEntryPoint()` (`emit.cpp`), to show that things are more self-contained than they used to be (e.g., that the `TypeLegalizationContext` is now only needed by one pass).
* Refactored the generic specialization code so that there is a stand-along free function that can perform specialization on a `specialize` instruction without all the other context being required. This is only to support the limited specialization that needs to be done as part of linking.
* Updated the `global-type-param.slang` test to actually test entry-point generic parameters. In a later pass we can/should rework all the tests/examples for global type parameters over to use explicit entry-point generic parameters (at which point we should rename the tests as well). For now I am leaving thigns with just one test case, with the expectation that bugs will be found and ironed out as we expand to more tests.
* fixup
* Fixup: don't leave entry-point decorations on stuff we don't want to keep
The IR `[entryPoint]` decoration is effectively a "keep this alive" decoration, which means that attaching it to something we don't intend to keep around can lead to Bad Things.
The approach to generic entry points was attaching `[entryPoint]` to the underlying `IRFunc` because that seemed to make sense, but that meant that the `specialize` instruction at global scope scould instantiate that generic and then keep it alive, even if the resulting function wouldn't be valid according to the language rules.
As a quick fix, I'm attaching `[entryPoint]` to the `specialize` instruction instead in such cases, and then re-attaching it to the result of explicit specialization during linking.
* Port most of remaining test and rename global type parameters
This change ports as many as possible of the existing tests for global type parameters over to use entry-point generic parameters instead. For the most part this is a mechanical change.
A few test cases remain using global generic parameters, as does the `model-viewer` example application.
The reason for this is that the shaders have either or both the following features:
* A vertex and fragment shader that can/shold agree on their parameters
* A type declaration (e.g., a `struct`) that is dependent on one of the generic type parameters
In these cases, it would really only make sense to switch to explicit parameters once we support shader entry points nested inside of a `struct` type, so that we can use an outer generic `struct` as a mechanism to scope the entry points and other type-dependent declrations.
Since global-scope type parameters need to persist for at least a bit longer, I went ahead and renamed all the use sites over to use `type_param` for consistency.
|
|
|
|
Before this change, code like the following would crash the compiler:
```hlsl
interface IThing { /* ... */ }
struct Outer
{
struct Inner : IThing
{}
}
/* go on to use Outer.Inner */
```
The problem was that the front-end logic for checking interface conformances was *only* checking declarations at the top level of a module, or nested under a generic.
This change fixes the logic to recurse through the entire tree of declarations.
I have added a test case that uses a nested `struct` type to satisfy an associated type requirement, to confirm that the new check works as intended.
|
|
* Initial support for uniform parameters on entry points
The basic feature this work adds is the ability to define a shader entry point like:
```hlsl
[shader("fragment")]
float4 main(
uniform Texture2D t,
uniform SamplerState s,
float2 uv : UV)
{
return t.Sample(s,uv);
}
```
In this example, the `uniform` keyword is used to mark that the given entry point parameters are *not* varying input/output flowing through the pipeline, but rather uniform shader parameters that should function as if the shader was declared more like:
```hlsl
Texture2D t,
SamplerState s,
[shader("fragment")]
float4 main(
float2 uv : UV)
{
return t.Sample(s,uv);
}
```
Allowing `uniform` parameters on entry points makes it easier to define multiple entry points in one file without accidentally polluting the global scope with shader parameters that only certain entry points care about.
This feature is also more or less a prerequisite for allowing generic type parameters directly on entry point functions, since the main use case for those type parameters is for determining what goes in various `ConstantBuffer`s or `ParameterBlock`s.
There are two main pieces to the implementation.
First, we need to be able to compute appropriate layout information for entry points that include `uniform` parameters.
Second, we need to transform the entry point function to move any `uniform` parameters to be ordinary global-scope shader parameters, to make sure that all other back-end passes don't need to worry about this special case.
The latter piece of the implementation is, relatively speaking, simpler.
The pass in `ir-entry-point-uniforms.{h,cpp}` converts entry point parameters that are determined to be uniform (using the already-computed layout information) into fields of a `struct` type and then declares a global shader parameter based on that `struct` type (and applies already-computed layout information to that parameter).
After that, the remaining IR passes (notably including type legalization) will handle things just as for any other global shader parameter.
The changes to the layout step are more significant, but most of the changes are just cleanups and fixes to enable the feature.
The two major changes that enable entry-point `uniform` parameters are:
* In `collectEntryPointParameters` we now dispatch out to a new `computeEntryPointParameterTypeLayout` function, which decided whether to compute the type layout for a `uniform` parameter, or for a varying parameter (what used to be the default behavior handled by `processEntryPointParameterDecl`).
* The main `generateParameterBindings` routine was extended so that it allocates registers/bindings to the resources required by each entry point (using `completeBindingsForParameter`) after it has allocated registers/binding to all of the global-scope parameters (this addition is mirrored in `specializeProgramLayout`).
The effect of these changes is that the `uniform` parameters of any entry points specified in a compile request will be laid out after the global-scope parameters, in the order the entry points were specified in the compile request.
A bunch of smaller changes were made around parameter layout that are worth enumerating so that the diffs make some sense:
* The `EntryPointLayout` type was changed so that instead of trying to *be* a `StructTypeLayout`, it instead *owns* one, in the same fashion as `ProgramLayout`. This commonality was factored into a base class `ScopeLayout`, and a bunch of edits followed from that change.
* Because `uniform` parameters are moved out of the entry point parameter list early in the IR transformations, the logic in `ir-glsl-legalize.cpp` that tried to look up parameter layout information by index would no longer work if the entry point parameter list had been altered. Instead, that logic now looks for the decorations directly on the parameters.
* The `UsedRange` type in `parameter-binding.cpp` was tracking the existing parameter associated with a range using a `ParameterInfo*` (which accounts for the possibility of multiple `VarDecl`s mapping to the same logical shader parameter), when just using a `VarLayout*` is sufficient for all current use cases. The overhead of allocating a `ParameterInfo` seems like overkill for entry-point parameters, where there can't possibly be multiple declarations of the "same" parameter, so avoiding these overheads was a focus when trying to deduplicate code between the global and entry-point parameter cases.
* A bunch of parameter binding logic that was specific to GLSL input has been deleted completely. There was no way to even execute this code in the compiler today, and there is pretty much zero chance of us needing (or wanting) to deal with GLSL input in the future. This includes custom `UsedRangeSet`s specific to each translation unit, which were only needed for global-scope `in` and `out` varying declarations in GLSL.
* A bunch of functions with `EntryPointParameter` in their names were renamed to use `EntryPointVaryingParameter` to help distinguish that they only apply to the varying case, while entry point `uniform` parameters are handled elsewhere.
* The `completeBindingsForParameter` function was re-worked into something that can be used for both global-scope shader parameters (where we have a `ParameterInfo` and possibly explicit bindings) and entry-point parameters (where we expect to have neither). This helps unify the (fairly subtle) logic for how we allocate and assign bindings for resources, constant buffers, parameter blocks, etc.
* A small change was made so that the entry-point stage is attached directly to top-level parameters of the entry point, and *not* recursively to every field along the way. This could be a breaking change for some applications, but it makes more logical sense (to me); we'll have to check if this affects Falcor. This change produces different output for several of the reflection tests, but the changes are consistent with no longer attaching stage information to sub-fields of varying `struct`-type parameters.
* Because there is a bunch of repeated logic in `parameter-binding.cpp` that has to do with computing a `struct` layout for ordinary/uniform data, I tried to factor that into a single `ScopeLayoutBuilder` type, which handles computing the offsets for any parameters with ordinary data, and then also handles wrapping up the layout in a constant buffer layout if there was any ordinary data at the end.
* A similar convenience routine `maybeAllocateConstantBufferBinding` was added because I noticed multiple places in `parameter-binding.cpp` that were trying to allocate a constant buffer binding for global uniforms, and they were wildly inconsistent (and in most cases used logic that would only work for D3D).
* The main `generateParameterBindings` routine is significantly shortened by using all of these utilities that were introduced. I tried to comment the places that changed to explain the overall flow correctly.
* The `specializeProgramLayout` routine (used to take a `ProgramLayout` from `generateParameterBindings` and specialize it based on knowledge of global generic arguments) had basically been rewritten with more explicit commenting/rationale for what happens in each step. It makes use of the same shared utilities as `generateParameterBindings` and `collectEntryPointParameters`.
In terms of testing:
* I added a test case to specifically test the new behavior, and in particular I made sure to include a mix of both global and entry-point parameters and also to have entry-point parameters of both ordinary and resource/object types.
* I tweaked an existing test for global type parameters to use an entry-point `uniform` parameter instead of a global one, in an effort to migrate it toward being able to use an explicitly generic entry point.
* fixups from merge
|
|
* Support function parameters of existential (interface) type
The basic idea here is that you can define a function that takes an interface-type parameter:
```hlsl
interface IThing { void doSOmething(); }
void coolFunction(IThing thing) { ... thing.doSomething() ... }
```
and call it with a concrete value that implements the given interface:
```hlsl
struct Stuff : IThing { void doSomething() { /* secret sauce */ } }
...
Stuff stuff;
coolFunction(stuff);
```
The compiler implementation will specialize `coolFunction` based on the concrete type that was actually passed in, resulting in output code along the lines of:
```hlsl
struct Stuff { ... }
void Stuff_doSomething(Stuff this) { /* secret sauce */ }
void coolFunction_Stuff(Stuff thing) { ... Stuff_doSomething(thing); }
```
In terms of implementation the new specialization approach has been integrated into the existing pass for generic specialization (which has been refactored significantly along the way), because generic specialization can open up opportunities for existential/interface simplification and vice versa, so there is no fixed interleaving of the two passes that can clean up everything.
The new logic therefore subsumes the old code for simplifying existential types (which only worked on local variables) in `ir-existential.{h,cpp}`. The local simplification rules from that implementation have become part of the core specialization pass instead, so that they can open up further transformation opportunities enabled by existential-type simplifications.
This code in place right now only handles the basic case of a function parameter that directly uses an interface type, and not one that wraps up an interface type in an array, structure, etc. Additional simplifications need to be introduced to deal with those cases as well.
* fixup: typos
|
