slang.git/source/slang/parameter-binding.cpp, branch master

Use slang- prefix on slang compiler and core source (#973)

2019-05-31T21:20:37+00:00

* Prefixing source files in source/slang with slang- * Prefix source in source/slang with slang- prefix. * Rename core source files with slang- prefix. * Update project files. * Fix problems from automatic merge.

String/List closer to conventions, and use Index type (#959)

2019-04-29T21:03:46+00:00

* List made members m_ Tweaked types to closer match conventions. * Use asserts for checking conditions on List. Other small improvements. * List.Count() -> getSize() * List Add -> add First -> getFirst Last -> getLast RemoveLast -> removeLast ReleaseBuffer -> detachBuffer GetArrayView -> getArrayView * List:: AddRange -> addRange Capacity -> getCapacity Insert -> insert InsertRange -> insertRange AddRange -> addRange RemoveRange -> removeRange RemoveAt -> removeAt Remove -> remove Reverse -> reverse FastRemove -> fastRemove FastRemoveAt -> fastRemoveAt Clear -> clear * List FreeBuffer -> _deallocateBuffer Free -> clearAndDeallocate SwapWith -> swapWith * List SetSize -> setSize Reserve -> reserve GrowToSize growToSize * UnsafeShrinkToSize -> unsafeShrinkToSize Compress -> compress FindLast -> findLastIndex FindLast -> findLastIndex Simplify Contains * List Removed m_allocator (wasn't used) Swap -> swapElements Sort -> sort Contains -> contains ForEach -> forEach QuickSort -> quickSort InsertionSort -> insertionSort BinarySearch -> binarySearch Max -> calcMax Min -> calcMin * Initializer::Initialize -> initialize List:: Allocate -> _allocate Init -> _init IndexOf -> indexOf * * Put #include in common.h, and remove unneeded inclusions * Small refactor of ArrayView - remove stride as not used * getSize -> getCount setSize -> setCount unsafeShrinkToSize->unsafeShrinkToCount growToSize -> growToCount m_size -> m_count * Some tidy up around Allocator. * Use Index type on List. * Refactor of IntSet. First tentative look at using Index. * Made Index an Int Did preliminary fixes. Made String use Index. * Partial refactor of String. * String::Buffer -> getBuffer ToWString -> toWString * Small improvements to String. String:: Buffer() -> getBuffer() Equals() -> equals * Try to use Index where appropriate. * Fix warnings on windows x86 builds.

Add the missing case for `AssocTypeDecl` in varying parameters' layout generation. (#947)

2019-04-16T14:46:02+00:00

Allow plugging in types with resources for interface parameters (#913)

2019-03-26T19:49:02+00:00

* Allow plugging in types with resources for interface parameters The key feature enabled by this change is that you can take a shader declared with interface-type parameters: ```hlsl ConstantBuffer gLight; float4 myShader(IMaterial material, ...) { ... } ``` and specialize its interface-type parameters to concrete type that can contain resources like textures, samplers, etc. The hard part of doing this layout is that we need to support signatures that include a mix of interface and non-interface types. Imagine this contrived example: ```hlsl float4 myShader( Texture2D diffuseMap, ILight light, Texture2D specularMap) { ... } ``` We end up wanting `diffuseMap` to get `register(t0)` and `specularMap` to get `register(t1)`, so that they have the same location no matter what we plug in for `light`. But if we plug in a concrete type for `light` that needs a texture register, we need to allocate it *somewhere*. We handle this by having the `TypeLayout` for `light` come back with a "primary" type layout that doesn't have any texture registers, but with a "pending" type layout that includes the texture register requirements of whatever concrete type we plug in. This split between "primary" and "pending" layout then needs to work its way up the hierarchy, so that an aggregate `struct` type with a mix of interface and non-interface fields (recursively), needs to compute an aggregate "primary type layout" and an aggregate "pending type layout," and then each field needs to be able to compute its offset in the primary/pending layout of the aggregate. A large chunk of the work in this PR is then just implementing the split between primary and pending data, and ensuring that layouts are computed appropriately. The next catch is that when a "parameter group" (either a parameter block or constant buffer) contains one or more values of interface type, then we can allow the parameter group to "mask" some of the resource usage of the concrete types we plug in, but others "bleed through." For example, if we have: ```hlsl struct MyStuff { float3 color; ILight light; } ConstantBuffer myStuff; struct SpotLight { float3 position; Texture2D shadowMap; } `` If we plug in the `SpotLight` type for `myStuff.light`, then the `float3` data for the light can be "masked" by the fact that we have a constant buffer (we can just allocate the `float3` `position` right after `color`), but the `Texture2D` needed for `shadowMap` needs to "bleed through" and become "pending" data for the `myStuff` shader parameter. Adding support for that detail more or less required a full rewrite of the logic for allocating parameter group type layouts. The next detail is that when we go to legalize a declaration like the `myStuff` buffer, we will end up with something like: ```hlsl struct MyStuff_stripped { float3 color; } struct Wrapped { MyStuff_stripped primary; SpotLight pending; } ConstantBuffer myStuff; ``` This "wrapped" version of the buffer type more accurately reflects the layout we need/want for the uniform/ordinary data, but in order to further legalize it and pull out the resource-type fields like `shadowMap` we need to have accurate layout information, and the problem is that layout information for the original buffer can't apply to this new "wrapped" buffer. The last major piece of this change is logic that runs during existential type legalization to compute new layouts for "wrapped" buffers like these that embeds correct offset/binding/register information for any resources nested inside them. A key challenge in that code is that existential legalization needs to erase any "pending" data from the program entirely, so that offset information that used to be relatie to the "pending" part of a surrounding type now needs to be relative to the primary part. The work here may not be 100% complete for all scenarios, but it does well enough on the new and existing tests that I want to checkpoint it. Note that a few other tests have had their output changed, but in all cases I've reviewed the diffs and determined that the change in observable behavior is consistent with what we intened Slang's behavior to be. Note that there is still one major piece of support for interface-type parameters that is missing here, and which might force us to revisit some of the decisions in this code: we don't properly support user-defined `struct` types with interface-type fields. * fixup: typos

Improve logic for when a "default space" is needed (#925)

2019-03-25T19:36:41+00:00

If the user declares global shader parameters for D3D SM5.1+ or Vulkan, then they need to go into an appropriate `space` or `set`: ```hlsl Texture2D t; // should go in space/set 0 SamplerState s; // same here... ``` This also applies to allocation of spaces/sets to parameter blocks: ```hlsl ParameterBlock x; // should get space/set 0 ParameterBlock y; // should get space/set 1 ``` In cases where there are a combination of explicitly and implicitly bound parameters, anything left implicitly bound goes into a "default" space/set: ``` ParameterBlock x : register(space0); // this has claimed space/set 0 Texture2D t; // this needs a space, so a "default" space/set of 1 will be claimed SamplerState s; // this also needs a space/set, and will use the default ``` The logic for deciding when a default space/set was needing was, more or less, looking at all the global shader parameters and seeing if any of them needed a `register`/`binding`, and if so determining that a default space /set would be needed. There was a bug in that logic, though, because of cases like the following: ```hlsl ParameterBlock x; Texture2D t : register(t0, space99); ``` In this case, the parameter `t` already has an explicit binding, so it doesn't actually need a default space to be allocated. If we allocate a default space/set of 0 on the basis of `t`, then `x` will end up being shifted to space/set 1. The fix is to only consider global parameters that need `register`s/`binding`s *if* they don't have an explicit binding already (which is luckily something we are tracking during parameter binding). Note: just to clarify the behavior here, the "do we need a default space/set?" logic is done *before* automatic binding of parameters, so in a shader with any global texture/buffer/sampler parameters, those will all end up in space/set zero (in the absence of explicit bindings), and explicit blocks will start at space/set one, independent of the order of declaration. This behavior is maybe too subtle, and we might decide we need to change it, but it will have to do for now.

Improve support for interfaces as shader parameters (#886)

2019-03-09T00:24:02+00:00

* Improve support for interfaces as shader parameters This change adds two main things over the existing support: 1. It is now possible to plug in concrete types that actually contain (uniform/ordinary) fields for the existential type parameters introduced by interface-type shader parameters. The `interface-shader-param2.slang` test shows that this works. 2. There is a limited amount of support for doing correct layout computation and generating output code that matches that layout, so that interface and ordinary-type fields can be interleaved to a limited extent. The `interface-shader-param3.slang` test confirms this behavior. There are several moving pieces in the change. * When it comes to terminology, we try to draw a more clear distinction between existial type parameters/arguments and existential/object value parametes/arguments. A simple way to look at it is that an `IFoo[3]` shader parameter introduces a single existential type parameter (so that a concrete type argument like `SomeThing` can be plugged in for the `IFoo`) but introduces three existential object/value parameters (to represent the concrete values for the array elements). * At the IR level, we support a few new operations. A `BindExistentialsType` can take a type that is not itself an interface/existential type but which depends on interfaces/existentials (e.g., `ConstantBuffer`) and plug in the concrete types to be used for its existential type slots. * Then a `wrapExistentials` instruction can take a type with all the existentials plugged in (possibly by `BindExistentialsType`) and wrap it into a value of the existential-using type (e.g., turn `ConstantBuffer` into a `ConstantBuffer`). * The IR passes for doing generic/existential specialization have been updated to be able to desugar uses of these new operations just enough so that a `ConstantBuffer` can be used. * When we specialize an IR parameter of an interface type like `IFoo` based on a concrete type `SomeThing`, we turn the parameter into an `ExistentialBox` to reflect the fact that we are conceptually referring to `SomeThing` indirectly (it shouldn't be factored into the layout of its surrounding type). * Parameter binding was updated so that it passes along the bound existential type arguments in a `Program` or `EntryPoint` to type layout, so that we can take them into account. The type layout code needs to do a little work to pass the appropriate range of arguments along to sub-fields when computing layout for aggregate types. * Type layout was updated to have a notion of "pending" items, which represent the concrete types of data that are logically being referenced by existential value slots. The basic idea is that these values aren't included in the layout of a type by default, but then they get "flushed" to come after all the non-existential-related data in a constant buffer, parameter block, etc. * The logic for computing a parameter group (`ConstantBuffer` or `ParameterBlock`) layout was updated to always "flush" the pending items on the element type of the group, so that the resource usage of specialized existential slots would be taken into account. * The type legalization pass has been adapted so that we can derive two different passes from it. One does resource-type legalization (which is all that the original pass did). The new pass uses the same basic machinery to legalize `ExistentialBox` types by moving them out of their containing type(s), and then turning them into ordinary variables/parameters of type `T`. Big things missing from this change include: - Nothing is making sure that "pending" items at the global or entry-point level will get proper registers/bindings allocated to them. For the uniform case, all that matters in the current compiler is that we declare them in the right order in the output HLSL/GLSL, but for resources to be supported we will need to compute this layout information and start associating it with the existential/interface-type fields. - Nothing is being done to support `BindExistentials` where `S` is a `struct` type that might have existential-type fields (or nested fields...). Eventually we need to desugar a type like this into a fresh `struct` type that has the same field keys as `S`, but with fields replaced by suitable `BindExistentials` as needed. (The hard part of this would seem to be computing which slots go to which fields). As a practial matter, this missing feature means that interface-type members of `cbuffer` declarations won't work. The current tests carefully avoid both of these problems. They don't declare any buffer/texture fields in the concrete types, and they don't make use of `cbuffer` declarations or `ConstantBuffer`s over structure types with interface-type fields. * fixup: add override to methods * fixup: typos

Move enumeration of shader parameters to Program/EntryPoint (#870)

2019-03-01T17:43:47+00:00

There's a certain amount of logic in `parameter-binding.cpp` that just has to do with the basic problem of enumerating the shader parameters of a `Program`. The main source of complexity is that for legacy/compatibility reasons we need to consider two shader parameters with the same name as being the "same" parameter for layout purposes, and then we need to do a bunch of validation to ensure that these parameters have compatible types. The biggest part of this change is moving that logic to `Program`, so that it builds up a list of its shader parameters during the front-end work, so that any errors related to bad redeclarations will now come up even if we aren't generated target-specific layouts/code. All of the code for `getReflectionName`, `StructuralTypeMatchStack`, etc. is pretty much copy-pasted from `parameter-binding.cpp` over to `check.cpp`, with the `ParameterBindingContext` replaced with a `DiagnosticSink`. The `Program::_collectShaderParameters` function (renamed from `_collectExistentialParams`) then deals with the enumeration and deduplication logic that used to happen in `collectGlobalScopeParameters()`. The new declarations in `compiler.h` reveal the underlying reason for this change: by letting `Program` and `EntryPoint` handle the canonical enumeration of parameters, we can associate each parameter with the range of existential type slots it uses, which will simplify certain work around interfaces (not in this change...). Moving the code out of parameter binding and into `check.cpp` revealed some unused GLSL-related code that I removed while I was at it. I also found that the `IsDeclaration` case of `VarLayoutFlag` wasn't actually being used, so I went ahead and removed it (we can easily re-add it if we ever find a need for it). Overall this isn't a big cleanup (mostly just code moving, rather than being eliminated), but it will facilitate other changes, and it seems cleaner overall to do this work once in target-independent logic, rather than per-target.

A small refactor to how implicit constant buffers are getting created. (#871)

2019-03-01T16:20:09+00:00

This affects layout computation for both the global and entry-point scopes, where multiple discrete shader parameters can be declared, but for layout purposes they must be treated as if they lived in the same `struct` type. If that `struct` type ends up consuming any "ordinary" data (`LayoutResourceKind::Uniform`) then an implicit constant buffer will be needed for that scope (e.g., the way fxc produces a `$Globals` constant buffer for the global scope). The logic for computing those scope layouts had a bug in it, in that the struct type was not being updated to have the right size for uniform data at the scope. That bug hasn't bitten anybody yet because no Slang users are relying on entry-point uniforms, and global-scope uniforms aren't fully implemented (and get diagnosed as an error elsewhere in the compiler). This change fixes that bug. This change also refactors things so that the logic for creating a constant buffer layout if and only if needed is moved into `type-layout.cpp` instead of relying on `parameter-binding.cpp` to compute whether or not it needs a block on its own. This is anticipating the rules for deciding whether or not a constant buffer is needed being slightly more thorny once interface types are in the mix.

Eliminate the specializeProgramLayout() function (#869)

2019-02-28T15:59:17+00:00

The `specializeProgramLayout()` primarily existed to support global generic parameters. The guiding idea of the design had been that plugging in concrete types for global generic parameters should only affect the layout/binding of shader parameters that depend on the global generic type parameters. All other shader parameters should keep the same layout/location across all specializations. This idea was implemented by conceptually having two phases of layout: * A generic-argument-independent phase would do layout on all the shader parameters that *don't* depend on global generic type parameters. * A second phase would then pick up where the other one left off (re-using existing parameter layouts to guarantee a match), and just layout out the shader parameters that *do* depend on global generic type parameters. Because the other parameters were already laid out, these new parameters would only ever fill in the gaps in layout (or come after all the other parameters, if explicit bindings aren't used). This implementation strategy proved to be a bit of a mess, since we had to duplicate most of the code between the two passes anyway. This commit eliminates `specializeProgramLayout()` entirely, and instead threads through global generic type arguments as part of the main layout pass. It is almost strictly a cleanup pass, now that the refactored logic for `Program` means the same layout algorithm can apply to specialized and unspecialized programs. This change has one important semantic consequence (which is technically a break in backwards compatiblity for anybody using global generic parameters). Parameters that depend on global generic type parameters now get laid out in declaration order, just like all other shader parameters. This simplifies the rules, and in my experience actually makes application code *easier* to write in a systematic way (whereas the original design was motivated by the idea that giving more things stable locations would be beneficial). A future improvement could be made so that we don't thread through the global generic substitution as part of layout. Instead, we could just attach a list of the global generic type arguments directly to the `TypeLayoutContext` and look those up on-demand when we encounter a global generic parameter type during layout. This would actually eliminate the need for global generics to appear as a `Substitution` entirely.

Simplify type layout (#867)

2019-02-27T22:58:02+00:00

* Make vector/matrix type layouts include element type layouts Previously the `MatrixTypeLayout` class was a leaf node in the layout hierarchy, and vector types just used `TypeLayout` with no further refinement. This change adds a `VectorTypeLayout`, and makes all of vector, matrix, and array types inherit form a common base class for `SequenceTypeLayout`s. The actual layout computation logic was updated to compute layouts for the element types of vectors, and for the row (and element) types of matrices. Notes: * Because of the way varying input/output parameters are being handled, their type layouts won't include this new information, and they will just use `TypeLayout`. This was true even for the matrix case before. * I made the design choice in this change to have a matrix type always treat rows as the logical element type (since that is what is surfaced to ther user in the HLSL syntax). We could potentially make our lives easier during layout computation if we made the element type of a `MatrixTypeLayout` depend on the row-/column-major layout choice, but that would then require any algorithm that uses the new layout information to take row-vs-column-major into account. * No code is actually *using* this new information yet, although the work in `ir-union.cpp` could probably benefit from it. The main expected use case going forward is representing constant buffers as a "bag of bits." * Add a specialized type layout approach for varying parameters There is a lot of complexity in `GetLayoutImpl` because it needs to service both the "normal" case, which always wants a `TypeLayout` object to be returned, and the varying parameter case, where we currently don't care about getting back a `TypeLayout` object. Confusingly, the varying parameter layout logic actually *does* construct `TypeLayout` objects, and it just does it inside of `parameter-binding.cpp` rather than in `type-layout.cpp`. That logic cannot (easily) be shared with the `GetLayoutImpl` path because: * The varying case needs to deal with system-value semantics and also parameters that may be inputs, outputs, or both (so that they need to combine resource usage computed for inputs and outputs). * The varying case needs to special-case vectors (and to a lesser extent matrices) because of the quirks of uniform layout (e.g., four `float` varying inputs consume four `locations`, but a `float4` consumes only one location). This change introduces a customized layout function just for varying parameters, that only handles the scalar, vector, and matrix cases (since `parameter-binding.cpp` will have recursed through any strucures/arrays, and should error out on any other types that are illegal in varying parameter lists). In the long run we could consider trying to deduplicate code and share more of this logic with `GetLayoutImpl`, but that would require a more significant refactoring of type layout, which should probably wait until we are doing layout on IR types. * Rename CreateTypeLayout to createTypeLayout This is just a fixup to better reflect our established naming conventions. * Simplify type layout so that it always returns a layout object The core `GetLayoutImpl` routine included a fair bit of complexity to deal with the fact that its `outTypeLayout` parameter was optional. The caller could pass in null to say that it doesn't want a `TypeLayout` object to be constructed, and the routine would conditionalize a lot of its logic to deal with this case. This change simplifies the logic so that a `TypeLayout` is always constructed and returned. Instead of using a combination of a function result (for the `SimpleLayoutInfo`) and an output parameter (for the `TypeLayout`) we use a new `TypeLayoutResult` that acts as a tuple over the two. I had hoped for a more significant cleanup by also eliminating the need to return the `SimpleLayoutInfo` separately from the `TypeLayout`, but the simple layout info is what the underlying per-API/-context "rules" implementations use (so that they can avoid all the complexity of `TypeLayout`), and refactoring to derive the simple layout infor from a computed `TypeLayout` would be a bigger refactoring than I was ready for. * fixup: typos