Improve support for interfaces as shader parameters (#886)

* Improve support for interfaces as shader parameters

This change adds two main things over the existing support:

1. It is now possible to plug in concrete types that actually contain (uniform/ordinary) fields for the existential type parameters introduced by interface-type shader parameters. The `interface-shader-param2.slang` test shows that this works.

2. There is a limited amount of support for doing correct layout computation and generating output code that matches that layout, so that interface and ordinary-type fields can be interleaved to a limited extent. The `interface-shader-param3.slang` test confirms this behavior.

There are several moving pieces in the change.

* When it comes to terminology, we try to draw a more clear distinction between existial type parameters/arguments and existential/object value parametes/arguments. A simple way to look at it is that an `IFoo[3]` shader parameter introduces a single existential type parameter (so that a concrete type argument like `SomeThing` can be plugged in for the `IFoo`) but introduces three existential object/value parameters (to represent the concrete values for the array elements).

* At the IR level, we support a few new operations. A `BindExistentialsType` can take a type that is not itself an interface/existential type but which depends on interfaces/existentials (e.g., `ConstantBuffer<IFoo>`) and plug in the concrete types to be used for its existential type slots.

* Then a `wrapExistentials` instruction can take a type with all the existentials plugged in (possibly by `BindExistentialsType`) and wrap it into a value of the existential-using type (e.g., turn `ConstantBuffer<SomeThing>` into a `ConstantBuffer<IFoo>`).

* The IR passes for doing generic/existential specialization have been updated to be able to desugar uses of these new operations just enough so that a `ConstantBuffer<IFoo>` can be used.

* When we specialize an IR parameter of an interface type like `IFoo` based on a concrete type `SomeThing`, we turn the parameter into an `ExistentialBox<SomeThing>` to reflect the fact that we are conceptually referring to `SomeThing` indirectly (it shouldn't be factored into the layout of its surrounding type).

* Parameter binding was updated so that it passes along the bound existential type arguments in a `Program` or `EntryPoint` to type layout, so that we can take them into account. The type layout code needs to do a little work to pass the appropriate range of arguments along to sub-fields when computing layout for aggregate types.

* Type layout was updated to have a notion of "pending" items, which represent the concrete types of data that are logically being referenced by existential value slots. The basic idea is that these values aren't included in the layout of a type by default, but then they get "flushed" to come after all the non-existential-related data in a constant buffer, parameter block, etc.

* The logic for computing a parameter group (`ConstantBuffer` or `ParameterBlock`) layout was updated to always "flush" the pending items on the element type of the group, so that the resource usage of specialized existential slots would be taken into account.

* The type legalization pass has been adapted so that we can derive two different passes from it. One does resource-type legalization (which is all that the original pass did). The new pass uses the same basic machinery to legalize `ExistentialBox<T>` types by moving them out of their containing type(s), and then turning them into ordinary variables/parameters of type `T`.

Big things missing from this change include:

- Nothing is making sure that "pending" items at the global or entry-point level will get proper registers/bindings allocated to them. For the uniform case, all that matters in the current compiler is that we declare them in the right order in the output HLSL/GLSL, but for resources to be supported we will need to compute this layout information and start associating it with the existential/interface-type fields.

- Nothing is being done to support `BindExistentials<S, ...>` where `S` is a `struct` type that might have existential-type fields (or nested fields...). Eventually we need to desugar a type like this into a fresh `struct` type that has the same field keys as `S`, but with fields replaced by suitable `BindExistentials` as needed. (The hard part of this would seem to be computing which slots go to which fields). As a practial matter, this missing feature means that interface-type members of `cbuffer` declarations won't work.

The current tests carefully avoid both of these problems. They don't declare any buffer/texture fields in the concrete types, and they don't make use of `cbuffer` declarations or `ConstantBuffer`s over structure types with interface-type fields.

* fixup: add override to methods

* fixup: typos

1 files changed, 160 insertions, 14 deletions

diff --git a/source/slang/type-layout.cpp b/source/slang/type-layout.cpp
index ea5287999..77588036d 100644
--- a/source/slang/type-layout.cpp
+++ b/source/slang/type-layout.cpp
@@ -973,10 +973,6 @@ static SimpleLayoutInfo getParameterGroupLayoutInfo(
     }
 }
 
-RefPtr<TypeLayout> createTypeLayout(
-    TypeLayoutContext const&    context,
-    Type*                       type);
-
 static bool isOpenGLTarget(TargetRequest*)
 {
     // We aren't officially supporting OpenGL right now
@@ -1184,12 +1180,76 @@ RefPtr<TypeLayout> applyOffsetToTypeLayout(
     return newTypeLayout;
 }
 
+    /// Take a type layout that might include pending items and fold them into the layout.
+static RefPtr<TypeLayout> flushPendingItems(
+    TypeLayoutContext const&    context,
+    RefPtr<TypeLayout>          layout)
+{
+    SLANG_UNUSED(context);
+
+    // If there are no pending items on the layout,
+    // then there is nothing to be done.
+    //
+    if(layout->pendingItems.Count() == 0)
+        return layout;
+
+    // We need to compute a new type layout that reflects
+    // the resource usage of the provided `layout`, plus
+    // any resource usage for the pending items.
+    //
+    // TODO: To be correct we should construct a new `TypeLayout`
+    // of the same class, but that would take more work, so
+    // we'll re-use the one we already have... kind of gross...
+    //
+    for( auto pendingItem : layout->pendingItems )
+    {
+        auto itemTypeLayout = pendingItem.getTypeLayout();
+
+        // Any resources used by a pending item should be
+        // billed against the flushed layout we are computing.
+        //
+        // TODO: We need to make this handlde ordinary/uniform
+        // data carefully, so that it respects alignment and
+        // other layout rules for the target.
+        //
+        // TODO: We should only be adding in resource usage
+        // that can be "hidden" by the type of parameter block
+        // being built (e.g., only a `ParameterBlock` that allocates
+        // full `set`s/`space`s can hide the `register`s/`binding`s
+        // used by resource fields).
+        //
+        // TODO: we need to write something back to the item,
+        // which should have a `VarLayout` or something like
+        // that attached to it!
+        //
+        for( auto resInfo : itemTypeLayout->resourceInfos )
+        {
+            layout->addResourceUsage(resInfo);
+        }
+    }
+    layout->pendingItems.Clear();
+
+    return layout;
+}
+
 static RefPtr<ParameterGroupTypeLayout> _createParameterGroupTypeLayout(
     TypeLayoutContext const&    context,
     RefPtr<ParameterGroupType>  parameterGroupType,
     SimpleLayoutInfo            parameterGroupInfo,
     RefPtr<TypeLayout>          rawElementTypeLayout)
 {
+    // If there are any "pending" items that need to be laid out in
+    // the element type of the parameter group, then we want to flush
+    // them here.
+    //
+    // TODO: We might need to make this only flush *parts* of the pending
+    // items, based on what the parameter group can absorb, and leave
+    // other parts still pending in the type layout we return...
+    //
+    rawElementTypeLayout = flushPendingItems(
+        context.with(context.getRulesFamily()->getConstantBufferRules()),
+        rawElementTypeLayout);
+
     auto parameterGroupRules = context.rules;
 
     RefPtr<ParameterGroupTypeLayout> typeLayout = new ParameterGroupTypeLayout();
@@ -1333,7 +1393,7 @@ static RefPtr<ParameterGroupTypeLayout> _createParameterGroupTypeLayout(
     {
         // A parameter block type that gets its own register space will only
         // include resource usage from the element type when it itself consumes
-        // while register spaces.
+        // whole register spaces.
         if (auto elementResInfo = rawElementTypeLayout->FindResourceInfo(LayoutResourceKind::RegisterSpace))
         {
             typeLayout->addResourceUsage(*elementResInfo);
@@ -1378,14 +1438,33 @@ static RefPtr<ParameterGroupTypeLayout> _createParameterGroupTypeLayout(
     return typeLayout;
 }
 
-    /// Doe we need to wrap the given element type in a constant buffer layout?
+static bool usesResourceKind(RefPtr<TypeLayout> typeLayout, LayoutResourceKind kind)
+{
+    auto resInfo = typeLayout->FindResourceInfo(kind);
+    return resInfo && resInfo->count != 0;
+}
+
+static bool usesOrdinaryData(RefPtr<TypeLayout> typeLayout)
+{
+    return usesResourceKind(typeLayout, LayoutResourceKind::Uniform);
+}
+
+    /// Do we need to wrap the given element type in a constant buffer layout?
 static bool needsConstantBuffer(RefPtr<TypeLayout> elementTypeLayout)
 {
-    auto uniformResInfo = elementTypeLayout->FindResourceInfo(LayoutResourceKind::Uniform);
-    if(uniformResInfo && uniformResInfo->count != 0)
+    // We need a constant buffer if the element type has ordinary/uniform data.
+    //
+    if(usesOrdinaryData(elementTypeLayout))
         return true;
 
-    // Note: additional cases are expected here, to deal with existential types.
+    // We also need a constant buffer if there are any "pending"
+    // items that need ordinary/uniform data allocated to them.
+    //
+    for( auto pendingItem : elementTypeLayout->pendingItems )
+    {
+        if(usesOrdinaryData(pendingItem.getTypeLayout()))
+            return true;
+    }
 
     return false;
 }
@@ -2302,8 +2381,32 @@ static TypeLayoutResult _createTypeLayout(
 
             for (auto field : GetFields(structDeclRef))
             {
+                // Static fields shouldn't take part in layout.
+                if(field.getDecl()->HasModifier<HLSLStaticModifier>())
+                    continue;
+
+                // The fields of a `struct` type may include existential (interface)
+                // types (including as nested sub-fields), and any types present
+                // in those fields will need to be specialized based on the
+                // input arguments being passed to `_createTypeLayout`.
+                //
+                // We won't know how many type slots each field consumes until
+                // we process it, but we can figure out the starting index for
+                // the slots its will consume by looking at the layout we've
+                // computed so far.
+                //
+                Int baseExistentialSlotIndex = 0;
+                if(auto resInfo = typeLayout->FindResourceInfo(LayoutResourceKind::ExistentialTypeParam))
+                    baseExistentialSlotIndex = Int(resInfo->count.getFiniteValue());
+                //
+                // When computing the layout for the field, we will give it access
+                // to all the incoming specialized type slots that haven't already
+                // been consumed/claimed by preceding fields.
+                //
+                auto fieldLayoutContext = context.withExistentialTypeSlotsOffsetBy(baseExistentialSlotIndex);
+
                 TypeLayoutResult fieldResult = _createTypeLayout(
-                    context,
+                    fieldLayoutContext,
                     GetType(field).Ptr(),
                     field.getDecl());
                 RefPtr<TypeLayout> fieldTypeLayout = fieldResult.layout;
@@ -2384,6 +2487,19 @@ static TypeLayoutResult _createTypeLayout(
                         structTypeResourceInfo->count += fieldTypeResourceInfo.count;
                     }
                 }
+
+                // Fields of a structure type may have existential/interface type,
+                // or nested existential/interface-type fields. When doing layout
+                // for a specialized program, these will show up as "pending" types
+                // that need to be laid out at the end of the surrounding block/container.
+                //
+                // Any pending types on fields of a structure become pending types
+                // on the structure itself.
+                //
+                for( auto pendingItem : fieldTypeLayout->pendingItems )
+                {
+                    typeLayout->pendingItems.Add(pendingItem);
+                }
             }
 
             rules->EndStructLayout(&info);
@@ -2446,10 +2562,35 @@ static TypeLayoutResult _createTypeLayout(
             // represents the indirections needed to reference the
             // data to be referenced by this field.
             //
-            return createSimpleTypeLayout(
-                SimpleLayoutInfo(LayoutResourceKind::ExistentialSlot, 1),
-                type,
-                rules);
+
+            RefPtr<TypeLayout> typeLayout = new TypeLayout();
+            typeLayout->type = type;
+            typeLayout->rules = rules;
+
+            typeLayout->addResourceUsage(LayoutResourceKind::ExistentialTypeParam, 1);
+            typeLayout->addResourceUsage(LayoutResourceKind::ExistentialObjectParam, 1);
+
+            // If there are any concrete types available, the first one will be
+            // the value that should be plugged into the slot we just introduced.
+            //
+            if( context.existentialTypeArgCount )
+            {
+                RefPtr<Type> concreteType = context.existentialTypeArgs[0].type;
+
+                RefPtr<TypeLayout> concreteTypeLayout = createTypeLayout(context, concreteType);
+
+                // Layout for this specialized interface type then results
+                // in a type layout that tracks both the resource usage of the
+                // interface type itself (just the type + value slots introduced
+                // above), plus a "pending" type that represents the value
+                // conceptually pointed to by the interface-type field/variable at runtime.
+                //
+                TypeLayout::PendingItem pendingItem;
+                pendingItem.typeLayout = concreteTypeLayout;
+                typeLayout->pendingItems.Add(pendingItem);
+            }
+
+            return TypeLayoutResult(typeLayout, SimpleLayoutInfo());
         }
     }
     else if (auto errorType = as<ErrorType>(type))
@@ -2487,6 +2628,11 @@ static TypeLayoutResult _createTypeLayout(
         //
         for( auto caseType : taggedUnionType->caseTypes )
         {
+            // Note: A tagged union type is not expected to have any existential/interface type
+            // slots; the case types that are provided must be fully specialized before the union is
+            // formed. Thus we don't need to mess around with existential type slots here the
+            // way we do for the `struct` case.
+
             auto caseTypeResult = _createTypeLayout(context, caseType);
             RefPtr<TypeLayout> caseTypeLayout = caseTypeResult.layout;
             UniformLayoutInfo caseTypeInfo = caseTypeResult.info.getUniformLayout();