Allow plugging in types with resources for interface parameters (#913)

* Allow plugging in types with resources for interface parameters

The key feature enabled by this change is that you can take a shader declared with interface-type parameters:

```hlsl
ConstantBuffer<ILight> gLight;
float4 myShader(IMaterial material, ...)
{ ... }
```

and specialize its interface-type parameters to concrete type that can contain resources like textures, samplers, etc.

The hard part of doing this layout is that we need to support signatures that include a mix of interface and non-interface types. Imagine this contrived example:

```hlsl
float4 myShader(
    Texture2D diffuseMap,
    ILight        light,
    Texture2D specularMap)
{ ... }
```

We end up wanting `diffuseMap` to get `register(t0)` and `specularMap` to get `register(t1)`, so that they have the same location no matter what we plug in for `light`.
But if we plug in a concrete type for `light` that needs a texture register, we need to allocate it *somewhere*.
We handle this by having the `TypeLayout` for `light` come back with a "primary" type layout that doesn't have any texture registers, but with a "pending" type layout that includes the texture register requirements of whatever concrete type we plug in.

This split between "primary" and "pending" layout then needs to work its way up the hierarchy, so that an aggregate `struct` type with a mix of interface and non-interface fields (recursively), needs to compute an aggregate "primary type layout" and an aggregate "pending type layout," and then each field needs to be able to compute its offset in the primary/pending layout of the aggregate.

A large chunk of the work in this PR is then just implementing the split between primary and pending data, and ensuring that layouts are computed appropriately.

The next catch is that when a "parameter group" (either a parameter block or constant buffer) contains one or more values of interface type, then we can allow the parameter group to "mask" some of the resource usage of the concrete types we plug in, but others "bleed through."

For example, if we have:

```hlsl
struct MyStuff { float3 color; ILight light; }
ConstantBuffer<MyStuff> myStuff;

struct SpotLight { float3 position; Texture2D shadowMap; }
``

If we plug in the `SpotLight` type for `myStuff.light`, then the `float3` data for the light can be "masked" by the fact that we have a constant buffer (we can just allocate the `float3` `position` right after `color`), but the `Texture2D` needed for `shadowMap` needs to "bleed through" and become "pending" data for the `myStuff` shader parameter.

Adding support for that detail more or less required a full rewrite of the logic for allocating parameter group type layouts.

The next detail is that when we go to legalize a declaration like the `myStuff` buffer, we will end up with something like:

```hlsl
struct MyStuff_stripped { float3 color; }
struct Wrapped
{
    MyStuff_stripped primary;
    SpotLight pending;
}
ConstantBuffer<Wrapped> myStuff;
```

This "wrapped" version of the buffer type more accurately reflects the layout we need/want for the uniform/ordinary data, but in order to further legalize it and pull out the resource-type fields like `shadowMap` we need to have accurate layout information, and the problem is that layout information for the original buffer can't apply to this new "wrapped" buffer.

The last major piece of this change is logic that runs during existential type legalization to compute new layouts for "wrapped" buffers like these that embeds correct offset/binding/register information for any resources nested inside them. A key challenge in that code is that existential legalization needs to erase any "pending" data from the program entirely, so that offset information that used to be relatie to the "pending" part of a surrounding type now needs to be relative to the primary part.

The work here may not be 100% complete for all scenarios, but it does well enough on the new and existing tests that I want to checkpoint it. Note that a few other tests have had their output changed, but in all cases I've reviewed the diffs and determined that the change in observable behavior is consistent with what we intened Slang's behavior to be.

Note that there is still one major piece of support for interface-type parameters that is missing here, and which might force us to revisit some of the decisions in this code: we don't properly support user-defined `struct` types with interface-type fields.

* fixup: typos

1 files changed, 128 insertions, 13 deletions

diff --git a/source/slang/type-layout.h b/source/slang/type-layout.h
index cdeadbb16..5ac9229b4 100644
--- a/source/slang/type-layout.h
+++ b/source/slang/type-layout.h
@@ -328,23 +328,20 @@ public:
     // the space storing it in the above array
     UInt uniformAlignment = 1;
 
-        /// An item that is conceptually owned by this type, but is pending layout.
+
+        /// The layout for data that is conceptually owned by this type, but which is pending layout.
         ///
         /// When a type contains interface/existential fields (recursively), the
         /// actual data referenced by these fields needs to get allocated somewhere,
         /// but it cannot go inline at the point where the interface/existential
-        /// type appears, or else 
+        /// type appears, or else the layout of a composite object would change
+        /// when the concrete type(s) we plug in change.
         ///
-    struct PendingItem
-    {
-        RefPtr<TypeLayout> typeLayout;
-
-        RefPtr<TypeLayout> getTypeLayout() const { return typeLayout; }
-        RefPtr<Type> getType() const { return typeLayout->type; }
-    };
-
-        /// The pending items owned by this type, but which are pending layout.
-    List<PendingItem> pendingItems;
+        /// We solve this problem by tracking this data that is "pending" layout,
+        /// and then "flushing" the pending data at appropriate places during
+        /// the layout process.
+        ///
+    RefPtr<TypeLayout> pendingDataTypeLayout;
 
     ResourceInfo* FindResourceInfo(LayoutResourceKind kind)
     {
@@ -383,6 +380,8 @@ public:
         addResourceUsage(info);
     }
 
+    void addResourceUsageFrom(TypeLayout* otherTypeLayout);
+
         /// "Unwrap" any layers of array-ness from this type layout.
         ///
         /// If this is an `ArrayTypeLayout`, returns the result of unwrapping the element type layout.
@@ -475,6 +474,8 @@ public:
 
         return AddResourceInfo(kind);
     }
+
+    RefPtr<VarLayout> pendingVarLayout;
 };
 
 // type layout for a variable that has a constant-buffer type
@@ -501,6 +502,16 @@ public:
     // so that any fields (if the element type is a `struct`)
     // will be offset by the resource usage of the container.
     RefPtr<TypeLayout>  offsetElementTypeLayout;
+
+    // If the element type layout had any "pending" data, then
+    // as much of that data as possible will be flushed to
+    // fit into the overall layout of the parameter group.
+    //
+    // This field stores the offset information for where
+    // the pending data got stored relative to the start of
+    // the group.
+    //
+//    RefPtr<VarLayout> flushedDataVarLayout;
 };
 
 // type layout for a variable that has a constant-buffer type
@@ -585,6 +596,12 @@ public:
     // in the array above, rather than to the actual pointer,
     // so that we 
     Dictionary<Decl*, RefPtr<VarLayout>> mapVarToLayout;
+
+    // As an accellerator for type layouts created at the
+    // IR layer, we include a second map that use IR "key"
+    // instructions to map to fields.
+    //
+    Dictionary<IRInst*, RefPtr<VarLayout>> mapKeyToLayout;
 };
 
 class GenericParamTypeLayout : public TypeLayout
@@ -924,6 +941,98 @@ struct TypeLayoutContext
     }
 };
 
+//
+
+    /// A custom tuple to capture the outputs of type layout
+struct TypeLayoutResult
+{
+        /// The actual heap-allocated layout object with all the details
+    RefPtr<TypeLayout>  layout;
+
+        /// A simplified representation of layout information.
+        ///
+        /// This information is suitable for the case where a type only
+        /// consumes a single resource.
+        ///
+    SimpleLayoutInfo    info;
+
+        /// Default constructor.
+    TypeLayoutResult()
+    {}
+
+        /// Construct a result from the given layout object and simple layout info.
+    TypeLayoutResult(RefPtr<TypeLayout> inLayout, SimpleLayoutInfo const& inInfo)
+        : layout(inLayout)
+        , info(inInfo)
+    {}
+};
+
+    /// Helper type for building `struct` type layouts
+struct StructTypeLayoutBuilder
+{
+public:
+        /// Begin the layout process for `type`, using `rules`
+    void beginLayout(
+        Type*               type,
+        LayoutRulesImpl*    rules);
+
+        /// Begin the layout process for `type`, using `rules`, if it hasn't already been begun.
+        ///
+        /// This functions allows for a `StructTypeLayoutBuilder` to be use lazily,
+        /// only allocating a type layout object if it is actaully needed.
+        ///
+    void beginLayoutIfNeeded(
+        Type*               type,
+        LayoutRulesImpl*    rules);
+
+        /// Add a field to the struct type layout.
+        ///
+        /// One of the `beginLayout*()` functions must have been called previously.
+        ///
+    RefPtr<VarLayout> addField(
+        DeclRef<VarDeclBase>    field,
+        TypeLayoutResult        fieldResult);
+
+        /// Add a field to the struct type layout.
+        ///
+        /// One of the `beginLayout*()` functions must have been called previously.
+        ///
+    RefPtr<VarLayout> addField(
+        DeclRef<VarDeclBase>    field,
+        RefPtr<TypeLayout>      fieldTypeLayout);
+
+        /// Complete layout.
+        ///
+        /// If layout was begun, ensures that the result of `getTypeLayout()` is usable.
+        /// If layout was never begin, does nothing.
+        ///
+    void endLayout();
+
+        /// Get the type layout.
+        ///
+        /// This can be called any time after `beginLayout*()`.
+        /// In particular, it can be called before `endLayout`.
+        ///
+    RefPtr<StructTypeLayout> getTypeLayout();
+
+        /// The the type layout result.
+        ///
+        /// This is primarily useful for implementation code in `_createTypeLayout`.
+        ///
+    TypeLayoutResult getTypeLayoutResult();
+
+private:
+        /// The layout rules being used, if layout has begun.
+    LayoutRulesImpl* m_rules = nullptr;
+
+        /// The type layout being computed, if layout has begun.
+    RefPtr<StructTypeLayout> m_typeLayout;
+
+        /// Uniform offset/alignment statte used when computing offset for uniform fields.
+    UniformLayoutInfo m_info;
+};
+
+//
 
 // Get an appropriate set of layout rules (packaged up
 // as a `TypeLayoutContext`) to perform type layout
@@ -963,7 +1072,7 @@ RefPtr<TypeLayout> createTypeLayout(
 //
 
     /// Create a layout for a parameter-group type (a `ConstantBuffer` or `ParameterBlock`).
-RefPtr<ParameterGroupTypeLayout> createParameterGroupTypeLayout(
+RefPtr<TypeLayout> createParameterGroupTypeLayout(
     TypeLayoutContext const&    context,
     RefPtr<ParameterGroupType>  parameterGroupType);
 
@@ -993,6 +1102,12 @@ createStructuredBufferTypeLayout(
 int findGenericParam(List<RefPtr<GenericParamLayout>> & genericParameters, GlobalGenericParamDecl * decl);
 //
 
+// Given an existing type layout `oldTypeLayout`, apply offsets
+// to any contained fields based on the resource infos in `offsetVarLayout`.
+RefPtr<TypeLayout> applyOffsetToTypeLayout(
+    RefPtr<TypeLayout>  oldTypeLayout,
+    RefPtr<VarLayout>   offsetVarLayout);
+
 }
 
 #endif