Basic layout and reflection for specialized types (#970)

* Basic layout and reflection for specialized types

Suppose I have an interface, and a simple implementation of it:

```hlsl
interface IModifier
{
    float modify(float value);
}

struct Doubler : IModifier
{
    float modify(float value) { return 2 * value; }
}
```

SAnd now suppose I want to define an implementation that recursively uses the same interface:

```hlsl
struct MultiModifier : IModifier
{
    IModifier first;
    IModifier second;

    float modify(float value)
    {
        value = first.modify(value);
        value = second.modify(value);
        return value;
    }
}
```

And now consider that I might have a generic entry point that uses the interface:

```hlsl
void myShader<M : IModifier>( uniform M modifier, ... )
{ ... }
```

I can easily specialize `myShader` for `M = Doubler`, but in order to specialze it for `M = MultiModifier` I need a way to specify what the types of `MultiModifier.first` and `.second` should be.

That is what the `spReflection_specializeType` function is used to do: take a type like `MultiModifier` and specialize it for, say, `first : Doubler` and `second : Doubler`. That function creates an `ExistentialSpecializedType` that records the base type (`MultiModifier`) and the specialization arguments (the concrete types plus the witness tables that prove they implement the required interfaces).

The change that introduced that logic neglected to include an implementation of type layout for `ExistentialSpecializedType`, and also didn't add any support for the new kind of type through the reflection API. That is what this change seeks to rectify.

When it comes to layout, a specialized type neeeds to apply layout to its base type (e.g., `MultiModifier`) with the appropriate existential type "slot" arguments bound, which luckily is stuff that type layout already supporst (to handle specialization of interface-type shader parameters).

Unlike the case for interface-type shader parameters where the "primary" and "pending" data for a type get propagated up the chain and allocated to different places, a specialized type should be allocated contiguously (e.g., `myShader<M>` is going to assume that the type `M` occupies a contiguous range in memory). The type layout for a specialized type thus computes a layout that is more-or-less a structure type consisting of the "primary" data followed by the "pending" data. This gets wrapped up in a new `ExistentialSpecializedTypeLayout` class.

The reflection API then needs to expose an `ExistentialSpecializedTypeLayout` as a new kind of type, and then also provide access to the relevant pieces. For the "base" type, I went ahead and re-used the `getElementType` entry point, just for simplicity (we can debate whether that or a new entry point is more appropriate/convenient). For the actual layout, all that was needed was a way to query the offset for where the "pending" data gets placed, and that is already conveniently encoded as a `VarLayout` field in the `ExistentialSpecializedTypeLayout`.

With this change, specialized types are closer to being truly usable, although there is still missing logic in IR lowering because we need to make sure that explicitly specialized types are represented differently from types that are specialized based on global shader parameters.

* fixup: review feedback

4 files changed, 93 insertions, 2 deletions

diff --git a/slang.h b/slang.h
index 6175d419e..4af17905c 100644
--- a/slang.h
+++ b/slang.h
@@ -1593,6 +1593,7 @@ extern "C"
         SLANG_TYPE_KIND_GENERIC_TYPE_PARAMETER,
         SLANG_TYPE_KIND_INTERFACE,
         SLANG_TYPE_KIND_OUTPUT_STREAM,
+        SLANG_TYPE_KIND_SPECIALIZED,
         SLANG_TYPE_KIND_COUNT,
     };
 
@@ -1814,6 +1815,8 @@ extern "C"
 
     SLANG_API SlangReflectionTypeLayout* spReflectionTypeLayout_getPendingDataTypeLayout(SlangReflectionTypeLayout* type);
 
+    SLANG_API SlangReflectionVariableLayout* spReflectionTypeLayout_getSpecializedTypePendingDataVarLayout(SlangReflectionTypeLayout* type);
+
     // Variable Reflection
 
     SLANG_API char const* spReflectionVariable_GetName(SlangReflectionVariable* var);
@@ -1970,7 +1973,8 @@ namespace slang
             ShaderStorageBuffer = SLANG_TYPE_KIND_SHADER_STORAGE_BUFFER,
             ParameterBlock = SLANG_TYPE_KIND_PARAMETER_BLOCK,
             GenericTypeParameter = SLANG_TYPE_KIND_GENERIC_TYPE_PARAMETER,
-            Interface = SLANG_TYPE_KIND_INTERFACE
+            Interface = SLANG_TYPE_KIND_INTERFACE,
+            Specialized = SLANG_TYPE_KIND_SPECIALIZED,
         };
 
         enum ScalarType : SlangScalarType
@@ -2252,6 +2256,11 @@ namespace slang
                 (SlangReflectionTypeLayout*) this);
         }
 
+        VariableLayoutReflection* getSpecializedTypePendingDataVarLayout()
+        {
+            return (VariableLayoutReflection*) spReflectionTypeLayout_getSpecializedTypePendingDataVarLayout(
+                (SlangReflectionTypeLayout*) this);
+        }
     };
 
     struct Modifier
diff --git a/source/slang/reflection.cpp b/source/slang/reflection.cpp
index 4ac48d2e7..4d13052f6 100644
--- a/source/slang/reflection.cpp
+++ b/source/slang/reflection.cpp
@@ -279,6 +279,10 @@ SLANG_API SlangTypeKind spReflectionType_GetKind(SlangReflectionType* inType)
             return SLANG_TYPE_KIND_INTERFACE;
         }
     }
+    else if( auto specializedType = as<ExistentialSpecializedType>(type) )
+    {
+        return SLANG_TYPE_KIND_SPECIALIZED;
+    }
     else if (auto errorType = as<ErrorType>(type))
     {
         // This means we saw a type we didn't understand in the user's code
@@ -746,6 +750,10 @@ SLANG_API SlangReflectionTypeLayout* spReflectionTypeLayout_GetElementTypeLayout
     {
         return convert(structuredBufferTypeLayout->elementTypeLayout.Ptr());
     }
+    else if( auto specializedTypeLayout = as<ExistentialSpecializedTypeLayout>(typeLayout) )
+    {
+        return convert(specializedTypeLayout->baseTypeLayout.Ptr());
+    }
 
     return nullptr;
 }
@@ -878,6 +886,22 @@ SLANG_API SlangReflectionVariableLayout* spReflectionVariableLayout_getPendingDa
     return convert(pendingDataLayout);
 }
 
+SLANG_API SlangReflectionVariableLayout* spReflectionTypeLayout_getSpecializedTypePendingDataVarLayout(SlangReflectionTypeLayout* inTypeLayout)
+{
+    auto typeLayout = convert(inTypeLayout);
+    if(!typeLayout) return nullptr;
+
+    if( auto specializedTypeLayout = as<ExistentialSpecializedTypeLayout>(typeLayout) )
+    {
+        auto pendingDataVarLayout = specializedTypeLayout->pendingDataVarLayout.Ptr();
+        return convert(pendingDataVarLayout);
+    }
+    else
+    {
+        return nullptr;
+    }
+}
+
 
 // Variable Reflection
 
diff --git a/source/slang/type-layout.cpp b/source/slang/type-layout.cpp
index e315c2fb6..ba6085e0a 100644
--- a/source/slang/type-layout.cpp
+++ b/source/slang/type-layout.cpp
@@ -2210,7 +2210,11 @@ RefPtr<VarLayout> StructTypeLayoutBuilder::addField(
     fieldLayout->varDecl = field;
     fieldLayout->typeLayout = fieldTypeLayout;
     m_typeLayout->fields.add(fieldLayout);
-    m_typeLayout->mapVarToLayout.Add(field.getDecl(), fieldLayout);
+
+    if( field )
+    {
+        m_typeLayout->mapVarToLayout.Add(field.getDecl(), fieldLayout);
+    }
 
     // Set up uniform offset information, if there is any uniform data in the field
     if( fieldTypeLayout->FindResourceInfo(LayoutResourceKind::Uniform) )
@@ -2948,6 +2952,52 @@ static TypeLayoutResult _createTypeLayout(
 
         return TypeLayoutResult(taggedUnionLayout, info);
     }
+    else if( auto existentialSpecializedType = as<ExistentialSpecializedType>(type) )
+    {
+        TypeLayoutContext subContext = context.withExistentialTypeArgs(
+            existentialSpecializedType->slots.args.getCount(),
+            existentialSpecializedType->slots.args.getBuffer());
+
+        auto baseTypeLayoutResult = _createTypeLayout(
+            subContext,
+            existentialSpecializedType->baseType);
+
+        UniformLayoutInfo info = rules->BeginStructLayout();
+        rules->AddStructField(&info, baseTypeLayoutResult.info.getUniformLayout());
+
+        RefPtr<ExistentialSpecializedTypeLayout> typeLayout = new ExistentialSpecializedTypeLayout();
+        typeLayout->type = type;
+        typeLayout->rules = rules;
+
+        RefPtr<VarLayout> pendingDataVarLayout = new VarLayout();
+        if(auto pendingDataTypeLayout = baseTypeLayoutResult.layout->pendingDataTypeLayout)
+        {
+            for( auto pendingResInfo : pendingDataTypeLayout->resourceInfos )
+            {
+                auto kind = pendingResInfo.kind;
+                UInt index = 0;
+                if( kind == LayoutResourceKind::Uniform )
+                {
+                    LayoutSize uniformOffset = rules->AddStructField(
+                        &info,
+                        makeTypeLayoutResult(pendingDataTypeLayout).info.getUniformLayout());
+
+                    index = uniformOffset.getFiniteValue();
+                }
+                else
+                {
+                    if(auto primaryResInfo = baseTypeLayoutResult.layout->FindResourceInfo(kind))
+                        index = primaryResInfo->count.getFiniteValue();
+                }
+                pendingDataVarLayout->AddResourceInfo(kind)->index = index;
+            }
+        }
+
+        typeLayout->baseTypeLayout = baseTypeLayoutResult.layout;
+        typeLayout->pendingDataVarLayout = pendingDataVarLayout;
+
+        return makeTypeLayoutResult(typeLayout);
+    }
 
     // catch-all case in case nothing matched
     SLANG_ASSERT(!"unimplemented case in type layout");
diff --git a/source/slang/type-layout.h b/source/slang/type-layout.h
index fcb2c3419..c58f92cfb 100644
--- a/source/slang/type-layout.h
+++ b/source/slang/type-layout.h
@@ -630,6 +630,14 @@ public:
     LayoutSize tagOffset;
 };
 
+    /// Layout information for a type with existential (sub-)field types specialized.
+class ExistentialSpecializedTypeLayout : public TypeLayout
+{
+public:
+    RefPtr<TypeLayout> baseTypeLayout;
+    RefPtr<VarLayout> pendingDataVarLayout;
+};
+
     /// Layout for a scoped entity like a program, module, or entry point
 class ScopeLayout : public Layout
 {