Fix parameter block binding for Vulkan (#308)

Fixes #307

This ends up being a major overhaul over how type layout computation is structured and exposed.
The big problems all arise around cases where both the "container" for a parameter block or CB, and the "element" type both use the same kind of resource.
E.g., if you define a CB with a texture in it, then in Vulkan both the CB and the texture use the same kind of resource, and so if you query the CB's resource usage it will just tell you it uses two descriptor-table slots, but nothing more than that.

Similar confusion still arises in the HLSL case, when a CB with a texture in it reports its parameter category as "mixed" so that a user might query for a category they didn't mean to. There were also cases in the existing code where a parameter block might expose *both* a register-space usage and another concrete resource type, which isn't right.

The most important changes here are:

- A `ParameterGroupTypeLayout` now has a more refined internal structure, consisting of:
  - A `containerTypeLayout`, which represents the resource usage of the buffer/block itself (e.g., if a constant buffer had to be allocated)
  - An `elementVarLayout` which stores the offsets that need to be applied to get from the `VarLayout` for an instance of this parameter-group type to the offsets of its elements. The `TypeLayout` for this variable layout should be the "raw" type of the block/CB element.
  - The `offsetElementTypeLayout` (formerly just `elementTypeLayout`) which represents the element type, but in the case of a `struct` element type, will have fields offset similar to the `elementVarLayout`. This is what all the old code used to use, so we need to keep it for compatibility.

- When doing reflection on a `ParameterGroupTypeLayout`, we now only report the resource usage of the `containerTypeLayout`. This is technically a potentially breaking change in the public API, but I don't think Falcor will mind, since they actually want something closer to this behavior.

- Add a new public API for querying the element variable layout of a parameter block of constant buffer. This could be used by savvy applications to fold the handling of CB element offsetting into some notion of a "reflection path." This would be required for applications that want to handle CBs or parameter blocks where the element type is *not* a `struct` type.

- Remove old logic for applying an offset when creating a type layout for constant buffer element, and instead perform offsetting more uniformly later, by constructing the `offsetElementTypeLayout` from the `rawElementTypeLayout`. This is useful both because we want to keep both (the "raw" type layout becomes the type layout of the `elementVarLayout`), and also because we can decide later whether we even want to allocate a CB register for a buffer, based on whether it actually contains any uniform data.

- Fix cases where we might end up with a parameter block type reporting both that it uses a whole register space (and thus should not expose the resource usage of the container/element type) *and* a constant-buffer register/slot. The latter should be hidden inside the regsiter space.

- Clean up the `spReflectionParameter_GetBinding{Index,Space}` functions to just route to `spReflectionVariableLayout_Get{Offset,Space}`, using the "default" category of the parameter

- Try to make the `GetSpace` query take into account cases where a variable also has an explicit `RegisterSpace` allocation.
  - This probably still needs some cleanup, since ideally we'd just move things into the `space` field on the `ReosurceInfo` and have an invariant that a variable *either* has a `RegisterSpace` allocation, or it has other resource infos, but never both...

- Add some ad-hoc logic so that if the user queries for a binding index/space using a parameter category that doesn't actually apply (e.g., they query for a D3D `t` register when using Vulkan), we can optionally remap it to the resource type they "probably" meant. This is a mess of Do What I Mean code, but it is also what our users want right now.

- Fix various bits of emit logic so that if a parameter block has a register space/set allocated to it, we properly output that as part of the binding information for it.
  - This is another thing that might be cleaned up if we rationale the way that things get split during legalization.

- Add a GLSL case for emitting a parameter block variable as a `cbuffer`.

1 files changed, 119 insertions, 23 deletions

diff --git a/source/slang/reflection.cpp b/source/slang/reflection.cpp
index 5962bce96..7068ecea4 100644
--- a/source/slang/reflection.cpp
+++ b/source/slang/reflection.cpp
@@ -529,7 +529,7 @@ SLANG_API SlangReflectionTypeLayout* spReflectionTypeLayout_GetElementTypeLayout
     }
     else if( auto constantBufferTypeLayout = dynamic_cast<ParameterGroupTypeLayout*>(typeLayout))
     {
-        return convert(constantBufferTypeLayout->elementTypeLayout.Ptr());
+        return convert(constantBufferTypeLayout->offsetElementTypeLayout.Ptr());
     }
     else if( auto structuredBufferTypeLayout = dynamic_cast<StructuredBufferTypeLayout*>(typeLayout))
     {
@@ -539,6 +539,19 @@ SLANG_API SlangReflectionTypeLayout* spReflectionTypeLayout_GetElementTypeLayout
     return nullptr;
 }
 
+SLANG_API SlangReflectionVariableLayout* spReflectionTypeLayout_GetElementVarLayout(SlangReflectionTypeLayout* inTypeLayout)
+{
+    auto typeLayout = convert(inTypeLayout);
+    if(!typeLayout) return nullptr;
+
+    if( auto constantBufferTypeLayout = dynamic_cast<ParameterGroupTypeLayout*>(typeLayout))
+    {
+        return convert(constantBufferTypeLayout->elementVarLayout.Ptr());
+    }
+
+    return nullptr;
+}
+
 static SlangParameterCategory getParameterCategory(
     LayoutResourceKind kind)
 {
@@ -566,6 +579,11 @@ SLANG_API SlangParameterCategory spReflectionTypeLayout_GetParameterCategory(Sla
     auto typeLayout = convert(inTypeLayout);
     if(!typeLayout) return SLANG_PARAMETER_CATEGORY_NONE;
 
+    if (auto parameterGroupTypeLayout = dynamic_cast<ParameterGroupTypeLayout*>(typeLayout))
+    {
+        typeLayout = parameterGroupTypeLayout->containerTypeLayout;
+    }
+
     return getParameterCategory(typeLayout);
 }
 
@@ -574,6 +592,11 @@ SLANG_API unsigned spReflectionTypeLayout_GetCategoryCount(SlangReflectionTypeLa
     auto typeLayout = convert(inTypeLayout);
     if(!typeLayout) return 0;
 
+    if (auto parameterGroupTypeLayout = dynamic_cast<ParameterGroupTypeLayout*>(typeLayout))
+    {
+        typeLayout = parameterGroupTypeLayout->containerTypeLayout;
+    }
+
     return (unsigned) typeLayout->resourceInfos.Count();
 }
 
@@ -582,6 +605,11 @@ SLANG_API SlangParameterCategory spReflectionTypeLayout_GetCategoryByIndex(Slang
     auto typeLayout = convert(inTypeLayout);
     if(!typeLayout) return SLANG_PARAMETER_CATEGORY_NONE;
 
+    if (auto parameterGroupTypeLayout = dynamic_cast<ParameterGroupTypeLayout*>(typeLayout))
+    {
+        typeLayout = parameterGroupTypeLayout->containerTypeLayout;
+    }
+
     return typeLayout->resourceInfos[index].kind;
 }
 
@@ -626,12 +654,68 @@ SLANG_API SlangReflectionTypeLayout* spReflectionVariableLayout_GetTypeLayout(Sl
     return convert(varLayout->getTypeLayout());
 }
 
+namespace Slang
+{
+    // Attempt "do what I mean" remapping from the parameter category the user asked about,
+    // over to a parameter category that they might have meant.
+    static SlangParameterCategory maybeRemapParameterCategory(
+        TypeLayout*             typeLayout,
+        SlangParameterCategory  category)
+    {
+        // Do we have an entry for the category they asked about? Then use that.
+        if (typeLayout->FindResourceInfo(LayoutResourceKind(category)))
+            return category;
+
+        // Do we have an entry for the `DescriptorTableSlot` category?
+        if (typeLayout->FindResourceInfo(LayoutResourceKind::DescriptorTableSlot))
+        {
+            // Is the category they were asking about one that makes sense for the type
+            // of this variable?
+            Type* type = typeLayout->getType();
+            while (auto arrayType = type->As<ArrayExpressionType>())
+                type = arrayType->baseType;
+            switch (spReflectionType_GetKind(convert(type)))
+            {
+            case SLANG_TYPE_KIND_CONSTANT_BUFFER:
+                if(category == SLANG_PARAMETER_CATEGORY_CONSTANT_BUFFER)
+                    return SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT;
+                break;
+
+            case SLANG_TYPE_KIND_RESOURCE:
+                if(category == SLANG_PARAMETER_CATEGORY_SHADER_RESOURCE)
+                    return SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT;
+                break;
+
+            case SLANG_TYPE_KIND_SAMPLER_STATE:
+                if(category == SLANG_PARAMETER_CATEGORY_SAMPLER_STATE)
+                    return SLANG_PARAMETER_CATEGORY_DESCRIPTOR_TABLE_SLOT;
+                break;
+
+            // TODO: implement more helpers here
+
+            default:
+                break;
+            }
+        }
+
+        return category;
+    }
+}
+
 SLANG_API size_t spReflectionVariableLayout_GetOffset(SlangReflectionVariableLayout* inVarLayout, SlangParameterCategory category)
 {
     auto varLayout = convert(inVarLayout);
     if(!varLayout) return 0;
 
     auto info = varLayout->FindResourceInfo(LayoutResourceKind(category));
+
+    if (!info)
+    {
+        // No match with requested category? Try again with one they might have meant...
+        category = maybeRemapParameterCategory(varLayout->getTypeLayout(), category);
+        info = varLayout->FindResourceInfo(LayoutResourceKind(category));
+    }
+
     if(!info) return 0;
 
     return info->index;
@@ -642,10 +726,30 @@ SLANG_API size_t spReflectionVariableLayout_GetSpace(SlangReflectionVariableLayo
     auto varLayout = convert(inVarLayout);
     if(!varLayout) return 0;
 
+
     auto info = varLayout->FindResourceInfo(LayoutResourceKind(category));
-    if(!info) return 0;
+    if (!info)
+    {
+        // No match with requested category? Try again with one they might have meant...
+        category = maybeRemapParameterCategory(varLayout->getTypeLayout(), category);
+        info = varLayout->FindResourceInfo(LayoutResourceKind(category));
+    }
+
+    UInt space = 0;
+
+    // First, deal with any offset applied to the specific resource kind specified
+    if (info)
+    {
+        space += info->space;
+    }
+
+    // Next, deal with any dedicated register-space offset applied to, e.g., a parameter block
+    if (auto spaceInfo = varLayout->FindResourceInfo(LayoutResourceKind::RegisterSpace))
+    {
+        space += spaceInfo->index;
+    }
 
-    return info->space;
+    return space;
 }
 
 SLANG_API char const* spReflectionVariableLayout_GetSemanticName(SlangReflectionVariableLayout* inVarLayout)
@@ -705,28 +809,20 @@ SLANG_API SlangStage spReflectionVariableLayout_getStage(
 
 SLANG_API unsigned spReflectionParameter_GetBindingIndex(SlangReflectionParameter* inVarLayout)
 {
-    auto varLayout = convert(inVarLayout);
-    if(!varLayout) return 0;
-
-    if(varLayout->resourceInfos.Count() > 0)
-    {
-        return (unsigned) varLayout->resourceInfos[0].index;
-    }
-
-    return 0;
+    SlangReflectionVariableLayout* varLayout = (SlangReflectionVariableLayout*)inVarLayout;
+    return (unsigned) spReflectionVariableLayout_GetOffset(
+        varLayout,
+        spReflectionTypeLayout_GetParameterCategory(
+            spReflectionVariableLayout_GetTypeLayout(varLayout)));
 }
 
 SLANG_API unsigned spReflectionParameter_GetBindingSpace(SlangReflectionParameter* inVarLayout)
 {
-    auto varLayout = convert(inVarLayout);
-    if(!varLayout) return 0;
-
-    if(varLayout->resourceInfos.Count() > 0)
-    {
-        return (unsigned) varLayout->resourceInfos[0].space;
-    }
-
-    return 0;
+    SlangReflectionVariableLayout* varLayout = (SlangReflectionVariableLayout*)inVarLayout;
+    return (unsigned) spReflectionVariableLayout_GetSpace(
+        varLayout,
+        spReflectionTypeLayout_GetParameterCategory(
+            spReflectionVariableLayout_GetTypeLayout(varLayout)));
 }
 
 // Helpers for getting parameter count
@@ -737,7 +833,7 @@ namespace Slang
     {
         if(auto parameterGroupLayout = typeLayout.As<ParameterGroupTypeLayout>())
         {
-            typeLayout = parameterGroupLayout->elementTypeLayout;
+            typeLayout = parameterGroupLayout->offsetElementTypeLayout;
         }
 
         if(auto structLayout = typeLayout.As<StructTypeLayout>())
@@ -752,7 +848,7 @@ namespace Slang
     {
         if(auto parameterGroupLayout = typeLayout.As<ParameterGroupTypeLayout>())
         {
-            typeLayout = parameterGroupLayout->elementTypeLayout;
+            typeLayout = parameterGroupLayout->offsetElementTypeLayout;
         }
 
         if(auto structLayout = typeLayout.As<StructTypeLayout>())