Revise new COM-lite API (#1007)

* Revise new COM-lite API

This change revises the "COM-lite" API that was recently introduced to try to streamline it and introduce some missing central/base concepts.

The central new abstraction in the API is the notion of a "component type," which is a unit of shader code composition. A component type can have:

* IR code for some number of functions/types/etc.
* Zero or more global shader parameters
* Zero or more "entry point" functions at which execution can start
* Zero or more "specialization" parameters (types or values that must be filled in before kernel code can be generated)
* Zero or more "requirements" (dependencies on other component types that must be satisfied before kernel code can be generated)

Both individual compiled modules, and validated entry points are then examples of component types, and we additionally define a few services that apply to all component types:

* We can take N component types and compose them to create a new component type that combines their code, shader parameters, entry points, and specialization parameters. A composed component type may also include requirements from the sub-component types, but it is also possible that by composing thing we satisfy requirements (if `A` requires `B`, and we compose `A` and `B`, then the requirement is now satisfied, and doesn't appear on the composite).

* We can take a component type with N specialization parameters, and specialize it by giving N compatible specialization arguments. The result of specialization is a new component type with zero specialization parameters. Under the right circumstances the specialzed component type will be layout compatible with the unspecialized one.

* One more example that isn't exposed in the public API today is that we can take a component with requirements and "complete" it by automatically composing it with component types that satisfy those requirements. This can be seen as a kind of linking step that pulls together the transitive closure of dependencies.

* We can query the layout for the shader parameters and entry points of a component type, for a specific target.

* We can query compiled kernel code for an entry point in a component type (for a specific target). This only works for component types with zero specialization parameters and zero requirements.

The idea is that by giving users a fairly general algebra of operations on component types, they can compose final programs in ways that meet their requirements. For example, it becomes possible to incrementally "grow" a component type to represent the global root signature for ray tracing shaders as new entry points are added, in such a way that it always stays layout-compatible with kernels that have already been compiled.

Much of the implementation work here is in implementing the unifying component type abstraction, and in particular re-writing code that used to assume a program consisted of a flat list of modules and entry points to work with a hierarchical representation that reflects the underlying algebra (e.g., with types to represent composite and specialized component types).

There's also a hidden "legacy" case of a component type to deal with some legacy compiler behaviors that can't be directly modeled on top of the simple algebra with modules and entry points.

This API is by no means feature-complete or fully developed. It is expected that we will flesh it out more when bringing up application code (e.g., Falcor) on top of the revamped API.

One notable thing that went away in this change is explicit support for "entry point groups" and notions of local root signatures (especially the Falcor-specific handling of the `shared` keyword, which a previous change turned into an explicitly supported feature). With the new "building blocks" approach, it should be possible for a DXR application to deal with local root signatures as a matter of policy (on top of the API we provide). If/when we need to provide some kind of emulation of local root signatures for Vulkan (and/or if Vulkan is extended with an explicit notion of local root signatures), we might need to revisit this choice.

* Fix debug build

There was invalid code inside an `assert()`, so the release build didn't catch it.

* fixup: warnings

* fixup: more warnings-as-errors

* fixup: review notes

* fixup: use component type visitors in place of dynamic casting

1 files changed, 81 insertions, 100 deletions

diff --git a/source/slang/slang-reflection.cpp b/source/slang/slang-reflection.cpp
index 3871e2ccc..5e6fd10cd 100644
--- a/source/slang/slang-reflection.cpp
+++ b/source/slang/slang-reflection.cpp
@@ -51,9 +51,9 @@ static inline SlangReflectionTypeLayout* convert(TypeLayout* type)
     return (SlangReflectionTypeLayout*) type;
 }
 
-static inline GenericParamLayout* convert(SlangReflectionTypeParameter * typeParam)
+static inline SpecializationParamLayout* convert(SlangReflectionTypeParameter * typeParam)
 {
-    return (GenericParamLayout*)typeParam;
+    return (SpecializationParamLayout*) typeParam;
 }
 
 static inline VarDeclBase* convert(SlangReflectionVariable* var)
@@ -86,16 +86,6 @@ static inline SlangReflectionEntryPoint* convert(EntryPointLayout* entryPoint)
     return (SlangReflectionEntryPoint*) entryPoint;
 }
 
-static inline EntryPointGroupLayout* convert(SlangEntryPointGroupLayout* entryPointGroup)
-{
-    return (EntryPointGroupLayout*) entryPointGroup;
-}
-
-static inline SlangEntryPointGroupLayout* convert(EntryPointGroupLayout* entryPointGroup)
-{
-    return (SlangEntryPointGroupLayout*) entryPointGroup;
-}
-
 static inline ProgramLayout* convert(SlangReflection* program)
 {
     return (ProgramLayout*) program;
@@ -865,7 +855,7 @@ SLANG_API int spReflectionTypeLayout_getGenericParamIndex(SlangReflectionTypeLay
 
     if(auto genericParamTypeLayout = as<GenericParamTypeLayout>(typeLayout))
     {
-        return genericParamTypeLayout->paramIndex;
+        return (int) genericParamTypeLayout->paramIndex;
     }
     else
     {
@@ -1222,7 +1212,7 @@ SLANG_API char const* spReflectionEntryPoint_getName(
     auto entryPointLayout = convert(inEntryPoint);
     if(!entryPointLayout) return 0;
 
-    return getText(entryPointLayout->entryPoint->getName()).begin();
+    return getText(entryPointLayout->entryPoint.GetName()).begin();
 }
 
 SLANG_API unsigned spReflectionEntryPoint_getParameterCount(
@@ -1270,7 +1260,7 @@ SLANG_API void spReflectionEntryPoint_getComputeThreadGroupSize(
     SlangUInt sizeAlongAxis[3] = { 1, 1, 1 };
 
     // First look for the HLSL case, where we have an attribute attached to the entry point function
-    auto numThreadsAttribute = entryPointFunc->FindModifier<NumThreadsAttribute>();
+    auto numThreadsAttribute = entryPointFunc.getDecl()->FindModifier<NumThreadsAttribute>();
     if (numThreadsAttribute)
     {
         sizeAlongAxis[0] = numThreadsAttribute->x;
@@ -1281,7 +1271,7 @@ SLANG_API void spReflectionEntryPoint_getComputeThreadGroupSize(
     {
         // Fall back to the GLSL case, which requires a search over global-scope declarations
         // to look for as with the `local_size_*` qualifier
-        auto module = as<ModuleDecl>(entryPointFunc->ParentDecl);
+        auto module = as<ModuleDecl>(entryPointFunc.getDecl()->ParentDecl);
         if (module)
         {
             for (auto dd : module->Members)
@@ -1323,11 +1313,43 @@ SLANG_API int spReflectionEntryPoint_usesAnySampleRateInput(
     return (entryPointLayout->flags & EntryPointLayout::Flag::usesAnySampleRateInput) != 0;
 }
 
+SLANG_API SlangReflectionVariableLayout* spReflectionEntryPoint_getVarLayout(
+    SlangReflectionEntryPoint* inEntryPoint)
+{
+    auto entryPointLayout = convert(inEntryPoint);
+    if(!entryPointLayout)
+        return nullptr;
+
+    return convert(entryPointLayout->parametersLayout);
+}
+
+static bool hasDefaultConstantBuffer(ScopeLayout* layout)
+{
+    auto typeLayout = layout->parametersLayout->getTypeLayout();
+    return as<ParameterGroupTypeLayout>(typeLayout) != nullptr;
+}
+
+SLANG_API int spReflectionEntryPoint_hasDefaultConstantBuffer(
+    SlangReflectionEntryPoint* inEntryPoint)
+{
+    auto entryPointLayout = convert(inEntryPoint);
+    if(!entryPointLayout)
+        return 0;
+
+    return hasDefaultConstantBuffer(entryPointLayout);
+}
+
+
 // SlangReflectionTypeParameter
 SLANG_API char const* spReflectionTypeParameter_GetName(SlangReflectionTypeParameter * inTypeParam)
 {
-    auto typeParam = convert(inTypeParam);
-    return typeParam->decl->getName()->text.getBuffer();
+    auto specializationParam = convert(inTypeParam);
+    if( auto genericParamLayout = as<GenericSpecializationParamLayout>(specializationParam) )
+    {
+        return genericParamLayout->decl->getName()->text.getBuffer();
+    }
+    // TODO: Add case for existential type parameter? They don't have as simple of a notion of "name" as the generic case...
+    return nullptr;
 }
 
 SLANG_API unsigned spReflectionTypeParameter_GetIndex(SlangReflectionTypeParameter * inTypeParam)
@@ -1338,16 +1360,34 @@ SLANG_API unsigned spReflectionTypeParameter_GetIndex(SlangReflectionTypeParamet
 
 SLANG_API unsigned int spReflectionTypeParameter_GetConstraintCount(SlangReflectionTypeParameter* inTypeParam)
 {
-    auto typeParam = convert(inTypeParam);
-    auto constraints = typeParam->decl->getMembersOfType<GenericTypeConstraintDecl>();
-    return (unsigned int)constraints.getCount();
+    auto specializationParam = convert(inTypeParam);
+    if(auto genericParamLayout = as<GenericSpecializationParamLayout>(specializationParam))
+    {
+        if( auto globalGenericParamDecl = as<GlobalGenericParamDecl>(genericParamLayout->decl) )
+        {
+            auto constraints = globalGenericParamDecl->getMembersOfType<GenericTypeConstraintDecl>();
+            return (unsigned int)constraints.getCount();
+        }
+        // TODO: Add case for entry-point generic parameters.
+    }
+    // TODO: Add case for existential type parameters.
+    return 0;
 }
 
 SLANG_API SlangReflectionType* spReflectionTypeParameter_GetConstraintByIndex(SlangReflectionTypeParameter * inTypeParam, unsigned index)
 {
-    auto typeParam = convert(inTypeParam);
-    auto constraints = typeParam->decl->getMembersOfType<GenericTypeConstraintDecl>();
-    return (SlangReflectionType*)constraints.toArray()[index]->sup.Ptr();
+    auto specializationParam = convert(inTypeParam);
+    if(auto genericParamLayout = as<GenericSpecializationParamLayout>(specializationParam))
+    {
+        if( auto globalGenericParamDecl = as<GlobalGenericParamDecl>(genericParamLayout->decl) )
+        {
+            auto constraints = globalGenericParamDecl->getMembersOfType<GenericTypeConstraintDecl>();
+            return (SlangReflectionType*)constraints.toArray()[index]->sup.Ptr();
+        }
+        // TODO: Add case for entry-point generic parameters.
+    }
+    // TODO: Add case for existential type parameters.
+    return 0;
 }
 
 // Shader Reflection
@@ -1379,22 +1419,32 @@ SLANG_API SlangReflectionParameter* spReflection_GetParameterByIndex(SlangReflec
 SLANG_API unsigned int spReflection_GetTypeParameterCount(SlangReflection * reflection)
 {
     auto program = convert(reflection);
-    return (unsigned int)program->globalGenericParams.getCount();
+    return (unsigned int) program->specializationParams.getCount();
 }
 
 SLANG_API SlangReflectionTypeParameter* spReflection_GetTypeParameterByIndex(SlangReflection * reflection, unsigned int index)
 {
     auto program = convert(reflection);
-    return (SlangReflectionTypeParameter*)program->globalGenericParams[index].Ptr();
+    return (SlangReflectionTypeParameter*) program->specializationParams[index].Ptr();
 }
 
 SLANG_API SlangReflectionTypeParameter * spReflection_FindTypeParameter(SlangReflection * inProgram, char const * name)
 {
     auto program = convert(inProgram);
     if (!program) return nullptr;
-    GenericParamLayout * result = nullptr;
-    program->globalGenericParamsMap.TryGetValue(name, result);
-    return (SlangReflectionTypeParameter*)result;
+    for( auto& param : program->specializationParams )
+    {
+        auto genericParamLayout = as<GenericSpecializationParamLayout>(param);
+        if(!genericParamLayout)
+            continue;
+
+        if(getText(genericParamLayout->decl->getName()) != UnownedTerminatedStringSlice(name))
+            continue;
+
+        return (SlangReflectionTypeParameter*) genericParamLayout;
+    }
+
+    return 0;
 }
 
 SLANG_API SlangUInt spReflection_getEntryPointCount(SlangReflection* inProgram)
@@ -1421,7 +1471,7 @@ SLANG_API SlangReflectionEntryPoint* spReflection_findEntryPointByName(SlangRefl
     // TODO: improve on naive linear search
     for(auto ep : program->entryPoints)
     {
-        if(ep->entryPoint->getName()->text == name)
+        if(ep->entryPoint.GetName()->text == name)
         {
             return convert(ep);
         }
@@ -1430,75 +1480,6 @@ SLANG_API SlangReflectionEntryPoint* spReflection_findEntryPointByName(SlangRefl
     return nullptr;
 }
 
-
-SLANG_API SlangInt spReflection_getEntryPointGroupCount(SlangReflection* inProgram)
-{
-    auto program = convert(inProgram);
-    if(!program) return 0;
-
-    return program->entryPointGroups.getCount();
-}
-
-SLANG_API SlangEntryPointGroupLayout* spReflection_getEntryPointGroupByIndex(SlangReflection* inProgram, SlangInt index)
-{
-    auto program = convert(inProgram);
-    if(!program) return 0;
-
-    if(index < 0) return nullptr;
-    if(index >= program->entryPointGroups.getCount()) return nullptr;
-
-    return convert(program->entryPointGroups[(int) index].Ptr());
-}
-
-SLANG_API SlangInt spEntryPointGroupLayout_getEntryPointCount(SlangEntryPointGroupLayout* inGroup)
-{
-    auto group = convert(inGroup);
-    if(!group) return 0;
-
-    return group->entryPoints.getCount();
-}
-
-SLANG_API SlangReflectionEntryPoint* spEntryPointGroupLayout_getEntryPointByIndex(SlangEntryPointGroupLayout* inGroup, SlangInt index)
-{
-    auto group = convert(inGroup);
-    if(!group) return 0;
-
-    if(index < 0) return nullptr;
-    if(index >= group->entryPoints.getCount()) return nullptr;
-
-    return convert(group->entryPoints[(int) index].Ptr());
-}
-
-SLANG_API SlangReflectionVariableLayout* spEntryPointGroupLayout_getVarLayout(SlangEntryPointGroupLayout* inGroup)
-{
-    auto group = convert(inGroup);
-    if(!group) return 0;
-
-    return convert(group->parametersLayout);
-}
-
-SLANG_API SlangInt spEntryPointGroupLayout_getParameterCount(SlangEntryPointGroupLayout* inGroup)
-{
-    auto groupLayout = convert(inGroup);
-    if(!groupLayout) return 0;
-
-    auto& params = groupLayout->group->getShaderParams();
-    return params.getCount();
-}
-
-SLANG_API SlangReflectionVariableLayout* spEntryPointGroupLayout_getParameterByIndex(SlangEntryPointGroupLayout* inGroup, SlangInt index)
-{
-    auto groupLayout = convert(inGroup);
-    if(!groupLayout) return nullptr;
-
-    auto& params = groupLayout->group->getShaderParams();
-    if(index < 0) return nullptr;
-    if(index >= params.getCount()) return nullptr;
-
-    return convert(getScopeStructLayout(groupLayout)->fields[index]);
-}
-
-
 SLANG_API SlangUInt spReflection_getGlobalConstantBufferBinding(SlangReflection* inProgram)
 {
     auto program = convert(inProgram);
@@ -1531,7 +1512,7 @@ SLANG_API  SlangReflectionType* spReflection_specializeType(
     auto unspecializedType = convert(inType);
     if(!unspecializedType) return nullptr;
 
-    auto linkage = programLayout->getProgram()->getLinkageImpl();
+    auto linkage = programLayout->getProgram()->getLinkage();
 
     DiagnosticSink sink(linkage->getSourceManager());