Start to support cross-compilation via "lowering" pass

- The big change here is the introduction of a "lowering" pass that takes an input AST from the semantic checker, and produces an output AST suitable for emitting. The intention is that he lowering pass is responsible for:
  - Stripping out unused code (when we have enough information to do so), by only outputting declarations that are transitively references from an entry point
  - When cross-compiling to GLSL, generating a suitable `void main()` entry point to wrap the user-written entry-point function
  - (Eventually) legalizing types in the program, by scalarizing aggregate types that mix uniform and resource types
  - (Eventually) instantiating generic declarations so that the resulting code only deals with fully specialized declarations
  - (Eventually) de-sugaring OOP constructs into basic "structs and functions" form
  - (Eventually) instantiating code that depends on interface types at the concrete types chosen

- It is clear that there is still a lot of work to be done there, to this change is really about getting infrastructure in place without breaking the existing test cases.

- One cleanup here is that we get rid of the idea of whole-translation-unit output, since that was specific to HLSL output, and there is really no strong reason for keeping it. Users should now just ask for the output for each entry point that they wanted to generate.

- The biggest source of complexity for the lowering process is that it needs to produce the same AST structure as the input, to deal with the complexity of the rewriter case. That is, we need the output to be able to reproduce the input exactly in the case where we are rewriting and nothing needs to change, so the output format needs at least the degrees of freedom of the input.
  - As a result, we end up having to distinguish "rewriter" and "full" modes in both lowering and code-emit steps, so that we can react appropriately.

- Generating a GLSL `main()` also adds a lot of complexity. Right now I'm using the simplest approach, where we always output the Slang/HLSL entry point as an ordinary function (as written) and then emit a simple GLSL `main()` to call it. I generate globals for all the shader inputs/outputs (these need to be scalarized and have explicit `location`s attached), and then collect these into the `struct` types of the original parameters as needed.
  - This approach will start to have some major down-sides once we have to deal with "arrayed" input/output
  - A long-term question here is how to replace entry-point parameter types with scalarized and/or "transposed" versions, while still letting the original code work as written (including copying those inputs to temporary arrays)

- Split `BlockStatementSyntaxNode` into:
  - `BlockStmt` which just provides a scope around a `body` statement
  - `SeqStmt` which just allows multiple statements to be treated as one

- Change how we emit `for` loops, to deal with the case where the initialization part might expand into multiple statements
  - Basically `for(A;B;C) {D}` becomes `{A; for(;B;C) {D}}`, so we can handle arbitrary statements for `A`
  - As an additional wrinkle, when we are rewriting HLSL, we just generate `A; for(;B;C) {D}` to deal with the broken scoping there
  - This change is needed because the lowering pass was sometimes expanding the original initialization statement `A` into a block `{A}`. Certainly if it declared multiple variables we'd need to handle it, and this seemed the easiest way

- A more significant challenge for lowering would come if/when we ever wanted to support true short-circuiting behavior for `&&` and `||`

- For right now I'm not changing the behavior of the "rewriter" mode, so we still have `UnparsedStmt` instances being generated, but it is clear that eventually we need to parse *all* input, even if we can't type-check 100% of it. This is required so that we can rewrite user code that might refer to a shader input with interface type.

1 files changed, 145 insertions, 75 deletions

diff --git a/source/slang/parameter-binding.cpp b/source/slang/parameter-binding.cpp
index 2a0c64892..8d008618f 100644
--- a/source/slang/parameter-binding.cpp
+++ b/source/slang/parameter-binding.cpp
@@ -767,86 +767,87 @@ SimpleSemanticInfo decomposeSimpleSemantic(
     return info;
 }
 
-enum class EntryPointParameterDirection
+enum EntryPointParameterDirection
 {
-    Input,
-    Output,
+    kEntryPointParameterDirection_Input  = 0x1,
+    kEntryPointParameterDirection_Output = 0x2,
 };
+typedef unsigned int EntryPointParameterDirectionMask;
 
 struct EntryPointParameterState
 {
-    String*                         optSemanticName;
-    int*                            ioSemanticIndex;
-    EntryPointParameterDirection    direction;
-    int                             semanticSlotCount;
+    String*                             optSemanticName;
+    int*                                ioSemanticIndex;
+    EntryPointParameterDirectionMask    directionMask;
+    int                                 semanticSlotCount;
 };
 
-static void processSimpleEntryPointInput(
+static RefPtr<TypeLayout> processSimpleEntryPointParameter(
     ParameterBindingContext*        context,
     RefPtr<ExpressionType>          type,
-    EntryPointParameterState const& state)
+    EntryPointParameterState const& inState,
+    int                             semanticSlotCount = 1)
 {
-    auto optSemanticName    =  state.optSemanticName;
-    auto semanticIndex      = *state.ioSemanticIndex;
-    auto semanticSlotCount  =  state.semanticSlotCount;
-}
+    EntryPointParameterState state = inState;
+    state.semanticSlotCount = semanticSlotCount;
 
-static void processSimpleEntryPointOutput(
-    ParameterBindingContext*        context,
-    RefPtr<ExpressionType>          type,
-    EntryPointParameterState const& state)
-{
     auto optSemanticName    =  state.optSemanticName;
     auto semanticIndex      = *state.ioSemanticIndex;
-    auto semanticSlotCount  =  state.semanticSlotCount;
-
-    if(!optSemanticName)
-        return;
 
-    auto semanticName = *optSemanticName;
+    String semanticName = optSemanticName ? *optSemanticName : "";
+    String sn = semanticName.ToLower();
 
-    // Note: I'm just doing something expedient here and detecting `SV_Target`
-    // outputs and claiming the appropriate register range right away.
-    //
-    // TODO: we should really be building up some representation of all of this,
-    // once we've gone to the trouble of looking it all up...
-    if( semanticName.ToLower() == "sv_target" )
+    RefPtr<TypeLayout> typeLayout =  new TypeLayout();
+    if (sn.StartsWith("sv_"))
     {
-        context->shared->usedResourceRanges[int(LayoutResourceKind::UnorderedAccess)].Add(semanticIndex, semanticIndex + semanticSlotCount);
-    }
-}
+        // System-value semantic.
 
-static void processSimpleEntryPointParameter(
-    ParameterBindingContext*        context,
-    RefPtr<ExpressionType>          type,
-    EntryPointParameterState const& inState,
-    int                             semanticSlotCount = 1)
-{
-    EntryPointParameterState state = inState;
-    state.semanticSlotCount = semanticSlotCount;
+        if (state.directionMask & kEntryPointParameterDirection_Output)
+        {
+            // Note: I'm just doing something expedient here and detecting `SV_Target`
+            // outputs and claiming the appropriate register range right away.
+            //
+            // TODO: we should really be building up some representation of all of this,
+            // once we've gone to the trouble of looking it all up...
+            if( sn == "sv_target" )
+            {
+                context->shared->usedResourceRanges[int(LayoutResourceKind::UnorderedAccess)].Add(semanticIndex, semanticIndex + semanticSlotCount);
+            }
+        }
 
-    switch( state.direction )
+        // TODO: add some kind of usage information for system input/output
+    }
+    else
     {
-    case EntryPointParameterDirection::Input:
-        processSimpleEntryPointInput(context, type, state);
-        break;
+        // user-defined semantic
 
-    case EntryPointParameterDirection::Output:
-        processSimpleEntryPointOutput(context, type, state);
-        break;
+        if (state.directionMask & kEntryPointParameterDirection_Input)
+        {
+            auto rules = context->layoutRules->getVaryingInputRules();
+            SimpleLayoutInfo layout = GetLayout(type, rules);
+            typeLayout->addResourceUsage(layout.kind, layout.size);
+        }
 
-    SLANG_EXHAUSTIVE_SWITCH()
+        if (state.directionMask & kEntryPointParameterDirection_Output)
+        {
+            auto rules = context->layoutRules->getVaryingOutputRules();
+            SimpleLayoutInfo layout = GetLayout(type, rules);
+            typeLayout->addResourceUsage(layout.kind, layout.size);
+        }
     }
 
     *state.ioSemanticIndex += state.semanticSlotCount;
+    typeLayout->type = type;
+
+    return typeLayout;
 }
 
-static void processEntryPointParameter(
+static RefPtr<TypeLayout> processEntryPointParameter(
     ParameterBindingContext*        context,
     RefPtr<ExpressionType>          type,
     EntryPointParameterState const& state);
 
-static void processEntryPointParameterWithPossibleSemantic(
+static RefPtr<TypeLayout> processEntryPointParameterWithPossibleSemantic(
     ParameterBindingContext*        context,
     Decl*                           declForSemantic,
     RefPtr<ExpressionType>          type,
@@ -873,11 +874,11 @@ static void processEntryPointParameterWithPossibleSemantic(
     // *or* we couldn't find an explicit semantic to apply on the given
     // declaration, so we will just recursive with whatever we have at
     // the moment.
-    processEntryPointParameter(context, type, state);
+    return processEntryPointParameter(context, type, state);
 }
 
 
-static void processEntryPointParameter(
+static RefPtr<TypeLayout> processEntryPointParameter(
     ParameterBindingContext*        context,
     RefPtr<ExpressionType>          type,
     EntryPointParameterState const& state)
@@ -885,31 +886,50 @@ static void processEntryPointParameter(
     // Scalar and vector types are treated as outputs directly
     if(auto basicType = type->As<BasicExpressionType>())
     {
-        processSimpleEntryPointParameter(context, basicType, state);
+        return processSimpleEntryPointParameter(context, basicType, state);
     }
     else if(auto basicType = type->As<VectorExpressionType>())
     {
-        processSimpleEntryPointParameter(context, basicType, state);
+        return processSimpleEntryPointParameter(context, basicType, state);
     }
     // A matrix is processed as if it was an array of rows
     else if( auto matrixType = type->As<MatrixExpressionType>() )
     {
         auto rowCount = GetIntVal(matrixType->getRowCount());
-        processSimpleEntryPointParameter(context, basicType, state, (int) rowCount);
+        return processSimpleEntryPointParameter(context, basicType, state, (int) rowCount);
     }
     else if( auto arrayType = type->As<ArrayExpressionType>() )
     {
+        // Note: Bad Things will happen if we have an array input
+        // without a semantic already being enforced.
+        
         auto elementCount = GetIntVal(arrayType->ArrayLength);
 
-        for( int ii = 0; ii < elementCount; ++ii )
+        // We use the first element to derive the layout for the element type
+        auto elementTypeLayout = processEntryPointParameter(context, arrayType->BaseType, state);
+
+        // We still walk over subsequent elements to make sure they consume resources
+        // as needed
+        for( int ii = 1; ii < elementCount; ++ii )
         {
             processEntryPointParameter(context, arrayType->BaseType, state);
         }
+
+        RefPtr<ArrayTypeLayout> arrayTypeLayout = new ArrayTypeLayout();
+        arrayTypeLayout->elementTypeLayout = elementTypeLayout;
+        arrayTypeLayout->type = arrayType;
+
+        for (auto rr : elementTypeLayout->resourceInfos)
+        {
+            arrayTypeLayout->findOrAddResourceInfo(rr.kind)->count = rr.count * elementCount;
+        }
+
+        return arrayTypeLayout;
     }
     // Ignore a bunch of types that don't make sense here...
-    else if(auto textureType = type->As<TextureType>()) {}
-    else if(auto samplerStateType = type->As<SamplerStateType>()) {}
-    else if(auto constantBufferType = type->As<ConstantBufferType>()) {}
+    else if (auto textureType = type->As<TextureType>()) { return nullptr;  }
+    else if(auto samplerStateType = type->As<SamplerStateType>()) { return nullptr;  }
+    else if(auto constantBufferType = type->As<ConstantBufferType>()) { return nullptr;  }
     // Catch declaration-reference types late in the sequence, since
     // otherwise they will include all of the above cases...
     else if( auto declRefType = type->As<DeclRefType>() )
@@ -918,15 +938,36 @@ static void processEntryPointParameter(
 
         if (auto structDeclRef = declRef.As<StructSyntaxNode>())
         {
+            RefPtr<StructTypeLayout> structLayout = new StructTypeLayout();
+            structLayout->type = type;
+
             // Need to recursively walk the fields of the structure now...
             for( auto field : GetFields(structDeclRef) )
             {
-                processEntryPointParameterWithPossibleSemantic(
+                auto fieldTypeLayout = processEntryPointParameterWithPossibleSemantic(
                     context,
                     field.getDecl(),
                     GetType(field),
                     state);
+
+                RefPtr<VarLayout> fieldVarLayout = new VarLayout();
+                fieldVarLayout->varDecl = field;
+                fieldVarLayout->typeLayout = fieldTypeLayout;
+
+                for (auto rr : fieldTypeLayout->resourceInfos)
+                {
+                    assert(rr.count != 0);
+
+                    auto structRes = structLayout->findOrAddResourceInfo(rr.kind);
+                    fieldVarLayout->findOrAddResourceInfo(rr.kind)->index = structRes->count;
+                    structRes->count += rr.count;
+                }
+
+                structLayout->fields.Add(fieldVarLayout);
+                structLayout->mapVarToLayout.Add(field.getDecl(), fieldVarLayout);
             }
+
+            return structLayout;
         }
         else
         {
@@ -937,6 +978,9 @@ static void processEntryPointParameter(
     {
         assert(!"unimplemented");
     }
+
+    assert(!"unexpected");
+    return nullptr;
 }
 
 static void collectEntryPointParameters(
@@ -998,42 +1042,68 @@ static void collectEntryPointParameters(
 
         // We have an entry-point parameter, and need to figure out what to do with it.
 
+        // TODO: need to handle `uniform`-qualified parameters here
+        if (paramDecl->HasModifier<UniformModifier>())
+            continue;
+
+        state.directionMask = 0;
+
         // If it appears to be an input, process it as such.
         if( paramDecl->HasModifier<InModifier>() || paramDecl->HasModifier<InOutModifier>() || !paramDecl->HasModifier<OutModifier>() )
         {
-            state.direction = EntryPointParameterDirection::Input;
-
-            processEntryPointParameterWithPossibleSemantic(
-                context,
-                paramDecl.Ptr(),
-                paramDecl->Type.type,
-                state);
+            state.directionMask |= kEntryPointParameterDirection_Input;
         }
 
         // If it appears to be an output, process it as such.
         if(paramDecl->HasModifier<OutModifier>() || paramDecl->HasModifier<InOutModifier>())
         {
-            state.direction = EntryPointParameterDirection::Output;
+            state.directionMask |= kEntryPointParameterDirection_Output;
+        }
+
+        auto paramTypeLayout = processEntryPointParameterWithPossibleSemantic(
+            context,
+            paramDecl.Ptr(),
+            paramDecl->Type.type,
+            state);
 
-            processEntryPointParameterWithPossibleSemantic(
-                context,
-                paramDecl.Ptr(),
-                paramDecl->Type.type,
-                state);
+        RefPtr<VarLayout> paramVarLayout = new VarLayout();
+        paramVarLayout->varDecl = makeDeclRef(paramDecl.Ptr());
+        paramVarLayout->typeLayout = paramTypeLayout;
+
+        for (auto rr : paramTypeLayout->resourceInfos)
+        {
+            auto entryPointRes = entryPointLayout->findOrAddResourceInfo(rr.kind);
+            paramVarLayout->findOrAddResourceInfo(rr.kind)->index = entryPointRes->count;
+            entryPointRes->count += rr.count;
         }
+
+        entryPointLayout->fields.Add(paramVarLayout);
+        entryPointLayout->mapVarToLayout.Add(paramDecl, paramVarLayout);
     }
 
     // If we can find an output type for the entry point, then process it as
     // an output parameter.
     if( auto resultType = entryPointFuncDecl->ReturnType.type )
     {
-        state.direction = EntryPointParameterDirection::Output;
+        state.directionMask = kEntryPointParameterDirection_Output;
 
-        processEntryPointParameterWithPossibleSemantic(
+        auto resultTypeLayout = processEntryPointParameterWithPossibleSemantic(
             context,
             entryPointFuncDecl,
             resultType,
             state);
+
+        RefPtr<VarLayout> resultLayout = new VarLayout();
+        resultLayout->typeLayout = resultTypeLayout;
+
+        for (auto rr : resultTypeLayout->resourceInfos)
+        {
+            auto entryPointRes = entryPointLayout->findOrAddResourceInfo(rr.kind);
+            resultLayout->findOrAddResourceInfo(rr.kind)->index = entryPointRes->count;
+            entryPointRes->count += rr.count;
+        }
+
+        entryPointLayout->resultLayout = resultLayout;
     }
 }