Split front- and back-ends (#846)

* Split front- and back-ends

This change is a major refactor of several of the types that provide the behind-the-scenes implementation of the public C API.
The goal of this refactor is primarily to allow for future API services that let the user operate both the front- and back-ends of the compiler in a more complex fashion.
For example, as user should be able to compile a bunch of source code into modules, look up types, functions, etc. in those modules, specialize generic types/functions to the types they've looked up, and then finally request target code to be gernerated for specialized entry points.
The back-end code generation they trigger should re-use the front-end compilation work (parsing, semantic checking, IR generation) that was already performed.

The most visible change is that `CompileRequest` has been split up into several smaller types that take responsibility for parts of what it did:

* The `Linkage` type owns the storage for `import`ed modules, and well as the `TargetRequest`s that represent code-generation targets. The intention is that an application could use a single `Linkage` for the duration of its runtime (so long as it was okay with the memory usage), so that each `import`ed module only gets loaded once. For now, this type needs to manage the search paths, file system, and source manager, because of its responsibility for loading files.

* A `FrontEndCompileRequest` owns the stuff related to parsing, semantic checking, and initial IR generation. This most notably includes the `TranslationUnitRequest`s and the `FrontEndEntryPointRequest`s (which used to be just `EntryPointRequest`s). It's main job is to produce AST and IR modules for each translation unit, and to find and validate the entry points. The front-end request does *not* interact with generic arguments for global or entry-point generic parameters.

* The main output of both `import` operations and front-end translation units is the `Module` type, which is just a simple container for both the AST module (to service the reflection/layout APIs, and also for semantic checking of code that `import`s the module) and the IR module (for linking and code generation). This type captures the commonalities between the old `LoadedModule` (which is now just an alias for `Module`) and `TranslationUnitRequest` (which now owns a `Module`).

* The secondary output of front-end compilation is a `Program`, which comprises a list of referenced `Module`s and validated `EntryPoint`s that will be used together. Layout and code generation both need a `Program` to tell them what modules and entry points will be used together (we don't want to just code-gen everythin that has ever been loaded into the linakge). The `Program`s created by the front-end do not include generic arguments, so they may provide incomplete layout information and/or be unsuitable for code generation.

* A `BackEndCompileRequest` owns stuff related to turning a `Program` into output kernels for the targets of a `Linkage`. Most of the data it owns beyond the `Program` to be compiled is minor, so this is a good candidate for demotion from a heap-allocated object to just a `struct` of options that gets passed around.

* The `CompileRequestBase` type is an attempt to wrap up the common functionality of both front-end and back-end compile requests. Most of it is just exposing the availability of a linkage and `DiagnosticSink`, so this type is a good candidate for subsequent removal. The main interesting thing it has is the flags related to dumping and validation of IR, so there is probably a good refactoring still to be made around deciding how options should be handled going forward.

* Behind the scenes, the `Program` type is set up to handle some level of on-line compilation and layout work. The `Program` knows the `Linkage` it belongs to, and allows for a `TargetProgram` to be looked up based on a specific `TargetRequest`. A `TargetProgram` then allows layout information and compiled kernel code to be asked for on-demand, in order to support eventual "live" compilation scenarios.

* The `EndToEndCompileRequest` type is a composition/coordination type that replaces the old `CompileRequest` in a way that uses the services of the various other types. It owns a few pieces of state that only make sense in the context of an end-to-end compile (e.g., there is really no way to "pass through" code when the front- and back-ends are run separately) or a command-line compile (everything to do with specifying output paths for files is really just for the benefit of `slangc`, and might even be moved there over time).

* One important detail is that the `EndToEndCompilRequest` owns all of the string-based generic arguments for both global and entry-point generic parameters. The logic in `check.cpp` for dealing with those arguments has been heavily refactored to separate out the parsings steps that are specific to end-to-end compilation with string-based type arguments, and the semantic checking  steps that result in a specialized `Program` (which can be exposed through new APIs that aren't tied to end-to-end compilation).

It is perhaps not surprising that this change had a lot of consequences, so I'll briefly run over some of the main categories of changes required:

* I changed the way that global generic arguments are passed via API (use `spSetGlobalGenericArgs` instead of the generic arguments for `spAddEntryPointEx`, which are not just for entry-point generics), which has been a change that we've needed for a long time. This is technically a breaking API change, although we should have very few client applications that care about it.

* A bunch of places that used to take "big" objects like `CompileRequest` now just take the sub-pieces they care about (e.g., a function might have only needed a `Linkage` and a `DiagnosticSink`). This makes many subroutines or "context" struct types more generally useful, at the cost of taking more parameters.

* In a few cases the conceptually clean separation of the layers breaks down (often for edge-case or compatibility features), and so we may pass along additional objects that are allowed to be null, but are used when present. A big example of this is how the back-end code generation routines accept an `EndToEndCompileRequest` that is optional, and only used to check whether "pass through" compilation is needed. We should probably look into cleaning this kind of logic up over time so that we don't need to violate the apparent separation of phases of compilation.

* In cases where separation of layers was being broken for the sake of GLSL features, I went ahead and ripped them out, since all of that should be dead code anyway.

* In many cases I increased the encapsulation of data in the core types to help track down use sites and make sure they are following invariants better.

* In cases where code was doing, e.g., `context->shared->compileRequest->session->getThing()` I have tried to introduce convenience routines so that the usage site is just `context->getThing()` to improve encapsulation and allow changes to be made more easily going forward.

* The `noteInternalErrorLoc` functionality was moved off of the compile request and into `DiagnosticSink`, since that is the one type you can rely on having around when you want to note an internal error. We may consider going forward if (and how) it should reset the counter used for noting locations on internal errors.

* A few APIs now take `DiagnosticSink*` arguments where they didn't before, and as a result some public APIs need to create `DiagnosticSink`s to pass in, before going ahead and ignoring the messages. In the future there should be variations of these APIs that accept an `ISlangBlob**` parameter for the output.

* fixup: missing include for compilers with accurate template checking (non-VS)

* fixup: review feedback

1 files changed, 395 insertions, 168 deletions

diff --git a/source/slang/compiler.cpp b/source/slang/compiler.cpp
index 21f56c9ee..3bc34692d 100644
--- a/source/slang/compiler.cpp
+++ b/source/slang/compiler.cpp
@@ -120,23 +120,123 @@ namespace Slang
         return blob;
     }
 
-    // EntryPointRequest
+    //
+    // FrontEndEntryPointRequest
+    //
+
+    FrontEndEntryPointRequest::FrontEndEntryPointRequest(
+        FrontEndCompileRequest* compileRequest,
+        int                     translationUnitIndex,
+        Name*                   name,
+        Profile                 profile)
+        : m_compileRequest(compileRequest)
+        , m_translationUnitIndex(translationUnitIndex)
+        , m_name(name)
+        , m_profile(profile)
+    {}
 
-    TranslationUnitRequest* EntryPointRequest::getTranslationUnit()
+
+    TranslationUnitRequest* FrontEndEntryPointRequest::getTranslationUnit()
     {
-        return compileRequest->translationUnits[translationUnitIndex].Ptr();
+        return getCompileRequest()->translationUnits[m_translationUnitIndex];
     }
 
-    DeclRef<FuncDecl> EntryPointRequest::getFuncDeclRef()
+    //
+    // EntryPoint
+    //
+
+    RefPtr<EntryPoint> EntryPoint::create(
+        DeclRef<FuncDecl>   funcDeclRef,
+        Profile             profile)
     {
-        return funcDeclRef;
+        RefPtr<EntryPoint> entryPoint = new EntryPoint(
+            funcDeclRef.GetName(),
+            profile,
+            funcDeclRef);
+        return entryPoint;
     }
 
-    RefPtr<FuncDecl> EntryPointRequest::getFuncDecl()
+    RefPtr<EntryPoint> EntryPoint::createDummyForPassThrough(
+        Name*       name,
+        Profile     profile)
     {
-        return getFuncDeclRef().getDecl();
+        RefPtr<EntryPoint> entryPoint = new EntryPoint(
+            name,
+            profile,
+            DeclRef<FuncDecl>());
+        return entryPoint;
     }
 
+    EntryPoint::EntryPoint(
+        Name*               name,
+        Profile             profile,
+        DeclRef<FuncDecl>   funcDeclRef)
+        : m_name(name)
+        , m_profile(profile)
+        , m_funcDeclRef(funcDeclRef)
+    {
+        // In order for later code generation to work, we need to track what
+        // modules each entry point depends on. We will build up the dependency
+        // list here when an `EntryPoint` gets created.
+        //
+        // We know an entry point depends on the module that declared the
+        // entry-point function itself.
+        //
+        // Note: we are carefully handling the case where `module` could
+        // be null, becase of "dummy" entry points created for pass-through
+        // compilation.
+        //
+        if(auto module = getModule())
+        {
+            m_dependencyList.addDependency(module);
+        }
+        //
+        // TODO: We also need to include the modules needed by any generic
+        // arguments in the dependency list, since in the general case they
+        // might come from modules other than the one defining the entry point.
+
+        // The following is a bit of a hack.
+        //
+        // Back-end code generation relies on us having computed layouts for all tagged
+        // unions that end up being used in the code, which means we need a way to find
+        // all such types that get used in a program (and the stuff it imports).
+        //
+        // For now we are assuming a tagged union type only comes into existence
+        // as a (top-level) argument for a generic type parameter, so that we
+        // can check for them here and cache them on the entry point.
+        //
+        // A longer-term strategy might need to consider any (tagged or untagged)
+        // union types that get used inside of a module, and also take
+        // those lists into account.
+        //
+        // An even longer-term strategy would be to allow type layout to
+        // be performed on IR types, so taht we don't need to have front-end
+        // code worrying about this stuff.
+        // 
+        for( auto subst = funcDeclRef.substitutions.substitutions; subst; subst = subst->outer )
+        {
+            if( auto genericSubst = as<GenericSubstitution>(subst) )
+            {
+                for( auto arg : genericSubst->args )
+                {
+                    if( auto taggedUnionType = as<TaggedUnionType>(arg) )
+                    {
+                        m_taggedUnionTypes.Add(taggedUnionType);
+                    }
+                }
+            }
+        }
+    }
+
+    Module* EntryPoint::getModule()
+    {
+        return Slang::getModule(getFuncDecl());
+    }
+
+    Linkage* EntryPoint::getLinkage()
+    {
+        return getModule()->getLinkage();
+    }
 
     //
 
@@ -279,13 +379,35 @@ namespace Slang
 
     //
 
+        /// If there is a pass-through compile going on, find the translation unit for the given entry point.
+    TranslationUnitRequest* findPassThroughTranslationUnit(
+        EndToEndCompileRequest* endToEndReq,
+        Int                     entryPointIndex)
+    {
+        // If there isn't an end-to-end compile going on,
+        // there can be no pass-through.
+        //
+        if(!endToEndReq) return nullptr;
+
+        // And if pass-through isn't set, we don't need
+        // access to the translation unit.
+        //
+        if(endToEndReq->passThrough == PassThroughMode::None) return nullptr;
+
+        auto frontEndReq = endToEndReq->getFrontEndReq();
+        auto entryPointReq = frontEndReq->getEntryPointReq(entryPointIndex);
+        auto translationUnit = entryPointReq->getTranslationUnit();
+        return translationUnit;
+    }
+
     String emitHLSLForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq)
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq)
     {
-        auto compileRequest = entryPoint->compileRequest;
-        auto translationUnit = entryPoint->getTranslationUnit();
-        if (compileRequest->passThrough != PassThroughMode::None)
+        if(auto translationUnit = findPassThroughTranslationUnit(endToEndReq, entryPointIndex))
         {
             // Generate a string that includes the content of
             // the source file(s), along with a line directive
@@ -294,7 +416,7 @@ namespace Slang
             // mode.
 
             StringBuilder codeBuilder;
-            for(auto sourceFile : translationUnit->sourceFiles)
+            for(auto sourceFile : translationUnit->getSourceFiles())
             {
                 codeBuilder << "#line 1 \"";
 
@@ -323,21 +445,21 @@ namespace Slang
         else
         {
             return emitEntryPoint(
+                compileRequest,
                 entryPoint,
-                targetReq->layout.Ptr(),
                 CodeGenTarget::HLSL,
                 targetReq);
         }
     }
 
     String emitGLSLForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq)
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq)
     {
-        auto compileRequest = entryPoint->compileRequest;
-        auto translationUnit = entryPoint->getTranslationUnit();
-
-        if (compileRequest->passThrough != PassThroughMode::None)
+        if(auto translationUnit = findPassThroughTranslationUnit(endToEndReq, entryPointIndex))
         {
             // Generate a string that includes the content of
             // the source file(s), along with a line directive
@@ -347,7 +469,7 @@ namespace Slang
 
             StringBuilder codeBuilder;
             int translationUnitCounter = 0;
-            for(auto sourceFile : translationUnit->sourceFiles)
+            for(auto sourceFile : translationUnit->getSourceFiles())
             {
                 int translationUnitIndex = translationUnitCounter++;
 
@@ -370,8 +492,8 @@ namespace Slang
             // TODO(tfoley): need to pass along the entry point
             // so that we properly emit it as the `main` function.
             return emitEntryPoint(
+                compileRequest,
                 entryPoint,
-                targetReq->layout.Ptr(),
                 CodeGenTarget::GLSL,
                 targetReq);
         }
@@ -484,9 +606,9 @@ namespace Slang
         sink->diagnoseRaw(SLANG_FAILED(res) ? Severity::Error : Severity::Warning, builder.getUnownedSlice());
     }
 
-    static String _getDisplayPath(const DiagnosticSink& sink, SourceFile* sourceFile)
+    static String _getDisplayPath(DiagnosticSink* sink, SourceFile* sourceFile)
     {
-        if (sink.flags & DiagnosticSink::Flag::VerbosePath)
+        if (sink->flags & DiagnosticSink::Flag::VerbosePath)
         {
             return sourceFile->calcVerbosePath();
         }
@@ -496,17 +618,17 @@ namespace Slang
         }
     }
 
-    String calcTranslationUnitSourcePath(TranslationUnitRequest* translationUnitRequest)
+    String calcSourcePathForEntryPoint(
+        EndToEndCompileRequest* endToEndReq,
+        UInt                    entryPointIndex)
     {
-        CompileRequest* compileRequest = translationUnitRequest->compileRequest;
-        if (compileRequest->passThrough == PassThroughMode::None)
-        {
+        auto translationUnitRequest = findPassThroughTranslationUnit(endToEndReq, entryPointIndex);
+        if(!translationUnitRequest)
             return "slang-generated";
-        }
 
-        auto& sink = translationUnitRequest->compileRequest->mSink;
+        auto sink = endToEndReq->getSink();
 
-        const auto& sourceFiles = translationUnitRequest->sourceFiles;
+        const auto& sourceFiles = translationUnitRequest->getSourceFiles();
 
         const int numSourceFiles = int(sourceFiles.Count());
 
@@ -542,22 +664,26 @@ namespace Slang
     }
 
     SlangResult emitDXBytecodeForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq,
-        List<uint8_t>&      byteCodeOut)
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq,
+        List<uint8_t>&          byteCodeOut)
     {
         byteCodeOut.Clear();
 
-        auto session = entryPoint->compileRequest->mSession;
+        auto session = compileRequest->getSession();
+        auto sink = compileRequest->getSink();
 
-        auto compileFunc = (pD3DCompile)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Fxc_D3DCompile, &entryPoint->compileRequest->mSink);
+        auto compileFunc = (pD3DCompile)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Fxc_D3DCompile, sink);
         if (!compileFunc)
         {
             return SLANG_FAIL;
         }
 
-        auto hlslCode = emitHLSLForEntryPoint(entryPoint, targetReq);
-        maybeDumpIntermediate(entryPoint->compileRequest, hlslCode.Buffer(), CodeGenTarget::HLSL);
+        auto hlslCode = emitHLSLForEntryPoint(compileRequest, entryPoint, entryPointIndex, targetReq, endToEndReq);
+        maybeDumpIntermediate(compileRequest, hlslCode.Buffer(), CodeGenTarget::HLSL);
 
         auto profile = getEffectiveProfile(entryPoint, targetReq);
 
@@ -569,16 +695,16 @@ namespace Slang
         //
         List<D3D_SHADER_MACRO> dxMacrosStorage;
         D3D_SHADER_MACRO const* dxMacros = nullptr;
-        if( entryPoint->compileRequest->passThrough != PassThroughMode::None )
+        if(auto translationUnit = findPassThroughTranslationUnit(endToEndReq, entryPointIndex))
         {
-            for( auto& define :  entryPoint->compileRequest->preprocessorDefinitions )
+            for( auto& define :  translationUnit->compileRequest->preprocessorDefinitions )
             {
                 D3D_SHADER_MACRO dxMacro;
                 dxMacro.Name = define.Key.Buffer();
                 dxMacro.Definition = define.Value.Buffer();
                 dxMacrosStorage.Add(dxMacro);
             }
-            for( auto& define : entryPoint->getTranslationUnit()->preprocessorDefinitions )
+            for( auto& define : translationUnit->preprocessorDefinitions )
             {
                 D3D_SHADER_MACRO dxMacro;
                 dxMacro.Name = define.Key.Buffer();
@@ -616,7 +742,7 @@ namespace Slang
         flags |= D3DCOMPILE_ENABLE_STRICTNESS;
         flags |= D3DCOMPILE_ENABLE_UNBOUNDED_DESCRIPTOR_TABLES;
 
-        const String sourcePath = calcTranslationUnitSourcePath(entryPoint->getTranslationUnit());
+        const String sourcePath = "slang-geneated";// calcTranslationUnitSourcePath(entryPoint->getTranslationUnit());
 
         ComPtr<ID3DBlob> codeBlob;
         ComPtr<ID3DBlob> diagnosticsBlob;
@@ -626,7 +752,7 @@ namespace Slang
             sourcePath.Buffer(),
             dxMacros,
             nullptr,
-            getText(entryPoint->name).begin(),
+            getText(entryPoint->getName()).begin(),
             GetHLSLProfileName(profile).Buffer(),
             flags,
             0, // unused: effect flags
@@ -640,23 +766,24 @@ namespace Slang
 
         if (FAILED(hr))
         {
-            reportExternalCompileError("fxc", hr, _getSlice(diagnosticsBlob), &entryPoint->compileRequest->mSink);
+            reportExternalCompileError("fxc", hr, _getSlice(diagnosticsBlob), sink);
         }
                 
         return hr;
     }
 
     SlangResult dissassembleDXBC(
-        CompileRequest*     compileRequest,
-        void const*         data,
-        size_t              size, 
-        String&             assemOut)
+        BackEndCompileRequest*  compileRequest,
+        void const*             data,
+        size_t                  size, 
+        String&                 assemOut)
     {
         assemOut = String();
 
-        auto session = compileRequest->mSession;
+        auto session = compileRequest->getSession();
+        auto sink = compileRequest->getSink();
 
-        auto disassembleFunc = (pD3DDisassemble)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Fxc_D3DDisassemble, &compileRequest->mSink);
+        auto disassembleFunc = (pD3DDisassemble)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Fxc_D3DDisassemble, sink);
         if (!disassembleFunc)
         {
             return SLANG_E_NOT_FOUND;
@@ -677,25 +804,34 @@ namespace Slang
         if (FAILED(res))
         {
             // TODO(tfoley): need to figure out what to diagnose here...
-            reportExternalCompileError("fxc", res, UnownedStringSlice(), &compileRequest->mSink);
+            reportExternalCompileError("fxc", res, UnownedStringSlice(), sink);
         }
 
         return res;
     }
 
     SlangResult emitDXBytecodeAssemblyForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq,
-        String&             assemOut)
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq,
+        String&                 assemOut)
     {
 
         List<uint8_t> dxbc;
-        SLANG_RETURN_ON_FAIL(emitDXBytecodeForEntryPoint(entryPoint, targetReq, dxbc));
+        SLANG_RETURN_ON_FAIL(emitDXBytecodeForEntryPoint(
+            compileRequest,
+            entryPoint,
+            entryPointIndex,
+            targetReq,
+            endToEndReq,
+            dxbc));
         if (!dxbc.Count())
         {
             return SLANG_FAIL;
         }
-        return dissassembleDXBC(entryPoint->compileRequest, dxbc.Buffer(), dxbc.Count(), assemOut);
+        return dissassembleDXBC(compileRequest, dxbc.Buffer(), dxbc.Count(), assemOut);
     }
 #endif
 
@@ -704,26 +840,30 @@ namespace Slang
 // Implementations in `dxc-support.cpp`
 
 int emitDXILForEntryPointUsingDXC(
-    EntryPointRequest*  entryPoint,
-    TargetRequest*      targetReq,
-    List<uint8_t>&      outCode);
+    BackEndCompileRequest*  compileRequest,
+    EntryPoint*             entryPoint,
+    Int                     entryPointIndex,
+    TargetRequest*          targetReq,
+    EndToEndCompileRequest* endToEndReq,
+    List<uint8_t>&          outCode);
 
 SlangResult dissassembleDXILUsingDXC(
-    CompileRequest*     compileRequest,
-    void const*         data,
-    size_t              size, 
-    String&             stringOut);
+    BackEndCompileRequest*  compileRequest,
+    void const*             data,
+    size_t                  size, 
+    String&                 stringOut);
 
 #endif
 
 #if SLANG_ENABLE_GLSLANG_SUPPORT
     SlangResult invokeGLSLCompiler(
-        CompileRequest*             slangCompileRequest,
+        BackEndCompileRequest*      slangCompileRequest,
         glslang_CompileRequest&     request)
     {
-        Session* session = slangCompileRequest->mSession;
+        Session* session = slangCompileRequest->getSession();
+        auto sink = slangCompileRequest->getSink();
 
-        auto glslang_compile = (glslang_CompileFunc)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Glslang_Compile, &slangCompileRequest->mSink);
+        auto glslang_compile = (glslang_CompileFunc)session->getSharedLibraryFunc(Session::SharedLibraryFuncType::Glslang_Compile, sink);
         if (!glslang_compile)
         {
             return SLANG_FAIL;
@@ -743,7 +883,7 @@ SlangResult dissassembleDXILUsingDXC(
 
         if (err)
         {
-            reportExternalCompileError("glslang", SLANG_FAIL, diagnosticOutput.getUnownedSlice(), &slangCompileRequest->mSink);
+            reportExternalCompileError("glslang", SLANG_FAIL, diagnosticOutput.getUnownedSlice(), sink);
             return SLANG_FAIL;
         }
 
@@ -751,10 +891,10 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     SlangResult dissassembleSPIRV(
-        CompileRequest*     slangRequest,
-        void const*         data,
-        size_t              size, 
-        String&             stringOut)
+        BackEndCompileRequest*  slangRequest,
+        void const*             data,
+        size_t                  size, 
+        String&                 stringOut)
     {
         stringOut = String();
 
@@ -782,21 +922,29 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     SlangResult emitSPIRVForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq,
-        List<uint8_t>&      spirvOut)
+        BackEndCompileRequest*  slangRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq,
+        List<uint8_t>&          spirvOut)
     {
         spirvOut.Clear();
 
-        String rawGLSL = emitGLSLForEntryPoint(entryPoint, targetReq);
-        maybeDumpIntermediate(entryPoint->compileRequest, rawGLSL.Buffer(), CodeGenTarget::GLSL);
+        String rawGLSL = emitGLSLForEntryPoint(
+            slangRequest,
+            entryPoint,
+            entryPointIndex,
+            targetReq,
+            endToEndReq);
+        maybeDumpIntermediate(slangRequest, rawGLSL.Buffer(), CodeGenTarget::GLSL);
 
         auto outputFunc = [](void const* data, size_t size, void* userData)
         {
             ((List<uint8_t>*)userData)->AddRange((uint8_t*)data, size);
         };
 
-        const String sourcePath = calcTranslationUnitSourcePath(entryPoint->getTranslationUnit());
+        const String sourcePath = calcSourcePathForEntryPoint(endToEndReq, entryPointIndex);
 
         glslang_CompileRequest request;
         request.action = GLSLANG_ACTION_COMPILE_GLSL_TO_SPIRV;
@@ -809,40 +957,56 @@ SlangResult dissassembleDXILUsingDXC(
         request.outputFunc = outputFunc;
         request.outputUserData = &spirvOut;
 
-        SLANG_RETURN_ON_FAIL(invokeGLSLCompiler(entryPoint->compileRequest, request));
+        SLANG_RETURN_ON_FAIL(invokeGLSLCompiler(slangRequest, request));
         return SLANG_OK;
     }
 
     SlangResult emitSPIRVAssemblyForEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq,
-        String&             assemblyOut)
+        BackEndCompileRequest*  slangRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq,
+        String&                 assemblyOut)
     {
         List<uint8_t> spirv;
-        SLANG_RETURN_ON_FAIL(emitSPIRVForEntryPoint(entryPoint, targetReq, spirv));
+        SLANG_RETURN_ON_FAIL(emitSPIRVForEntryPoint(
+            slangRequest,
+            entryPoint,
+            entryPointIndex,
+            targetReq,
+            endToEndReq,
+            spirv));
 
         if (spirv.Count() == 0)
             return SLANG_FAIL;
 
-        return dissassembleSPIRV(entryPoint->compileRequest, spirv.begin(), spirv.Count(), assemblyOut);
+        return dissassembleSPIRV(slangRequest, spirv.begin(), spirv.Count(), assemblyOut);
     }
 #endif
 
     // Do emit logic for a single entry point
     CompileResult emitEntryPoint(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq)
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
+        Int                     entryPointIndex,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq)
     {
         CompileResult result;
 
-        auto compileRequest = entryPoint->compileRequest;
         auto target = targetReq->target;
 
         switch (target)
         {
         case CodeGenTarget::HLSL:
             {
-                String code = emitHLSLForEntryPoint(entryPoint, targetReq);
+                String code = emitHLSLForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq);
                 maybeDumpIntermediate(compileRequest, code.Buffer(), target);
                 result = CompileResult(code);
             }
@@ -850,7 +1014,12 @@ SlangResult dissassembleDXILUsingDXC(
 
         case CodeGenTarget::GLSL:
             {
-                String code = emitGLSLForEntryPoint(entryPoint, targetReq);
+                String code = emitGLSLForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq);
                 maybeDumpIntermediate(compileRequest, code.Buffer(), target);
                 result = CompileResult(code);
             }
@@ -860,7 +1029,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::DXBytecode:
             {
                 List<uint8_t> code;
-                if (SLANG_SUCCEEDED(emitDXBytecodeForEntryPoint(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitDXBytecodeForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     maybeDumpIntermediate(compileRequest, code.Buffer(), code.Count(), target);
                     result = CompileResult(code);
@@ -871,7 +1046,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::DXBytecodeAssembly:
             {
                 String code;
-                if (SLANG_SUCCEEDED(emitDXBytecodeAssemblyForEntryPoint(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitDXBytecodeAssemblyForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     maybeDumpIntermediate(compileRequest, code.Buffer(), target);
                     result = CompileResult(code);
@@ -884,7 +1065,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::DXIL:
             {
                 List<uint8_t> code;
-                if (SLANG_SUCCEEDED(emitDXILForEntryPointUsingDXC(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitDXILForEntryPointUsingDXC(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     maybeDumpIntermediate(compileRequest, code.Buffer(), code.Count(), target);
                     result = CompileResult(code);
@@ -895,7 +1082,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::DXILAssembly:
             {
                 List<uint8_t> code;
-                if (SLANG_SUCCEEDED(emitDXILForEntryPointUsingDXC(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitDXILForEntryPointUsingDXC(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     String assembly; 
                     dissassembleDXILUsingDXC(
@@ -915,7 +1108,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::SPIRV:
             {
                 List<uint8_t> code;
-                if (SLANG_SUCCEEDED(emitSPIRVForEntryPoint(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitSPIRVForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     maybeDumpIntermediate(compileRequest, code.Buffer(), code.Count(), target);
                     result = CompileResult(code);
@@ -926,7 +1125,13 @@ SlangResult dissassembleDXILUsingDXC(
         case CodeGenTarget::SPIRVAssembly:
             {
                 String code;
-                if (SLANG_SUCCEEDED(emitSPIRVAssemblyForEntryPoint(entryPoint, targetReq, code)))
+                if (SLANG_SUCCEEDED(emitSPIRVAssemblyForEntryPoint(
+                    compileRequest,
+                    entryPoint,
+                    entryPointIndex,
+                    targetReq,
+                    endToEndReq,
+                    code)))
                 {
                     maybeDumpIntermediate(compileRequest, code.Buffer(), target);
                     result = CompileResult(code);
@@ -957,16 +1162,16 @@ SlangResult dissassembleDXILUsingDXC(
     };
 
     static void writeOutputFile(
-        CompileRequest* compileRequest,
-        FILE*           file,
-        String const&   path,
-        void const*     data,
-        size_t          size)
+        BackEndCompileRequest*  compileRequest,
+        FILE*                   file,
+        String const&           path,
+        void const*             data,
+        size_t                  size)
     {
         size_t count = fwrite(data, size, 1, file);
         if (count != 1)
         {
-            compileRequest->mSink.diagnose(
+            compileRequest->getSink()->diagnose(
                 SourceLoc(),
                 Diagnostics::cannotWriteOutputFile,
                 path);
@@ -974,16 +1179,16 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     static void writeOutputFile(
-        CompileRequest* compileRequest,
-        ISlangWriter*   writer, 
-        String const&   path,
-        void const*     data,
-        size_t          size)
+        BackEndCompileRequest*  compileRequest,
+        ISlangWriter*           writer,
+        String const&           path,
+        void const*             data,
+        size_t                  size)
     {
 
         if (SLANG_FAILED(writer->write((const char*)data, size)))
         {
-            compileRequest->mSink.diagnose(
+            compileRequest->getSink()->diagnose(
                 SourceLoc(),
                 Diagnostics::cannotWriteOutputFile,
                 path);
@@ -991,18 +1196,18 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     static void writeOutputFile(
-        CompileRequest* compileRequest,
-        String const&   path,
-        void const*     data,
-        size_t          size,
-        OutputFileKind  kind)
+        BackEndCompileRequest*  compileRequest,
+        String const&           path,
+        void const*             data,
+        size_t                  size,
+        OutputFileKind          kind)
     {
         FILE* file = fopen(
             path.Buffer(),
             kind == OutputFileKind::Binary ? "wb" : "w");
         if (!file)
         {
-            compileRequest->mSink.diagnose(
+            compileRequest->getSink()->diagnose(
                 SourceLoc(),
                 Diagnostics::cannotWriteOutputFile,
                 path);
@@ -1014,11 +1219,12 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     static void writeEntryPointResultToFile(
-        EntryPointRequest*      entryPoint,
+        BackEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
         String const&           outputPath,
         CompileResult const&    result)
     {
-        auto compileRequest = entryPoint->compileRequest;
+        SLANG_UNUSED(entryPoint);
 
         switch (result.format)
         {
@@ -1059,13 +1265,15 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     static void writeEntryPointResultToStandardOutput(
-        EntryPointRequest*      entryPoint,
+        EndToEndCompileRequest*  compileRequest,
+        EntryPoint*             entryPoint,
         TargetRequest*          targetReq,
         CompileResult const&    result)
     {
-        auto compileRequest = entryPoint->compileRequest;
+        SLANG_UNUSED(entryPoint);
 
         ISlangWriter* writer = compileRequest->getWriter(WriterChannel::StdOutput);
+        auto backEndReq = compileRequest->getBackEndReq();
 
         switch (result.format)
         {
@@ -1087,7 +1295,7 @@ SlangResult dissassembleDXILUsingDXC(
                     case CodeGenTarget::DXBytecode:
                         {
                             String assembly;
-                            dissassembleDXBC(compileRequest,
+                            dissassembleDXBC(backEndReq,
                                 data.begin(),
                                 data.end() - data.begin(), assembly);
                             writeOutputToConsole(writer, assembly);
@@ -1099,7 +1307,7 @@ SlangResult dissassembleDXILUsingDXC(
                     case CodeGenTarget::DXIL:
                         {
                             String assembly; 
-                            dissassembleDXILUsingDXC(compileRequest,
+                            dissassembleDXILUsingDXC(backEndReq,
                                 data.begin(),
                                 data.end() - data.begin(), 
                                 assembly);
@@ -1111,7 +1319,7 @@ SlangResult dissassembleDXILUsingDXC(
                     case CodeGenTarget::SPIRV:
                         {
                             String assembly;
-                            dissassembleSPIRV(compileRequest,
+                            dissassembleSPIRV(backEndReq,
                                 data.begin(),
                                 data.end() - data.begin(), assembly);
                             writeOutputToConsole(writer, assembly);
@@ -1129,7 +1337,7 @@ SlangResult dissassembleDXILUsingDXC(
                     writer->setMode(SLANG_WRITER_MODE_BINARY);
 
                     writeOutputFile(
-                        compileRequest,
+                        backEndReq,
                         writer,
                         "stdout",
                         data.begin(),
@@ -1146,89 +1354,108 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     static void writeEntryPointResult(
-        EntryPointRequest*  entryPoint,
-        TargetRequest*      targetReq,
-        UInt                entryPointIndex)
+        EndToEndCompileRequest* compileRequest,
+        EntryPoint*             entryPoint,
+        TargetRequest*          targetReq,
+        Int                     entryPointIndex)
     {
-        // It is possible that we are dynamically discovering entry
-        // points (using `[shader(...)]` attributes), so that the
-        // number of entry points on the compile request does not
-        // match the number of entries in the `entryPointOutputPaths`
-        // array.
-        //
-        String outputPath;
-        if( entryPointIndex < targetReq->entryPointOutputPaths.Count() )
-        {
-            outputPath = targetReq->entryPointOutputPaths[entryPointIndex];
-        }
+        auto program = compileRequest->getSpecializedProgram();
+        auto targetProgram = program->getTargetProgram(targetReq);
+        auto backEndReq = compileRequest->getBackEndReq();
 
-        auto& result = targetReq->entryPointResults[entryPointIndex];
+        auto& result = targetProgram->getExistingEntryPointResult(entryPointIndex);
 
         // Skip the case with no output
         if (result.format == ResultFormat::None)
             return;
 
-        if (outputPath.Length())
-        {
-            writeEntryPointResultToFile(entryPoint, outputPath, result);
-        }
-        else
+        // It is possible that we are dynamically discovering entry
+        // points (using `[shader(...)]` attributes), so that there
+        // might be entry points added to the program that did not
+        // get paths specified via command-line options.
+        //
+        RefPtr<EndToEndCompileRequest::TargetInfo> targetInfo;
+        if(compileRequest->targetInfos.TryGetValue(targetReq, targetInfo))
         {
-            writeEntryPointResultToStandardOutput(entryPoint, targetReq, result);
+            String outputPath;
+            if(targetInfo->entryPointOutputPaths.TryGetValue(entryPointIndex, outputPath))
+            {
+                writeEntryPointResultToFile(backEndReq, entryPoint, outputPath, result);
+                return;
+            }
         }
+
+        writeEntryPointResultToStandardOutput(compileRequest, entryPoint, targetReq, result);
     }
 
     void generateOutputForTarget(
-        TargetRequest*  targetReq)
+        BackEndCompileRequest*  compileReq,
+        TargetRequest*          targetReq,
+        EndToEndCompileRequest* endToEndReq)
     {
-        CompileRequest* compileReq = targetReq->compileRequest;
+        auto program = compileReq->getProgram();
+        auto targetProgram = program->getTargetProgram(targetReq);
 
         // Generate target code any entry points that
         // have been requested for compilation.
-        for (auto& entryPoint : compileReq->entryPoints)
+        auto entryPointCount = program->getEntryPointCount();
+        for(UInt ii = 0; ii < entryPointCount; ++ii)
         {
-            CompileResult entryPointResult = emitEntryPoint(entryPoint, targetReq);
-            targetReq->entryPointResults.Add(entryPointResult);
+            auto entryPoint = program->getEntryPoint(ii);
+            CompileResult entryPointResult = emitEntryPoint(
+                compileReq,
+                entryPoint,
+                ii,
+                targetReq,
+                endToEndReq);
+            targetProgram->setEntryPointResult(ii, entryPointResult);
         }
     }
 
-    void generateOutput(
-        CompileRequest* compileRequest)
+    static void _generateOutput(
+        BackEndCompileRequest* compileRequest,
+        EndToEndCompileRequest* endToEndReq)
     {
         // Go through the code-generation targets that the user
         // has specified, and generate code for each of them.
         //
-        for (auto targetReq : compileRequest->targets)
+        auto linkage = compileRequest->getLinkage();
+        for (auto targetReq : linkage->targets)
         {
-            generateOutputForTarget(targetReq);
+            generateOutputForTarget(compileRequest, targetReq, endToEndReq);
         }
+    }
+
+    void generateOutput(
+        BackEndCompileRequest* compileRequest)
+    {
+        _generateOutput(compileRequest, nullptr);
+    }
+
+    void generateOutput(
+        EndToEndCompileRequest* compileRequest)
+    {
+        _generateOutput(compileRequest->getBackEndReq(), compileRequest);
 
         // If we are in command-line mode, we might be expected to actually
         // write output to one or more files here.
 
         if (compileRequest->isCommandLineCompile)
         {
-            for (auto targetReq : compileRequest->targets)
+            auto linkage = compileRequest->getLinkage();
+            auto program = compileRequest->getSpecializedProgram();
+            for (auto targetReq : linkage->targets)
             {
-                UInt entryPointCount = compileRequest->entryPoints.Count();
+                UInt entryPointCount = program->getEntryPointCount();
                 for (UInt ee = 0; ee < entryPointCount; ++ee)
                 {
                     writeEntryPointResult(
-                        compileRequest->entryPoints[ee],
+                        compileRequest,
+                        program->getEntryPoint(ee),
                         targetReq,
                         ee);
                 }
             }
-
-            if (compileRequest->containerOutputPath.Length() != 0)
-            {
-                auto& data = compileRequest->generatedBytecode;
-                writeOutputFile(compileRequest,
-                    compileRequest->containerOutputPath,
-                    data.begin(),
-                    data.end() - data.begin(),
-                    OutputFileKind::Binary);
-            }
         }
     }
 
@@ -1237,7 +1464,7 @@ SlangResult dissassembleDXILUsingDXC(
     //
 
     void dumpIntermediate(
-        CompileRequest*,
+        BackEndCompileRequest*,
         void const*     data,
         size_t          size,
         char const*     ext,
@@ -1271,7 +1498,7 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     void dumpIntermediateText(
-        CompileRequest* compileRequest,
+        BackEndCompileRequest* compileRequest,
         void const*     data,
         size_t          size,
         char const*     ext)
@@ -1280,7 +1507,7 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     void dumpIntermediateBinary(
-        CompileRequest* compileRequest,
+        BackEndCompileRequest* compileRequest,
         void const*     data,
         size_t          size,
         char const*     ext)
@@ -1289,7 +1516,7 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     void maybeDumpIntermediate(
-        CompileRequest* compileRequest,
+        BackEndCompileRequest* compileRequest,
         void const*     data,
         size_t          size,
         CodeGenTarget   target)
@@ -1362,7 +1589,7 @@ SlangResult dissassembleDXILUsingDXC(
     }
 
     void maybeDumpIntermediate(
-        CompileRequest* compileRequest,
+        BackEndCompileRequest* compileRequest,
         char const*     text,
         CodeGenTarget   target)
     {