IDownstreamCompiler interface (#2361)

* WIP with hierarchical enums.

* Some small fixes and improvements around artifact desc related types.

* Improvements around hierarchical enum.

* Fixes to get Artifact types refactor to be able to execute tests.

* Attempt to better categorize PTX.

* Work around for potentially unused function warning.

* Typo fix.

* Simplify Artifact header.

* Small improvements around Artifact kind/payload/style.

* Added IDestroyable/ICastable

* Add IArtifactList.

* First impl of IArtifactUtil.

* Use the ICastable interface for IArtifactRepresentation.

* Added IArtifactRepresentation & IArtifactAssociated.

* Add SLANG_OVERRIDE to avoid gcc/clang warning.

* Fix calling convention issue on win32.

* Fix missing SLANG_OVERRIDE.

* First attempt at file abstraction around Artifact.

* Added creation of lock file.

* Move functionality for determining file paths to the IArtifactUtil.
Add casting to ICastable.

* Added some casting/finding mechanisms.

* Simplify IArtifact interface, and use Items for file reps.

* Fix problem with libraries on DXIL.

* Split out ArtifactRepresentation.

* Move ArtifactDesc functionality to ArtifactDescUtil. ArtifactInfoUtil becomes ArtifactDescUtil.

* Split implementations from the interfaces for Artifact.

* Use TypeTextUtil for target name outputting.

* Add artifact impls.

* Add ICastableList

* Added UnknownCastableAdapter

* Make ISlangSharedLibrary derive from ICastable, and remain backwards compatible with slang-llvm.

* Refactor Representation on Artifact.

* Make our ISlangBlobs also derive from ICastable.
Make ISlangBlob atomic ref counted.

* Split out CastableList and related types, and placed in core.

* Small fixes around IArtifact.
Improve IArtifact docs.
First impl of getChildren for IArtifact.

* Documentation improvements for Artifact related types.

* Fix typo.

* Special case adding a ICastableList to a LazyCastableList.

* Small simplification of LazyCastableList, by adding State member.

* Removed the ILockFile interface because IFileArtifactRepresentation can be used.

* Implement DiagnosticsArtifactRepresentation.

* Added PostEmitMetadataArtifactRepresentation

* Add searching by predicate.
Added handling of accessing Artifact as ISharedLibrary

* Fix typo.

* Add find to IArtifacgtList.
Fix some missing SLANG_NO_THROW.

* Small improvements around ArtifactDesc types.

* Another small change around ArtifactKind.

* Some more shuffling of ArtifactDesc.

* Make IArtifact castable
Remove IArtifactList
Made IArtifactContainer derive from IArtifact
Made ModuleLibrary atomic ref counted/given IModuleLibrary interface.

* Must call _requireChildren before any children access.

* Fix missing SLANG_MCALL on castAs.

* Fix missing SLANG_OVERRIDE.

* Added IArtifactHandler

* Use ICastable for basis of scope/lookup.

* WIP first attempt to remove CompileResult.

* Fix support for for downstream compiler shared library adapter.

* Fix issues found when replacing CompileResult.

* Fix typo.

* Fix getting items form 'significant' member of an Artifact.

* Split out ArtifactUtil & ArtifactHandler.

* Work around for problem on Visual studio.

* Improve searching.

* Add missing files.

* Split out Artifact associated types.
Don't produce a container by default - use associated for 'metadata'.

* Remove no longer used ArtifactPayload type.

* Generalized converting representations.
Small improvements to artifacts.

* Fix intermediate dumping issue.

* Removed #if 0 out CompileResult.
Remove DownstreamCompileResult maybeDumpIntermediate.

* Pull out functionality for dumping artifact output into ArtifactOutputUtil
Fixed a bug in naming files based on ArtifactDesc.

* std::atomic issue.

* Pull out types from DownstreamCompile to simplify moving to an interface.

* Fix typo.

* Use IDownstreamCompiler interface.
Split out DownstreamCompilerUtil and DownstreamCompilerSet.

* Update projects.

* Fix missing SLANG_MCALL.

* Fix calling convention of IDownstreamCompiler impls.

* Split out binary work arounds into a dep1.cpp/h

* Small reorganising around DownstreamCompilerInfo.

* Remove Desc library functionality to DownstreamCompilerUtil.

* Expand IDiagnostics interface.
Rename associated impls with Impl suffix.

* Fix outputting as text bug.
Some small improvements.

* Add fix around prefix for dumping.
Improved how handling for extensions work form ArtifactDesc.

* Dump assembly if available.

* Simplify some of Dep1 definitions.

8 files changed, 40 insertions, 39 deletions

diff --git a/source/slang/slang-artifact-output-util.cpp b/source/slang/slang-artifact-output-util.cpp
index 2694fcb10..ccf49ed30 100644
--- a/source/slang/slang-artifact-output-util.cpp
+++ b/source/slang/slang-artifact-output-util.cpp
@@ -51,7 +51,7 @@ static CodeGenTarget _getDisassemblyCodeGenTarget(const ArtifactDesc& desc)
     auto downstreamCompiler = getDownstreamCompilerRequiredForTarget(target);
 
     // Get the required downstream compiler
-    DownstreamCompiler* compiler = session->getOrLoadDownstreamCompiler(downstreamCompiler, sink);
+    IDownstreamCompiler* compiler = session->getOrLoadDownstreamCompiler(downstreamCompiler, sink);
 
     if (!compiler)
     {
diff --git a/source/slang/slang-check.cpp b/source/slang/slang-check.cpp
index 633dec215..bcc74a6d0 100644
--- a/source/slang/slang-check.cpp
+++ b/source/slang/slang-check.cpp
@@ -83,7 +83,7 @@ namespace Slang
         m_downstreamCompilers[int(type)].setNull();
     }
 
-    DownstreamCompiler* Session::getOrLoadDownstreamCompiler(PassThroughMode type, DiagnosticSink* sink)
+    IDownstreamCompiler* Session::getOrLoadDownstreamCompiler(PassThroughMode type, DiagnosticSink* sink)
     {
         if (m_downstreamCompilerInitialized & (1 << int(type)))
         {
@@ -144,7 +144,7 @@ namespace Slang
             DownstreamCompilerUtil::updateDefaults(m_downstreamCompilerSet);
         }
 
-        DownstreamCompiler* compiler = nullptr;
+        IDownstreamCompiler* compiler = nullptr;
 
         if (type == PassThroughMode::GenericCCpp)
         {
@@ -152,7 +152,7 @@ namespace Slang
         }
         else
         {
-            DownstreamCompiler::Desc desc;
+            DownstreamCompilerDesc desc;
             desc.type = SlangPassThrough(type);
             compiler = DownstreamCompilerUtil::findCompiler(m_downstreamCompilerSet, DownstreamCompilerUtil::MatchType::Newest, desc);
         }
diff --git a/source/slang/slang-compiler.cpp b/source/slang/slang-compiler.cpp
index 02d312b61..d175277c8 100644
--- a/source/slang/slang-compiler.cpp
+++ b/source/slang/slang-compiler.cpp
@@ -828,7 +828,7 @@ namespace Slang
         /// and generally there will not be a match with emitted source.
         ///
         /// This test is only used for pass through mode. 
-    static bool _useEmittedSource(DownstreamCompiler* compiler, TranslationUnitRequest* translationUnit)
+    static bool _useEmittedSource(IDownstreamCompiler* compiler, TranslationUnitRequest* translationUnit)
     {
         // We only bother if it's a file based compiler.
         if (compiler->isFileBased())
@@ -952,7 +952,7 @@ namespace Slang
         SLANG_ASSERT(compilerType != PassThroughMode::None);
 
         // Get the required downstream compiler
-        DownstreamCompiler* compiler = session->getOrLoadDownstreamCompiler(compilerType, sink);
+        IDownstreamCompiler* compiler = session->getOrLoadDownstreamCompiler(compilerType, sink);
         if (!compiler)
         {
             auto compilerName = TypeTextUtil::getPassThroughAsHumanText((SlangPassThrough)compilerType);
@@ -963,7 +963,7 @@ namespace Slang
         Dictionary<String, String> preprocessorDefinitions;
         List<String> includePaths;
 
-        typedef DownstreamCompiler::CompileOptions CompileOptions;
+        typedef DownstreamCompileOptions CompileOptions;
         CompileOptions options;
 
         // Set compiler specific args
@@ -1115,8 +1115,8 @@ namespace Slang
 
                 if (cudaTracker->m_smVersion.isSet())
                 {
-                    DownstreamCompiler::CapabilityVersion version;
-                    version.kind = DownstreamCompiler::CapabilityVersion::Kind::CUDASM;
+                    DownstreamCompileOptions::CapabilityVersion version;
+                    version.kind = DownstreamCompileOptions::CapabilityVersion::Kind::CUDASM;
                     version.version = cudaTracker->m_smVersion;
 
                     options.requiredCapabilityVersions.add(version);
@@ -1129,8 +1129,8 @@ namespace Slang
             }
             else if (GLSLExtensionTracker* glslTracker = as<GLSLExtensionTracker>(extensionTracker))
             {
-                DownstreamCompiler::CapabilityVersion version;
-                version.kind = DownstreamCompiler::CapabilityVersion::Kind::SPIRV;
+                DownstreamCompileOptions::CapabilityVersion version;
+                version.kind = DownstreamCompileOptions::CapabilityVersion::Kind::SPIRV;
                 version.version = glslTracker->getSPIRVVersion();
 
                 options.requiredCapabilityVersions.add(version);
@@ -1260,28 +1260,28 @@ namespace Slang
 
             switch (linkage->optimizationLevel)
             {
-                case OptimizationLevel::None:       options.optimizationLevel = DownstreamCompiler::OptimizationLevel::None; break;
-                case OptimizationLevel::Default:    options.optimizationLevel = DownstreamCompiler::OptimizationLevel::Default;  break;
-                case OptimizationLevel::High:       options.optimizationLevel = DownstreamCompiler::OptimizationLevel::High;  break;
-                case OptimizationLevel::Maximal:    options.optimizationLevel = DownstreamCompiler::OptimizationLevel::Maximal;  break;
+                case OptimizationLevel::None:       options.optimizationLevel = DownstreamCompileOptions::OptimizationLevel::None; break;
+                case OptimizationLevel::Default:    options.optimizationLevel = DownstreamCompileOptions::OptimizationLevel::Default;  break;
+                case OptimizationLevel::High:       options.optimizationLevel = DownstreamCompileOptions::OptimizationLevel::High;  break;
+                case OptimizationLevel::Maximal:    options.optimizationLevel = DownstreamCompileOptions::OptimizationLevel::Maximal;  break;
                 default: SLANG_ASSERT(!"Unhandled optimization level"); break;
             }
 
             switch (linkage->debugInfoLevel)
             {
-                case DebugInfoLevel::None:          options.debugInfoType = DownstreamCompiler::DebugInfoType::None; break; 
-                case DebugInfoLevel::Minimal:       options.debugInfoType = DownstreamCompiler::DebugInfoType::Minimal; break; 
+                case DebugInfoLevel::None:          options.debugInfoType = DownstreamCompileOptions::DebugInfoType::None; break;
+                case DebugInfoLevel::Minimal:       options.debugInfoType = DownstreamCompileOptions::DebugInfoType::Minimal; break;
                 
-                case DebugInfoLevel::Standard:      options.debugInfoType = DownstreamCompiler::DebugInfoType::Standard; break; 
-                case DebugInfoLevel::Maximal:       options.debugInfoType = DownstreamCompiler::DebugInfoType::Maximal; break; 
+                case DebugInfoLevel::Standard:      options.debugInfoType = DownstreamCompileOptions::DebugInfoType::Standard; break;
+                case DebugInfoLevel::Maximal:       options.debugInfoType = DownstreamCompileOptions::DebugInfoType::Maximal; break;
                 default: SLANG_ASSERT(!"Unhandled debug level"); break;
             }
 
             switch( getTargetReq()->getFloatingPointMode())
             {
-                case FloatingPointMode::Default:    options.floatingPointMode = DownstreamCompiler::FloatingPointMode::Default; break;
-                case FloatingPointMode::Precise:    options.floatingPointMode = DownstreamCompiler::FloatingPointMode::Precise; break;
-                case FloatingPointMode::Fast:       options.floatingPointMode = DownstreamCompiler::FloatingPointMode::Fast; break;
+                case FloatingPointMode::Default:    options.floatingPointMode = DownstreamCompileOptions::FloatingPointMode::Default; break;
+                case FloatingPointMode::Precise:    options.floatingPointMode = DownstreamCompileOptions::FloatingPointMode::Precise; break;
+                case FloatingPointMode::Fast:       options.floatingPointMode = DownstreamCompileOptions::FloatingPointMode::Fast; break;
                 default: SLANG_ASSERT(!"Unhandled floating point mode");
             }
 
@@ -1311,7 +1311,7 @@ namespace Slang
                         break;
 
                     case Stage::Compute:
-                        options.pipelineType = DownstreamCompiler::PipelineType::Compute;
+                        options.pipelineType = DownstreamCompileOptions::PipelineType::Compute;
                         break;
 
                     case Stage::Vertex:
@@ -1319,7 +1319,7 @@ namespace Slang
                     case Stage::Domain:
                     case Stage::Geometry:
                     case Stage::Fragment:
-                        options.pipelineType = DownstreamCompiler::PipelineType::Rasterization;
+                        options.pipelineType = DownstreamCompileOptions::PipelineType::Rasterization;
                         break;
 
                     case Stage::RayGeneration:
@@ -1328,7 +1328,7 @@ namespace Slang
                     case Stage::ClosestHit:
                     case Stage::Miss:
                     case Stage::Callable:
-                        options.pipelineType = DownstreamCompiler::PipelineType::RayTracing;
+                        options.pipelineType = DownstreamCompileOptions::PipelineType::RayTracing;
                         break;
                     }
                 }   
@@ -1341,7 +1341,7 @@ namespace Slang
             {
                 for(auto& def : preprocessorDefinitions)
                 {
-                    DownstreamCompiler::Define define;
+                    DownstreamCompileOptions::Define define;
                     define.nameWithSig = def.Key;
                     define.value = def.Value;
 
@@ -1366,7 +1366,7 @@ namespace Slang
         if (diagnostics.diagnostics.getCount())
         {
             StringBuilder compilerText;
-            compiler->getDesc().appendAsText(compilerText);
+            DownstreamCompilerUtil::appendAsText(compiler->getDesc(), compilerText);
 
             StringBuilder builder;
 
diff --git a/source/slang/slang-compiler.h b/source/slang/slang-compiler.h
index 6b79b2679..97111f540 100755
--- a/source/slang/slang-compiler.h
+++ b/source/slang/slang-compiler.h
@@ -7,6 +7,8 @@
 #include "../core/slang-file-system.h"
 
 #include "../compiler-core/slang-downstream-compiler.h"
+#include "../compiler-core/slang-downstream-compiler-util.h"
+
 #include "../compiler-core/slang-name.h"
 #include "../compiler-core/slang-include-system.h"
 #include "../compiler-core/slang-command-line-args.h"
@@ -2802,7 +2804,7 @@ namespace Slang
         }
         
             /// Get the downstream compiler for a transition
-        DownstreamCompiler* getDownstreamCompiler(CodeGenTarget source, CodeGenTarget target);
+        IDownstreamCompiler* getDownstreamCompiler(CodeGenTarget source, CodeGenTarget target);
         
         Scope* baseLanguageScope = nullptr;
         Scope* coreLanguageScope = nullptr;
@@ -2856,7 +2858,7 @@ namespace Slang
         void _setSharedLibraryLoader(ISlangSharedLibraryLoader* loader);
 
             /// Will try to load the library by specified name (using the set loader), if not one already available.
-        DownstreamCompiler* getOrLoadDownstreamCompiler(PassThroughMode type, DiagnosticSink* sink);
+        IDownstreamCompiler* getOrLoadDownstreamCompiler(PassThroughMode type, DiagnosticSink* sink);
             /// Will unload the specified shared library if it's currently loaded 
         void resetDownstreamCompiler(PassThroughMode type);
 
@@ -2883,7 +2885,7 @@ namespace Slang
         int m_downstreamCompilerInitialized = 0;                                        
 
         RefPtr<DownstreamCompilerSet> m_downstreamCompilerSet;                                  ///< Information about all available downstream compilers.
-        RefPtr<DownstreamCompiler> m_downstreamCompilers[int(PassThroughMode::CountOf)];        ///< A downstream compiler for a pass through
+        ComPtr<IDownstreamCompiler> m_downstreamCompilers[int(PassThroughMode::CountOf)];        ///< A downstream compiler for a pass through
         DownstreamCompilerLocatorFunc m_downstreamCompilerLocators[int(PassThroughMode::CountOf)];
         Name* m_completionTokenName = nullptr; ///< The name of a completion request token.
 
diff --git a/source/slang/slang-emit.cpp b/source/slang/slang-emit.cpp
index 2a1b78833..4cbfa97ab 100644
--- a/source/slang/slang-emit.cpp
+++ b/source/slang/slang-emit.cpp
@@ -837,7 +837,7 @@ Result linkAndOptimizeIR(
 #endif
     validateIRModuleIfEnabled(codeGenContext, irModule);
 
-    auto metadata = new PostEmitMetadata;
+    auto metadata = new PostEmitMetadataImpl;
     outLinkedIR.metadata = metadata;
 
     collectMetadata(irModule, *metadata);
diff --git a/source/slang/slang-ir-metadata.cpp b/source/slang/slang-ir-metadata.cpp
index 7d4694ec5..b6ab2b922 100644
--- a/source/slang/slang-ir-metadata.cpp
+++ b/source/slang/slang-ir-metadata.cpp
@@ -39,7 +39,7 @@ static void _insertBinding(List<ShaderBindingRange>& ranges, LayoutResourceKind
 }
 
 // Collects the metadata from the provided IR module, saves it in outMetadata.
-void collectMetadata(const IRModule* irModule, PostEmitMetadata& outMetadata)
+void collectMetadata(const IRModule* irModule, PostEmitMetadataImpl& outMetadata)
 {
     // Scan the instructions looking for global resource declarations
     for (const auto& inst : irModule->getGlobalInsts())
diff --git a/source/slang/slang-ir-metadata.h b/source/slang/slang-ir-metadata.h
index e72e10a94..f5e3229ae 100644
--- a/source/slang/slang-ir-metadata.h
+++ b/source/slang/slang-ir-metadata.h
@@ -4,9 +4,9 @@
 namespace Slang
 {
 
-class PostEmitMetadata;
+class PostEmitMetadataImpl;
 struct IRModule;
 
-void collectMetadata(const IRModule* irModule, PostEmitMetadata& outMetadata);
+void collectMetadata(const IRModule* irModule, PostEmitMetadataImpl& outMetadata);
 
 }
diff --git a/source/slang/slang.cpp b/source/slang/slang.cpp
index 710cc0b45..b86617f1f 100644
--- a/source/slang/slang.cpp
+++ b/source/slang/slang.cpp
@@ -695,7 +695,7 @@ SLANG_NO_THROW const char* SLANG_MCALL Session::getBuildTagString()
 
 SLANG_NO_THROW SlangResult SLANG_MCALL Session::setDefaultDownstreamCompiler(SlangSourceLanguage sourceLanguage, SlangPassThrough defaultCompiler)
 {
-    if (DownstreamCompiler::canCompile(defaultCompiler, sourceLanguage))
+    if (DownstreamCompilerInfo::canCompile(defaultCompiler, sourceLanguage))
     {
         m_defaultDownstreamCompilers[int(sourceLanguage)] = PassThroughMode(defaultCompiler);
         return SLANG_OK;
@@ -740,8 +740,7 @@ SlangPassThrough Session::getDownstreamCompilerForTransition(SlangCompileTarget
         (source == CodeGenTarget::CSource || source == CodeGenTarget::CPPSource))
     {
         // We prefer LLVM if it's available
-        DownstreamCompiler* llvm = getOrLoadDownstreamCompiler(PassThroughMode::LLVM, nullptr);
-        if (llvm)
+        if (auto llvm = getOrLoadDownstreamCompiler(PassThroughMode::LLVM, nullptr))
         {
             return SLANG_PASS_THROUGH_LLVM;
         }
@@ -760,7 +759,7 @@ SlangPassThrough Session::getDownstreamCompilerForTransition(SlangCompileTarget
     return SLANG_PASS_THROUGH_NONE;
 }
 
-DownstreamCompiler* Session::getDownstreamCompiler(CodeGenTarget source, CodeGenTarget target)
+IDownstreamCompiler* Session::getDownstreamCompiler(CodeGenTarget source, CodeGenTarget target)
 {
     PassThroughMode compilerType = (PassThroughMode)getDownstreamCompilerForTransition(SlangCompileTarget(source), SlangCompileTarget(target));
     return getOrLoadDownstreamCompiler(compilerType, nullptr);
@@ -5054,7 +5053,7 @@ SlangResult EndToEndCompileRequest::isParameterLocationUsed(Int entryPointIndex,
 
     
     // TODO: optimize this with a binary search through a sorted list
-    for (const auto& range : metadata->getBindingRanges())
+    for (const auto& range : metadata->getUsedBindingRanges())
     {
         if (range.containsBinding((slang::ParameterCategory)category, spaceIndex, registerIndex))
         {