Make render-test use Slang for all shader compilation (#597)

* Make render-test use Slang for all shader compilation

This streamlines the code for render-test by having all its shader compilation go through the Slang API, so that it doesn't have to deal with custom logic to compile HLSL->DXBC and HLSL->DXIL. We were already leaning on Slang to generate SPIR-V for Vulkan, so this makes all the paths more consistent.

My original plan with this change was to make the D3D12 render path start using DXIL at this point, since the change would make that easy, but it turns out that some aspects of how we handle parameter binding are not compatible with that right now, so it would need to come as a later change.

There's a lot of details here, so I will try to walk through the changes, including the incidental ones:

* Add logic to `premake5.lua` so that we copy the necessary libraries for HLSL shader compilation to our target directory from the Windows SDK. This is necessary so that our tests can actually invoke `dxcompiler.dll`

* Re-run Premake to generate new project files. This moves around a few files that I manually added in previous changes without re-running Premake.

* When invoking `fxc` as a pass-through compiler, be sure to pass along any macros defines via API or command-line. This isn't a strictly required change with how things worked out, but it is a positive one anyway, because it makes `slangc -pass-through fxc` more useful.

* Don't print output from a downstream `fxc` invocation if it produces warnings but no errors. The main reason for this is so that our tests don't fail because of `fxc` warnings on Slang's output (which then don't match the baselines), but it can also be rationalized as not wanting to confuse users with warnings that don't come from the "real" compiler they are using. This probably needs fine-tuning as a policy.

* Add the HLSL `NonUniformResourceIndex` function. This was an oversight because it isn't documented as a builtin on MSDN, and only gets mentioned obliquely when they talk about resource indexing.

* Add `glsl_<version>` profiles to match our `sm_<version>` profiles, so that it is easy for a user to use the profile mechanism to request a specific GLSL version without also specifying a stage name.

* Update the render-test logic so that there is a single `ShaderCompiler` implementation that *always* uses Slang, and get rid of all of the renderer-specific `ShaderCompiler` implementations.

* Update logic in render-test `main.cpp` to select the options to use for the eventual Slang compile based on the choice of renderer and input language. I didn't change the options that render-test exposes, even though they are getting increasingly silly (e.g., `-glsl-rewrite` doesn't use GLSL as its input...).

* Note: the D3D12 renderer will still use fxc, DXBC, and SM 5.0 for now, since trying to update it to switch to dxc, DXIL, and SM 6.0 didn't work well at the time.

* Add a bit of supporting D3D12 code to make sure that we don't allocate a structured buffer when a buffer has a format.

* Make sure to *also* define the `__HLSL__` macro when compiling Slang code, because otherwise a bunch of tests don't work (I'm not clear on how it worked before...).

* fixup: missing file

1 files changed, 157 insertions, 155 deletions

diff --git a/tools/render-test/slang-support.cpp b/tools/render-test/slang-support.cpp
index 342af571c..d69060449 100644
--- a/tools/render-test/slang-support.cpp
+++ b/tools/render-test/slang-support.cpp
@@ -11,167 +11,169 @@
 
 namespace renderer_test {
 
-struct SlangShaderCompilerWrapper : public ShaderCompiler
+ShaderProgram* ShaderCompiler::compileProgram(
+    ShaderCompileRequest const& request)
 {
-    ShaderCompiler*     innerCompiler;
-    SlangCompileTarget  target;
-    SlangSourceLanguage sourceLanguage;
-
-	virtual ShaderProgram* compileProgram(ShaderCompileRequest const& request) override
-	{
-		SlangSession* slangSession = spCreateSession(NULL);
-		SlangCompileRequest* slangRequest = spCreateCompileRequest(slangSession);
-
-		spSetCodeGenTarget(slangRequest, target);
-
-		// Define a macro so that shader code in a test can detect what language we
-		// are nominally working with.
-		char const* langDefine = nullptr;
-		switch (sourceLanguage)
-		{
-		case SLANG_SOURCE_LANGUAGE_GLSL:    langDefine = "__GLSL__";    break;
-		case SLANG_SOURCE_LANGUAGE_HLSL:    langDefine = "__HLSL__";    break;
-		case SLANG_SOURCE_LANGUAGE_SLANG:   langDefine = "__SLANG__";   break;
-		default:
-			assert(!"unexpected");
-			break;
-		}
-		spAddPreprocessorDefine(slangRequest, langDefine, "1");
-
-        // If we are dealing with GLSL input, then we need to set up
-        // Slang to pass through to glslang instead of actually running
-        // the compiler (this is a workaround to make direct comparisons
-        // possible)
-        if (sourceLanguage == SLANG_SOURCE_LANGUAGE_GLSL)
+    SlangSession* slangSession = spCreateSession(NULL);
+    SlangCompileRequest* slangRequest = spCreateCompileRequest(slangSession);
+
+    spSetCodeGenTarget(slangRequest, target);
+    spSetTargetProfile(slangRequest, 0,
+        spFindProfile(slangSession, profile));
+
+    // Define a macro so that shader code in a test can detect what language we
+    // are nominally working with.
+    char const* langDefine = nullptr;
+    switch (sourceLanguage)
+    {
+    case SLANG_SOURCE_LANGUAGE_GLSL:
+        spAddPreprocessorDefine(slangRequest, "__GLSL__", "1");
+        break;
+
+    case SLANG_SOURCE_LANGUAGE_SLANG:
+        spAddPreprocessorDefine(slangRequest, "__SLANG__", "1");
+        // fall through
+    case SLANG_SOURCE_LANGUAGE_HLSL:
+        spAddPreprocessorDefine(slangRequest, "__HLSL__", "1");
+        break;
+
+    default:
+        assert(!"unexpected");
+        break;
+    }
+
+    if (passThrough != SLANG_PASS_THROUGH_NONE)
+    {
+        spSetPassThrough(slangRequest, passThrough);
+    }
+
+    // Preocess any additional command-line options specified for Slang using
+    // the `-xslang <arg>` option to `render-test`.
+    spProcessCommandLineArguments(slangRequest, &gOptions.slangArgs[0], gOptions.slangArgCount);
+
+    int computeTranslationUnit = 0;
+    int vertexTranslationUnit = 0;
+    int fragmentTranslationUnit = 0;
+    char const* vertexEntryPointName = request.vertexShader.name;
+    char const* fragmentEntryPointName = request.fragmentShader.name;
+    char const* computeEntryPointName = request.computeShader.name;
+
+    if (sourceLanguage == SLANG_SOURCE_LANGUAGE_GLSL)
+    {
+        // GLSL presents unique challenges because, frankly, it got the whole
+        // compilation model wrong. One aspect of working around this is that
+        // we will compile the same source file multiple times: once per
+        // entry point, and we will have different preprocessor definitions
+        // active in each case.
+
+        vertexTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
+        spAddTranslationUnitSourceString(slangRequest, vertexTranslationUnit, request.source.path, request.source.dataBegin);
+        spTranslationUnit_addPreprocessorDefine(slangRequest, vertexTranslationUnit, "__GLSL_VERTEX__", "1");
+        vertexEntryPointName = "main";
+
+        fragmentTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
+        spAddTranslationUnitSourceString(slangRequest, fragmentTranslationUnit, request.source.path, request.source.dataBegin);
+        spTranslationUnit_addPreprocessorDefine(slangRequest, fragmentTranslationUnit, "__GLSL_FRAGMENT__", "1");
+        fragmentEntryPointName = "main";
+
+        computeTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
+        spAddTranslationUnitSourceString(slangRequest, computeTranslationUnit, request.source.path, request.source.dataBegin);
+        spTranslationUnit_addPreprocessorDefine(slangRequest, computeTranslationUnit, "__GLSL_COMPUTE__", "1");
+        computeEntryPointName = "main";
+    }
+    else
+    {
+        int translationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
+        spAddTranslationUnitSourceString(slangRequest, translationUnit, request.source.path, request.source.dataBegin);
+
+        vertexTranslationUnit = translationUnit;
+        fragmentTranslationUnit = translationUnit;
+        computeTranslationUnit = translationUnit;
+    }
+
+
+    ShaderProgram * result = nullptr;
+    Slang::List<const char*> rawTypeNames;
+    for (auto typeName : request.entryPointTypeArguments)
+        rawTypeNames.Add(typeName.Buffer());
+    if (request.computeShader.name)
+    {
+        int computeEntryPoint = spAddEntryPointEx(slangRequest, computeTranslationUnit, 
+            computeEntryPointName,
+            SLANG_STAGE_COMPUTE,
+            (int)rawTypeNames.Count(),
+            rawTypeNames.Buffer());
+
+        spSetLineDirectiveMode(slangRequest, SLANG_LINE_DIRECTIVE_MODE_NONE);
+        int compileErr = spCompile(slangRequest);
+        if (auto diagnostics = spGetDiagnosticOutput(slangRequest))
         {
-            spSetPassThrough(slangRequest, SLANG_PASS_THROUGH_GLSLANG);
+            fprintf(stderr, "%s", diagnostics);
         }
+        if (!compileErr)
+        {
+            size_t codeSize = 0;
+            char const* code = (char const*) spGetEntryPointCode(slangRequest, computeEntryPoint, &codeSize);
+
+            ShaderProgram::KernelDesc kernelDesc;
+            kernelDesc.stage = StageType::Compute;
+            kernelDesc.codeBegin = code;
+            kernelDesc.codeEnd = code + codeSize;
 
-        // Preocess any additional command-line options specified for Slang using
-        // the `-xslang <arg>` option to `render-test`.
-		spProcessCommandLineArguments(slangRequest, &gOptions.slangArgs[0], gOptions.slangArgCount);
-
-		int computeTranslationUnit = 0;
-		int vertexTranslationUnit = 0;
-		int fragmentTranslationUnit = 0;
-		char const* vertexEntryPointName = request.vertexShader.name;
-		char const* fragmentEntryPointName = request.fragmentShader.name;
-		char const* computeEntryPointName = request.computeShader.name;
-
-		if (sourceLanguage == SLANG_SOURCE_LANGUAGE_GLSL)
-		{
-			// GLSL presents unique challenges because, frankly, it got the whole
-			// compilation model wrong. One aspect of working around this is that
-			// we will compile the same source file multiple times: once per
-			// entry point, and we will have different preprocessor definitions
-			// active in each case.
-
-			vertexTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
-			spAddTranslationUnitSourceString(slangRequest, vertexTranslationUnit, request.source.path, request.source.dataBegin);
-			spTranslationUnit_addPreprocessorDefine(slangRequest, vertexTranslationUnit, "__GLSL_VERTEX__", "1");
-			vertexEntryPointName = "main";
-
-			fragmentTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
-			spAddTranslationUnitSourceString(slangRequest, fragmentTranslationUnit, request.source.path, request.source.dataBegin);
-			spTranslationUnit_addPreprocessorDefine(slangRequest, fragmentTranslationUnit, "__GLSL_FRAGMENT__", "1");
-			fragmentEntryPointName = "main";
-
-			computeTranslationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
-			spAddTranslationUnitSourceString(slangRequest, computeTranslationUnit, request.source.path, request.source.dataBegin);
-			spTranslationUnit_addPreprocessorDefine(slangRequest, computeTranslationUnit, "__GLSL_COMPUTE__", "1");
-			computeEntryPointName = "main";
-		}
-		else
-		{
-			int translationUnit = spAddTranslationUnit(slangRequest, sourceLanguage, nullptr);
-			spAddTranslationUnitSourceString(slangRequest, translationUnit, request.source.path, request.source.dataBegin);
-
-			vertexTranslationUnit = translationUnit;
-			fragmentTranslationUnit = translationUnit;
-			computeTranslationUnit = translationUnit;
-		}
-
-
-		ShaderProgram * result = nullptr;
-        Slang::List<const char*> rawTypeNames;
-        for (auto typeName : request.entryPointTypeArguments)
-            rawTypeNames.Add(typeName.Buffer());
-		if (request.computeShader.name)
-		{
-		    int computeEntryPoint = spAddEntryPointEx(slangRequest, computeTranslationUnit, 
-                computeEntryPointName, 
-                spFindProfile(slangSession, request.computeShader.profile),
-                (int)rawTypeNames.Count(),
-                rawTypeNames.Buffer());
-
-            spSetLineDirectiveMode(slangRequest, SLANG_LINE_DIRECTIVE_MODE_NONE);
-			int compileErr = spCompile(slangRequest);
-			if (auto diagnostics = spGetDiagnosticOutput(slangRequest))
-			{
-				fprintf(stderr, "%s", diagnostics);
-			}
-			if (!compileErr)
-			{
-				ShaderCompileRequest innerRequest = request;
-
-                size_t codeSize = 0;
-                char const* code = (char const*) spGetEntryPointCode(slangRequest, computeEntryPoint, &codeSize);
-                innerRequest.computeShader.source.dataBegin = code;
-                innerRequest.computeShader.source.dataEnd = code + codeSize;
-				result = innerCompiler->compileProgram(innerRequest);
-			}
-		}
-		else
-		{
-			int vertexEntryPoint = spAddEntryPointEx(slangRequest, vertexTranslationUnit, vertexEntryPointName, spFindProfile(slangSession, request.vertexShader.profile), (int)rawTypeNames.Count(), rawTypeNames.Buffer());
-			int fragmentEntryPoint = spAddEntryPointEx(slangRequest, fragmentTranslationUnit, fragmentEntryPointName, spFindProfile(slangSession, request.fragmentShader.profile), (int)rawTypeNames.Count(), rawTypeNames.Buffer());
-
-			int compileErr = spCompile(slangRequest);
-			if (auto diagnostics = spGetDiagnosticOutput(slangRequest))
-			{
-				// TODO(tfoley): re-enable when I get a logging solution in place
-	//            OutputDebugStringA(diagnostics);
-				fprintf(stderr, "%s", diagnostics);
-			}
-			if (!compileErr)
-			{
-				ShaderCompileRequest innerRequest = request;
-
-                size_t vertexCodeSize = 0;
-				char const* vertexCode = (char const*) spGetEntryPointCode(slangRequest, vertexEntryPoint, &vertexCodeSize);
-
-                size_t fragmentCodeSize = 0;
-                char const* fragmentCode = (char const*) spGetEntryPointCode(slangRequest, fragmentEntryPoint, &fragmentCodeSize);
-
-				innerRequest.vertexShader.source.dataBegin = vertexCode;
-                innerRequest.vertexShader.source.dataEnd   = vertexCode + vertexCodeSize;
-
-				innerRequest.fragmentShader.source.dataBegin = fragmentCode;
-                innerRequest.fragmentShader.source.dataEnd   = fragmentCode + fragmentCodeSize;
-
-				result = innerCompiler->compileProgram(innerRequest);
-			}
-		}
-        // We clean up the Slang compilation context and result *after*
-        // we have run the downstream compiler, because Slang
-        // owns the memory allocation for the generated text, and will
-        // free it when we destroy the compilation result.
-        spDestroyCompileRequest(slangRequest);
-        spDestroySession(slangSession);
-
-        return result;
+            ShaderProgram::Desc desc;
+            desc.pipelineType = PipelineType::Compute;
+            desc.kernels = &kernelDesc;
+            desc.kernelCount = 1;
+
+            result = renderer->createProgram(desc);
+        }
     }
-};
+    else
+    {
+        int vertexEntryPoint = spAddEntryPointEx(slangRequest, vertexTranslationUnit, vertexEntryPointName, SLANG_STAGE_VERTEX, (int)rawTypeNames.Count(), rawTypeNames.Buffer());
+        int fragmentEntryPoint = spAddEntryPointEx(slangRequest, fragmentTranslationUnit, fragmentEntryPointName, SLANG_STAGE_FRAGMENT, (int)rawTypeNames.Count(), rawTypeNames.Buffer());
 
-ShaderCompiler* createSlangShaderCompiler(
-    ShaderCompiler*     innerCompiler,
-    SlangSourceLanguage sourceLanguage,
-    SlangCompileTarget  target)
-{
-    auto result = new SlangShaderCompilerWrapper();
-    result->innerCompiler = innerCompiler;
-    result->sourceLanguage = sourceLanguage;
-    result->target = target;
+        int compileErr = spCompile(slangRequest);
+        if (auto diagnostics = spGetDiagnosticOutput(slangRequest))
+        {
+            // TODO(tfoley): re-enable when I get a logging solution in place
+//            OutputDebugStringA(diagnostics);
+            fprintf(stderr, "%s", diagnostics);
+        }
+        if (!compileErr)
+        {
+            size_t vertexCodeSize = 0;
+            char const* vertexCode = (char const*) spGetEntryPointCode(slangRequest, vertexEntryPoint, &vertexCodeSize);
+
+            size_t fragmentCodeSize = 0;
+            char const* fragmentCode = (char const*) spGetEntryPointCode(slangRequest, fragmentEntryPoint, &fragmentCodeSize);
+
+            static const int kDescCount = 2;
+
+            ShaderProgram::KernelDesc kernelDescs[kDescCount];
+
+            kernelDescs[0].stage = StageType::Vertex;
+            kernelDescs[0].codeBegin = vertexCode;
+            kernelDescs[0].codeEnd = vertexCode + vertexCodeSize;
+
+            kernelDescs[1].stage = StageType::Fragment;
+            kernelDescs[1].codeBegin = fragmentCode;
+            kernelDescs[1].codeEnd = fragmentCode + fragmentCodeSize;
+
+            ShaderProgram::Desc desc;
+            desc.pipelineType = PipelineType::Graphics;
+            desc.kernels = &kernelDescs[0];
+            desc.kernelCount = kDescCount;
+
+            result = renderer->createProgram(desc);
+        }
+    }
+    // We clean up the Slang compilation context and result *after*
+    // we have run the downstream compiler, because Slang
+    // owns the memory allocation for the generated text, and will
+    // free it when we destroy the compilation result.
+    spDestroyCompileRequest(slangRequest);
+    spDestroySession(slangSession);
 
     return result;
 }