Make render-test use Slang for all shader compilation (#597)

* Make render-test use Slang for all shader compilation

This streamlines the code for render-test by having all its shader compilation go through the Slang API, so that it doesn't have to deal with custom logic to compile HLSL->DXBC and HLSL->DXIL. We were already leaning on Slang to generate SPIR-V for Vulkan, so this makes all the paths more consistent.

My original plan with this change was to make the D3D12 render path start using DXIL at this point, since the change would make that easy, but it turns out that some aspects of how we handle parameter binding are not compatible with that right now, so it would need to come as a later change.

There's a lot of details here, so I will try to walk through the changes, including the incidental ones:

* Add logic to `premake5.lua` so that we copy the necessary libraries for HLSL shader compilation to our target directory from the Windows SDK. This is necessary so that our tests can actually invoke `dxcompiler.dll`

* Re-run Premake to generate new project files. This moves around a few files that I manually added in previous changes without re-running Premake.

* When invoking `fxc` as a pass-through compiler, be sure to pass along any macros defines via API or command-line. This isn't a strictly required change with how things worked out, but it is a positive one anyway, because it makes `slangc -pass-through fxc` more useful.

* Don't print output from a downstream `fxc` invocation if it produces warnings but no errors. The main reason for this is so that our tests don't fail because of `fxc` warnings on Slang's output (which then don't match the baselines), but it can also be rationalized as not wanting to confuse users with warnings that don't come from the "real" compiler they are using. This probably needs fine-tuning as a policy.

* Add the HLSL `NonUniformResourceIndex` function. This was an oversight because it isn't documented as a builtin on MSDN, and only gets mentioned obliquely when they talk about resource indexing.

* Add `glsl_<version>` profiles to match our `sm_<version>` profiles, so that it is easy for a user to use the profile mechanism to request a specific GLSL version without also specifying a stage name.

* Update the render-test logic so that there is a single `ShaderCompiler` implementation that *always* uses Slang, and get rid of all of the renderer-specific `ShaderCompiler` implementations.

* Update logic in render-test `main.cpp` to select the options to use for the eventual Slang compile based on the choice of renderer and input language. I didn't change the options that render-test exposes, even though they are getting increasingly silly (e.g., `-glsl-rewrite` doesn't use GLSL as its input...).

* Note: the D3D12 renderer will still use fxc, DXBC, and SM 5.0 for now, since trying to update it to switch to dxc, DXIL, and SM 6.0 didn't work well at the time.

* Add a bit of supporting D3D12 code to make sure that we don't allocate a structured buffer when a buffer has a format.

* Make sure to *also* define the `__HLSL__` macro when compiling Slang code, because otherwise a bunch of tests don't work (I'm not clear on how it worked before...).

* fixup: missing file

5 files changed, 110 insertions, 11 deletions

diff --git a/source/slang/compiler.cpp b/source/slang/compiler.cpp
index b7cbc2327..66df01db0 100644
--- a/source/slang/compiler.cpp
+++ b/source/slang/compiler.cpp
@@ -279,13 +279,43 @@ namespace Slang
 
         auto profile = getEffectiveProfile(entryPoint, targetReq);
 
+        // If we have been invoked in a pass-through mode, then we need to make sure
+        // that the downstream compiler sees whatever options were passed to Slang
+        // via the command line or API.
+        //
+        // TODO: more pieces of information should be added here as needed.
+        //
+        List<D3D_SHADER_MACRO> dxMacrosStorage;
+        D3D_SHADER_MACRO const* dxMacros = nullptr;
+        if( entryPoint->compileRequest->passThrough != PassThroughMode::None )
+        {
+            for( auto& define :  entryPoint->compileRequest->preprocessorDefinitions )
+            {
+                D3D_SHADER_MACRO dxMacro;
+                dxMacro.Name = define.Key.Buffer();
+                dxMacro.Definition = define.Value.Buffer();
+                dxMacrosStorage.Add(dxMacro);
+            }
+            for( auto& define : entryPoint->getTranslationUnit()->preprocessorDefinitions )
+            {
+                D3D_SHADER_MACRO dxMacro;
+                dxMacro.Name = define.Key.Buffer();
+                dxMacro.Definition = define.Value.Buffer();
+                dxMacrosStorage.Add(dxMacro);
+            }
+            D3D_SHADER_MACRO nullTerminator = { 0, 0 };
+            dxMacrosStorage.Add(nullTerminator);
+
+            dxMacros = dxMacrosStorage.Buffer();
+        }
+
         ID3DBlob* codeBlob;
         ID3DBlob* diagnosticsBlob;
         HRESULT hr = D3DCompile_(
             hlslCode.begin(),
             hlslCode.Length(),
             "slang",
-            nullptr,
+            dxMacros,
             nullptr,
             getText(entryPoint->name).begin(),
             GetHLSLProfileName(profile),
@@ -300,7 +330,17 @@ namespace Slang
             data.AddRange((uint8_t const*)codeBlob->GetBufferPointer(), (int)codeBlob->GetBufferSize());
             codeBlob->Release();
         }
-        if (diagnosticsBlob)
+
+        // Note: we will only output diagnostics coming from a downstream
+        // compiler in the event of an error (although in that case we will
+        // end up including any warning diagnostics that are produced as well).
+        //
+        // TODO: some day we should aspire to make Slang's output always compile
+        // cleanly without warnings on downstream compilers (or else suppress those
+        // warnings), but this is difficult to do in practice without a lot of
+        // tailoring for the quirks of each compiler (version).
+        //
+        if (diagnosticsBlob && FAILED(hr))
         {
             // TODO(tfoley): need a better policy for how we translate diagnostics
             // back into the Slang world (although we should always try to generate
diff --git a/source/slang/hlsl.meta.slang b/source/slang/hlsl.meta.slang
index 86f9eb946..10665915d 100644
--- a/source/slang/hlsl.meta.slang
+++ b/source/slang/hlsl.meta.slang
@@ -750,6 +750,27 @@ __generic<T : __BuiltinArithmeticType, let R : int, let N : int, let C : int> __
 float noise(float x);
 __generic<let N : int> float noise(vector<float, N> x);
 
+/// Indicate that an index may be non-uniform at execution time.
+///
+/// Shader Model 5.1 and 6.x introduce support for dynamic indexing
+/// of arrays of resources, but place the restriction that *by default*
+/// the implementation can assume that any value used as an index into
+/// such arrays will be dynamically uniform across an entire `Draw` or `Dispatch`
+/// (when using instancing, the value must be uniform across all instances;
+/// it does not seem that the restriction extends to draws within a multi-draw).
+///
+/// In order to indicate to the implementation that it cannot make the
+/// uniformity assumption, a shader programmer is required to pass the index
+/// to the `NonUniformResourceIndex` function before using it as an index.
+/// The function superficially acts like an identity function.
+///
+/// Note: a future version of Slang may take responsibility for inserting calls
+/// to this function as necessary in output code, rather than make this
+/// the user's responsibility, so that the default behavior of the language
+/// is more semantically "correct."
+uint NonUniformResourceIndex(uint index);
+int NonUniformResourceIndex(int index);
+
 // Normalize a vector
 __generic<T : __BuiltinFloatingPointType, let N : int> vector<T,N> normalize(vector<T,N> x);
 
diff --git a/source/slang/hlsl.meta.slang.h b/source/slang/hlsl.meta.slang.h
index c05e81cd1..d63f38e47 100644
--- a/source/slang/hlsl.meta.slang.h
+++ b/source/slang/hlsl.meta.slang.h
@@ -783,6 +783,27 @@ SLANG_RAW("// noise (deprecated)\n")
 SLANG_RAW("float noise(float x);\n")
 SLANG_RAW("__generic<let N : int> float noise(vector<float, N> x);\n")
 SLANG_RAW("\n")
+SLANG_RAW("/// Indicate that an index may be non-uniform at execution time.\n")
+SLANG_RAW("///\n")
+SLANG_RAW("/// Shader Model 5.1 and 6.x introduce support for dynamic indexing\n")
+SLANG_RAW("/// of arrays of resources, but place the restriction that *by default*\n")
+SLANG_RAW("/// the implementation can assume that any value used as an index into\n")
+SLANG_RAW("/// such arrays will be dynamically uniform across an entire `Draw` or `Dispatch`\n")
+SLANG_RAW("/// (when using instancing, the value must be uniform across all instances;\n")
+SLANG_RAW("/// it does not seem that the restriction extends to draws within a multi-draw).\n")
+SLANG_RAW("///\n")
+SLANG_RAW("/// In order to indicate to the implementation that it cannot make the\n")
+SLANG_RAW("/// uniformity assumption, a shader programmer is required to pass the index\n")
+SLANG_RAW("/// to the `NonUniformResourceIndex` function before using it as an index.\n")
+SLANG_RAW("/// The function superficially acts like an identity function.\n")
+SLANG_RAW("///\n")
+SLANG_RAW("/// Note: a future version of Slang may take responsibility for inserting calls\n")
+SLANG_RAW("/// to this function as necessary in output code, rather than make this\n")
+SLANG_RAW("/// the user's responsibility, so that the default behavior of the language\n")
+SLANG_RAW("/// is more semantically \"correct.\"\n")
+SLANG_RAW("uint NonUniformResourceIndex(uint index);\n")
+SLANG_RAW("int NonUniformResourceIndex(int index);\n")
+SLANG_RAW("\n")
 SLANG_RAW("// Normalize a vector\n")
 SLANG_RAW("__generic<T : __BuiltinFloatingPointType, let N : int> vector<T,N> normalize(vector<T,N> x);\n")
 SLANG_RAW("\n")
diff --git a/source/slang/profile-defs.h b/source/slang/profile-defs.h
index 2f029aff0..93703f3b9 100644
--- a/source/slang/profile-defs.h
+++ b/source/slang/profile-defs.h
@@ -205,6 +205,19 @@ PROFILE_ALIAS(DX_None_6_3,  DX_Lib_6_3,     sm_6_3)
 
 // Define all the GLSL profiles
 
+PROFILE(GLSL_None_110, glsl_110, Unknown, GLSL_110)
+PROFILE(GLSL_None_120, glsl_120, Unknown, GLSL_120)
+PROFILE(GLSL_None_130, glsl_130, Unknown, GLSL_130)
+PROFILE(GLSL_None_140, glsl_140, Unknown, GLSL_140)
+PROFILE(GLSL_None_150, glsl_150, Unknown, GLSL_150)
+PROFILE(GLSL_None_330, glsl_330, Unknown, GLSL_330)
+PROFILE(GLSL_None_400, glsl_400, Unknown, GLSL_400)
+PROFILE(GLSL_None_410, glsl_410, Unknown, GLSL_410)
+PROFILE(GLSL_None_420, glsl_420, Unknown, GLSL_420)
+PROFILE(GLSL_None_430, glsl_430, Unknown, GLSL_430)
+PROFILE(GLSL_None_440, glsl_440, Unknown, GLSL_440)
+PROFILE(GLSL_None_450, glsl_450, Unknown, GLSL_450)
+
 #define P(UPPER, LOWER, VERSION) \
 PROFILE(GLSL_##UPPER##_##VERSION,   glsl_##LOWER##_##VERSION, UPPER, GLSL_##VERSION)
 
diff --git a/source/slang/slang.vcxproj.filters b/source/slang/slang.vcxproj.filters
index cb0235e61..f12163f27 100644
--- a/source/slang/slang.vcxproj.filters
+++ b/source/slang/slang.vcxproj.filters
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
   <ItemGroup>
     <Filter Include="Header Files">
@@ -54,6 +54,12 @@
     <ClInclude Include="ir-insts.h">
       <Filter>Header Files</Filter>
     </ClInclude>
+    <ClInclude Include="ir-restructure-scoping.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="ir-restructure.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
     <ClInclude Include="ir-ssa.h">
       <Filter>Header Files</Filter>
     </ClInclude>
@@ -153,10 +159,6 @@
     <ClInclude Include="vm.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="ir-restructure.h">
-      <Filter>Header Files</Filter>
-    </ClInclude>
-    <ClInclude Include="ir-restructure-scoping.h" />
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="bytecode.cpp">
@@ -186,6 +188,12 @@
     <ClCompile Include="ir-legalize-types.cpp">
       <Filter>Source Files</Filter>
     </ClCompile>
+    <ClCompile Include="ir-restructure-scoping.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="ir-restructure.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
     <ClCompile Include="ir-ssa.cpp">
       <Filter>Source Files</Filter>
     </ClCompile>
@@ -258,10 +266,6 @@
     <ClCompile Include="vm.cpp">
       <Filter>Source Files</Filter>
     </ClCompile>
-    <ClCompile Include="ir-restructure.cpp">
-      <Filter>Source Files</Filter>
-    </ClCompile>
-    <ClCompile Include="ir-restructure-scoping.cpp" />
   </ItemGroup>
   <ItemGroup>
     <None Include="slang.natvis">