Fix handling of DXR profiles (#704)

The logic in `getEffectiveProfile()` function was mapping these to use `Stage::Unknown` in an early attempt to handle the way that dxc requires the `lib_*` profile for DXR shaders, instead of anything that mentions the stage name (in constrast to, e.g., `vs_5_1`). At the same time, the `GetHLSLProfileName()` function was updated to explicitly handle the DXR shaders and map anything it doesn't expect (including `Stage::Unknown`) to a profile named `unknown`, which dxc obviously doesn't like.

This change tries to fix both issues by:

* Having `getEffectiveProfile()` no longer clobber the stage part of a profile for DXR shaders.

* Having `GetHLSLProfileName()` map all unhandled cases to the `lib_*` profiles, since that seems likely to be how any future stages will need to be handled as well (based on the precedent with DXR)

Along the way, I also fixed a bug where invoking command-line `slangc` with no `-stage` options and then relying on `[shader(...)]` attributes to pick up the entry points would lead to a crash since the array of per-entry-point output paths on each target would not be sized appropriately.

1 files changed, 30 insertions, 11 deletions

diff --git a/source/slang/compiler.cpp b/source/slang/compiler.cpp
index ea1b650a8..ed222d181 100644
--- a/source/slang/compiler.cpp
+++ b/source/slang/compiler.cpp
@@ -301,9 +301,26 @@ namespace Slang
         char const* stagePrefix = nullptr;
         switch( profile.GetStage() )
         {
+            // Note: All of the raytracing-related stages require
+            // compiling for a `lib_*` profile, even when only a
+            // single entry point is present.
+            //
+            // We also go ahead and use this target in any case
+            // where we don't know the actual stage to compiel for,
+            // as a fallback option.
+            //
+            // TODO: We also want to use this option when compiling
+            // multiple entry points to a DXIL library.
+            //
         default:
-            return "unknown";
+            stagePrefix = "lib";
+            break;
 
+            // The traditional rasterization pipeline and compute
+            // shaders all have custom profile names that identify
+            // both the stage and shader model, which need to be
+            // used when compiling a single entry point.
+            //
     #define CASE(NAME, PREFIX) case Stage::NAME: stagePrefix = #PREFIX; break
         CASE(Vertex,    vs);
         CASE(Hull,      hs);
@@ -311,15 +328,6 @@ namespace Slang
         CASE(Geometry,  gs);
         CASE(Fragment,  ps);
         CASE(Compute,   cs);
-
-        // Note: dxc requires a `lib` target for all
-        // ray tracing stages.
-        CASE(RayGeneration, lib);
-        CASE(Intersection, lib);
-        CASE(AnyHit, lib);
-        CASE(ClosestHit, lib);
-        CASE(Miss, lib);
-        CASE(Callable, lib);
     #undef CASE
         }
 
@@ -976,7 +984,18 @@ String dissassembleDXILUsingDXC(
         TargetRequest*      targetReq,
         UInt                entryPointIndex)
     {
-        auto outputPath = targetReq->entryPointOutputPaths[entryPointIndex];
+        // It is possible that we are dynamically discovering entry
+        // points (using `[shader(...)]` attributes), so that the
+        // number of entry points on the compile request does not
+        // match the number of entries in teh `entryPointOutputPaths`
+        // array.
+        //
+        String outputPath;
+        if( entryPointIndex < targetReq->entryPointOutputPaths.Count() )
+        {
+            outputPath = targetReq->entryPointOutputPaths[entryPointIndex];
+        }
+
         auto& result = targetReq->entryPointResults[entryPointIndex];
 
         // Skip the case with no output