Add Vulkan/SPIR-V support for TraceRayInline() (#1737)

For the most part, this translation is straightforward because the `GL_EXT_ray_query` extension is well aligned with the DXR 1.1 `RayQuery` feature. Many function map one-to-one from one extension to the other.

A few notable details:

* The equivalent of the `RayQuery<Flags>` type is non-generic in GLSL, and the GLSL path previously didn't have support for trying to look up an intrinsic type name on an IR type declaration, so that required some tweaks to the emit logic.

* All the GLSL functions are free functions instead of member functions, but our IR doesn't recognize that distinction anyway

* The main `TraceRayInline()` call is the one that took the most tweaking, just because it takes a `RayDesc` structure for D3D/HLSL but takes individual vector sand scalars for VK/GLSL. The approach here is a standard one for how we manage this stuff in the stdlib (and I wanted to avoid adding even more `$` magic for intrinsics).

* For several other calls, the HLSL API had distinct `Candidate***()` and `Committed***()` calls that return information about a candidate hit vs. the one committed into the query. In contrast, the GLSL API uses a single call that takes an additional "must be compile-time constant" `bool` parameter to select between the two behaviors. This is even the case for one call that basically returns a value of a different `enum` type depending on the state of that `bool`. The D3D API model here seems almost strictly better and I have no idea why the GLSL extension was defined this way.

* Because both the `GL_EXT_ray_query` and `GL_EXT_ray_tracing` extensions declare the `accelerationStructureEXT` type, we can no longer infer what extension is supposed to be used based only on the presene of such a type. The logic right now is a bit slippery, because in theory a program that declares an acceleration structure but never traces into it could end up getting a compilation error now. We will have to see if that corner case comes up in practice. :(

The one big detail that is looming after doing this work is that both the HLSL and GLSL exposures of ray queries are extremely "slippery" about the actual identity of queries (e.g., when is one query a copy of another, vs. just being a new variable that references the existing query). Somehow queries get their identity from the original declaration, and as such our "default constructor" approach to them seems semanticay correct, but the whole thing is kind of slippery at a foundational level and I don't know how to fix it with the API as defined. Oh well; just something to keep an eye on.

Co-authored-by: Yong He <yonghe@outlook.com>

3 files changed, 244 insertions, 23 deletions

diff --git a/source/slang/hlsl.meta.slang b/source/slang/hlsl.meta.slang
index 77b371662..761016866 100644
--- a/source/slang/hlsl.meta.slang
+++ b/source/slang/hlsl.meta.slang
@@ -4678,6 +4678,9 @@ static const CANDIDATE_TYPE CANDIDATE_PROCEDURAL_PRIMITIVE = 1;
 // back to the user code at interesting events along the ray.
 //
 __target_intrinsic(hlsl, RayQuery)
+__target_intrinsic(glsl, rayQueryEXT)
+__glsl_extension(GL_EXT_ray_query)
+__glsl_version(460)
 struct RayQuery <let rayFlags : RAY_FLAG = RAY_FLAG_NONE>
 {
     // Initialize the query object in a "fresh" state.
@@ -4697,7 +4700,42 @@ struct RayQuery <let rayFlags : RAY_FLAG = RAY_FLAG_NONE>
     // `RayQuery` to get the effective ray flags, which
     // must obey any API-imposed restrictions.
     //
-    void TraceRayInline(RaytracingAccelerationStructure accelerationStructure, RAY_FLAG rayFlags, uint instanceInclusionMask, RayDesc ray);
+    __target_intrinsic(hlsl)
+    void TraceRayInline(
+        RaytracingAccelerationStructure accelerationStructure,
+        RAY_FLAG                        rayFlags,
+        uint                            instanceInclusionMask,
+        RayDesc                         ray);
+
+    __target_intrinsic(glsl, "rayQueryInitializeEXT($0, $1, $2, $3, $4, $5, $6, $7)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
+    void __rayQueryInitializeEXT(
+        RaytracingAccelerationStructure accelerationStructure,
+        RAY_FLAG                        rayFlags,
+        uint                            instanceInclusionMask,
+        float3                          origin,
+        float                           tMin,
+        float3                          direction,
+        float                           tMax);
+
+    [__unsafeForceInlineEarly]
+    __specialized_for_target(glsl)
+    void TraceRayInline(
+        RaytracingAccelerationStructure accelerationStructure,
+        RAY_FLAG                        rayFlags,
+        uint                            instanceInclusionMask,
+        RayDesc                         ray)
+    {
+        __rayQueryInitializeEXT(
+            accelerationStructure,
+            rayFlags,
+            instanceInclusionMask,
+            ray.Origin,
+            ray.TMin,
+            ray.Direction,
+            ray.TMax);
+    }
 
     // Resume the ray query coroutine.
     //
@@ -4713,6 +4751,9 @@ struct RayQuery <let rayFlags : RAY_FLAG = RAY_FLAG_NONE>
     // functions to appropriately handle the closest hit (it any)
     // that was found.
     //
+    __target_intrinsic(glsl, rayQueryProceedEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     bool Proceed();
 
     // Causes the ray query to terminate.
@@ -4721,6 +4762,9 @@ struct RayQuery <let rayFlags : RAY_FLAG = RAY_FLAG_NONE>
     // traversal has terminated, so that subsequent
     // `Proceed()` calls will return `false`.
     //
+    __target_intrinsic(glsl, rayQueryTerminateEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     void Abort();
 
     // Get the type of candidate hit being considered.
@@ -4732,73 +4776,210 @@ struct RayQuery <let rayFlags : RAY_FLAG = RAY_FLAG_NONE>
     // the kind of candidate hit that must be resolved by
     // user code.
     //
+    __target_intrinsic(glsl, "rayQueryGetIntersectionTypeEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     CANDIDATE_TYPE CandidateType();
 
     // Access properties of a candidate hit.
-    //
+
+    __target_intrinsic(glsl, "transpose(rayQueryGetIntersectionObjectToWorldEXT($0, false))")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3x4 CandidateObjectToWorld3x4();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectToWorldEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float4x3 CandidateObjectToWorld4x3();
+
+    __target_intrinsic(glsl, "transpose(rayQueryGetIntersectionWorldToObjectEXT($0, false))")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3x4 CandidateWorldToObject3x4();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionWorldToObjectEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float4x3 CandidateWorldToObject4x3();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceCustomIndexEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CandidateInstanceIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceIdEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CandidateInstanceID();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionGeometryIndexEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CandidateGeometryIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionPrimitiveIndexEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CandidatePrimitiveIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CandidateInstanceContributionToHitGroupIndex();
 
     // Access properties of the ray being traced
     // in the object space of a candidate hit.
-    //
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectRayOriginEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 CandidateObjectRayOrigin();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectRayDirectionEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 CandidateObjectRayDirection();
 
     // Access properties of a candidate procedural primitive hit.
-    //
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionCandidateAABBOpaqueEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     bool CandidateProceduralPrimitiveNonOpaque();
 
-    // Access properties of a candidate no-opaque triangle hit.
-    //
+    // Access properties of a candidate non-opaque triangle hit.
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionFrontFaceEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     bool CandidateTriangleFrontFace();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionBarycentricsEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float2 CandidateTriangleBarycentrics();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionTEXT($0, false)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float CandidateTriangleRayT();
 
     // Commit the current non-opaque triangle hit.
+    __target_intrinsic(glsl, rayQueryConfirmIntersectionEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     void CommitNonOpaqueTriangleHit();
 
     // Commit the current procedural primitive hit, with hit time `t`.
+    __target_intrinsic(glsl, rayQueryGenerateIntersectionEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     void CommitProceduralPrimitiveHit(float t);
 
     // Get the status of the committed (closest) hit, if any.
+    __target_intrinsic(glsl, "rayQueryGetIntersectionTypeEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     COMMITTED_STATUS CommittedStatus();
 
     // Access properties of the committed hit.
     //
+    __target_intrinsic(glsl, "transpose(rayQueryGetIntersectionObjectToWorldEXT($0, true))")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3x4 CommittedObjectToWorld3x4();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectToWorldEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float4x3 CommittedObjectToWorld4x3();
+
+    __target_intrinsic(glsl, "transpose(rayQueryGetIntersectionWorldToObjectEXT($0, true))")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3x4 CommittedWorldToObject3x4();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionWorldToObjectEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float4x3 CommittedWorldToObject4x3();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionTEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float CommittedRayT();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceCustomIndexEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CommittedInstanceIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceIdEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CommittedInstanceID();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionGeometryIndexEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CommittedGeometryIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionPrimitiveIndexEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CommittedPrimitiveIndex();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionInstanceShaderBindingTableRecordOffsetEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint CommittedInstanceContributionToHitGroupIndex();
 
     // Access properties of the ray being traced
     // in the object space of a committed hit.
-    //
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectRayOriginEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 CommittedObjectRayOrigin();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionObjectRayDirectionEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 CommittedObjectRayDirection();
 
     // Access properties of a committed triangle hit.
-    //
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionFrontFaceEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     bool CommittedTriangleFrontFace();
+
+    __target_intrinsic(glsl, "rayQueryGetIntersectionBarycentricsEXT($0, true)")
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float2 CommittedTriangleBarycentrics();
 
     // Access properties of the ray being traced.
+
+    __target_intrinsic(glsl, rayQueryGetRayFlagsEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     uint RayFlags();
+
+    __target_intrinsic(glsl, rayQueryGetWorldRayOriginEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 WorldRayOrigin();
+
+    __target_intrinsic(glsl, rayQueryGetWorldRayDirectionEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float3 WorldRayDirection();
+
+    __target_intrinsic(glsl, rayQueryGetRayTMinEXT)
+    __glsl_extension(GL_EXT_ray_query)
+    __glsl_version(460)
     float RayTMin();
 }
 
diff --git a/source/slang/slang-emit-glsl.cpp b/source/slang/slang-emit-glsl.cpp
index 7b784de8d..fb4417119 100644
--- a/source/slang/slang-emit-glsl.cpp
+++ b/source/slang/slang-emit-glsl.cpp
@@ -95,6 +95,7 @@ void GLSLSourceEmitter::_requireGLSLVersion(int version)
         CASE(430);
         CASE(440);
         CASE(450);
+        CASE(460);
 
 #undef CASE
     }
@@ -1801,14 +1802,46 @@ void GLSLSourceEmitter::emitSimpleTypeImpl(IRType* type)
         {
             case kIROp_RaytracingAccelerationStructureType:
             {
-                _requireRayTracing();
-
+                // Note: We have the problem here that we want to do `_requireRayTracing()`,
+                // but just based on the use of a ray-tracing acceleration structure we
+                // cannot know which extension the user means to use. The current options are:
+                //
+                //  * GL_NV_ray_tracing
+                //  * GL_EXT_ray_tracing
+                //  * GL_EXT_ray_query
+                //
+                // The first two options there are basically equivalent extensions with
+                // different GLSL syntax. We end up requiring the user to opt in to
+                // `GL_NV_ray_tracing` using target capabilities, and will always default
+                // to `GL_EXT_ray_tracing` otherwise.
+                //
                 if( getTargetCaps().implies(CapabilityAtom::GL_NV_ray_tracing) )
                 {
+                    // If the user has explicitly opted in to `GL_NV_ray_tracing`,
+                    // then we don't need to explicitly request the extentsion again.
+                    // We know that the acceleration structure type will translate
+                    // to the one from that extension:
+                    //
                     m_writer->emit("accelerationStructureNV");
                 }
                 else
                 {
+                    // If the user does *not* opt into a specific extension, then we
+                    // have the problem that either `GL_EXT_ray_tracing` or `GL_EXT_ray-query`
+                    // could provide the `accelerationSturctureEXT` type, but there
+                    // can be drivers that provide only one and not the other.
+                    //
+                    // Because we can't pick one upon just seeing the type, we need to
+                    // emit the type here but *not* call `_requireRayTracing()` or
+                    // anything like it, because we don't yet know the specific extension
+                    // we should ask for.
+                    //
+                    // TODO: We might eventually want to have this step set a flag that
+                    // will cause a compilation error if nothing else in the code requires
+                    // a specific concrete ray-tracing extension. Ideally all of these
+                    // details could be subusmed under the capability system sooner or
+                    // later.
+                    // 
                     m_writer->emit("accelerationStructureEXT");
                 }
                 break;
@@ -1827,6 +1860,13 @@ void GLSLSourceEmitter::emitSimpleTypeImpl(IRType* type)
         return;
     }
 
+    auto decorated = getResolvedInstForDecorations(type);
+    if(auto targetIntrinsicDecor = findBestTargetIntrinsicDecoration(decorated))
+    {
+        m_writer->emit(targetIntrinsicDecor->getDefinition());
+        return;
+    }
+
     SLANG_DIAGNOSE_UNEXPECTED(getSink(), SourceLoc(), "unhandled type");
 }
 
diff --git a/source/slang/slang.natvis b/source/slang/slang.natvis
index 63ee313da..cb272f974 100644
--- a/source/slang/slang.natvis
+++ b/source/slang/slang.natvis
@@ -72,9 +72,9 @@
     </Expand>
   </Type>
   <Type Name="Slang::IRInst">
-    <DisplayString>{{{op}}}</DisplayString>
+    <DisplayString>{{{m_op}}}</DisplayString>
     <Expand>
-      <Item Name="[op]">op</Item>
+      <Item Name="[op]">m_op</Item>
       <Item Name="[type]">typeUse.usedValue</Item>
       <CustomListItems MaxItemsPerView="3">
 		  <Variable Name="child" InitialValue="m_decorationsAndChildren.first"/>
@@ -82,20 +82,20 @@
 			  <If Condition="child == 0">
 				  <Break/>
 			  </If>
-			  <If Condition="child->op == Slang::kIROp_NameHintDecoration">
+			  <If Condition="child->m_op == Slang::kIROp_NameHintDecoration">
 				  <Item Name="[name]">((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.chars,[((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.numChars]s8</Item>
 			  </If>
-			  <If Condition="child->op == Slang::kIROp_ExportDecoration">
+			  <If Condition="child->m_op == Slang::kIROp_ExportDecoration">
 				  <Item Name="[exportName]">((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.chars,[((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.numChars]s8</Item>
 			  </If>
-			  <If Condition="child->op == Slang::kIROp_ImportDecoration">
+			  <If Condition="child->m_op == Slang::kIROp_ImportDecoration">
 				  <Item Name="[importName]">((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.chars,[((Slang::IRStringLit*)(((Slang::IRUse*)(child + 1))->usedValue))->value.stringVal.numChars]s8</Item>
 			  </If>
 			  <Exec>child = child->next</Exec>
 		  </Loop>
       </CustomListItems>
-      <Item Name="[value]" Condition="op == Slang::kIROp_StringLit">((IRStringLit*)this)->value.stringVal.chars,[((IRStringLit*)this)->value.stringVal.numChars]s8</Item>
-      <Item Name="[value]" Condition="op == Slang::kIROp_IntLit">((IRIntLit*)this)->value.intVal</Item>
+      <Item Name="[value]" Condition="m_op == Slang::kIROp_StringLit">((IRStringLit*)this)->value.stringVal.chars,[((IRStringLit*)this)->value.stringVal.numChars]s8</Item>
+      <Item Name="[value]" Condition="m_op == Slang::kIROp_IntLit">((IRIntLit*)this)->value.intVal</Item>
 	  <!--
       <Synthetic Name="[operands]">
         <DisplayString>{{count = {operandCount}}}</DisplayString>
@@ -120,14 +120,14 @@
 			<Exec>child = pOperandInst->m_decorationsAndChildren.first</Exec>
 		    <Exec>nameDecoration = 0</Exec>
 		    <Loop Condition="child != 0">
-			    <If Condition="child->op == Slang::kIROp_NameHintDecoration">
+			    <If Condition="child->m_op == Slang::kIROp_NameHintDecoration">
 				    <Exec>nameDecoration = child</Exec>
 				    <Break/>
 			    </If>
-			    <If Condition="child->op == Slang::kIROp_ExportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->op != Slang::kIROp_NameHintDecoration)">
+			    <If Condition="child->m_op == Slang::kIROp_ExportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->m_op != Slang::kIROp_NameHintDecoration)">
 				    <Exec>nameDecoration = child</Exec>
 			    </If>
-			    <If Condition="child->op == Slang::kIROp_ImportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->op != Slang::kIROp_NameHintDecoration)">
+			    <If Condition="child->m_op == Slang::kIROp_ImportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->m_op != Slang::kIROp_NameHintDecoration)">
 				    <Exec>nameDecoration = child</Exec>
 			    </If>
 			    <Exec>child = child->next</Exec>
@@ -149,14 +149,14 @@
 				  <Exec>child = pItem->m_decorationsAndChildren.first </Exec>
 			      <Exec>nameDecoration = 0</Exec>
 				  <Loop Condition="child != 0">
-				      <If Condition="child->op == Slang::kIROp_NameHintDecoration">
+				      <If Condition="child->m_op == Slang::kIROp_NameHintDecoration">
 						  <Exec>nameDecoration = child</Exec>
 						  <Break/>
 				      </If>
-				      <If Condition="child->op == Slang::kIROp_ExportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->op != Slang::kIROp_NameHintDecoration)">
+				      <If Condition="child->m_op == Slang::kIROp_ExportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->m_op != Slang::kIROp_NameHintDecoration)">
 						  <Exec>nameDecoration = child</Exec>
 				      </If>
-					  <If Condition="child->op == Slang::kIROp_ImportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->op != Slang::kIROp_NameHintDecoration)">
+					  <If Condition="child->m_op == Slang::kIROp_ImportDecoration &amp;&amp; (nameDecoration == 0 || nameDecoration->m_op != Slang::kIROp_NameHintDecoration)">
 						  <Exec>nameDecoration = child</Exec>
 					  </If>
 				      <Exec>child = child->next</Exec>