Unify handling of static and dynamic dispatch for interfaces (#1612)

Overview
========

Prior to this change, we had two different code generation strategies for interface/existential types in Slang, that didn't always play nicely together:

* The "legacy" static specialization approach could handle plugging in an arbitrary concrete type for an existential type parameter (including types with resources, etc.), but wouldn't work well with things like a `StructuredBuffer<>` of an interface type, and requires somewhat counter-intuitive layout rules to make work.

* The new dynamic dispatch approach produces simpler, more easily understood layouts by assuming that values of interface type can fit into a fixed number of bytes. The tradeoff there is that it cannot handle types that include resources (only POD types).

The goal of this change is to make it so that the two strategies can co-exist. In particular, in cases where a shader is amenable to both static specialization and dynamic dispatch, the type layouts should agree.

In order to make the type layouts agree, we:

* Declare that *all* values of existential type reserve storage according to the dynamic-dispatch rules (so 16 bytes for the RTTI and witness-table information, plus whatever bytes are needed to story "any value" of a conforming type).

* Then we modify the "legacy" layout rules so that if a value of concrete type can fit in the reserved "any value" space for a given interface, then it is laid out there exactly like the dynamic dispatch rules would do. Otherwise, we fall back to the previous legacy rules (since we don't need to agree with the dynamic-dispatch layout on types that can't be used with dynamic dispatch).

Details
=======

* Renamed `ExistentialBox` to `BoundInterfaceType` to better clarify how it relates to `BindExistentialsType`

* Unconditionally apply the `lowerGenerics` pass during emit, since it is now responsible for aspects of the lowering of existential types when specialization is used.

* Made IR type layout take the target into account, so that the layout of resource types can vary by target (e.g., being POD on some targets, and invalid on others)

* Cleaned up some issues around using global shader parameters as the "key" for their layout information in the global-scope layout (only comes up when there are global-scope `uniform` parameters)

* Made there be a default any-value size (16) instead of making it be an error to leave out. This was the simplest option; we could try to go back to having an error, but we'd need to only issue it if we are sure a type/interface is being used with dynamic dispatch, since static dispatch doesn't have to obey the restrictions.

* Changed lowering of existential types to tuples so that bound interfaces where the concrete type won't fit use a "pseudo-pointer" instead of an "any-value" to hold the payload

* Changed IR type legalization to handle the "pseudo-pointer" case and apply layout information from an interface type over to the payload part when static specialization was used.

* Changed some details of how witness tables were being lowered, so that we didn't have to create "proxy" witness tables for the constraints on associated types (just use the actual requirement entries we generate)

* Changed witness tables so that they know the subtype doing the conforming

* Added logic so that we don't generate pack/unpack logic and witness table wrapper functions for types that are incompatible with any-value/dynamic dispatch for a given interface.

* Changed the core AST-level type layout logic to use the dynamic-dispatch layout in case things fit, and the legacy static specialization case when things don't (while also reserving space for the dynamic-dispatch fields)

* Changed a bunch of test cases for static specialization to properly use the new layout (which introduces new buffers in some cases, and moves data around in others).

Future Work
===========

The experience of trying to reconcile our older way of handling interface-type specialization with our newer model (that supports dynamic dispatch) makes it clear that we really need to make similar changes to our handling of generic type parameters on entry points and at the global scope.

A future change should make it so that a global type parameter is lowered with a type layout similar to a value parameter of interface type, including the RTTI and witness-table pieces, and just leaving out the "any value" piece. A similar translation strategy should apply to entry-point generic parameters (mirroring how we lower generic functions for dynamic dispatch already), and value specialization parameters.

Co-authored-by: Yong He <yonghe@outlook.com>

7 files changed, 56 insertions, 21 deletions

diff --git a/tests/compute/dynamic-dispatch-bindless-texture.slang b/tests/compute/dynamic-dispatch-bindless-texture.slang
index 49265fac4..d3d40b2c5 100644
--- a/tests/compute/dynamic-dispatch-bindless-texture.slang
+++ b/tests/compute/dynamic-dispatch-bindless-texture.slang
@@ -2,7 +2,7 @@
 //TEST(compute):COMPARE_COMPUTE:-cpu
 //TEST(compute):COMPARE_COMPUTE:-cuda
 
-[anyValueSize(8)]
+[anyValueSize(16)]
 interface IInterface
 {
     float run();
diff --git a/tests/compute/interface-shader-param-in-struct.slang b/tests/compute/interface-shader-param-in-struct.slang
index 05db7fb39..93d96ed81 100644
--- a/tests/compute/interface-shader-param-in-struct.slang
+++ b/tests/compute/interface-shader-param-in-struct.slang
@@ -3,8 +3,10 @@
 // This test puts interface-type shader parameters
 // inside of structure types to make sure that works
 
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12
+// Note: D3D11 test disabled because of missing `uint64_t`; re-enable when `uint2` used instead.
+//DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
+
+//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12 -profile sm_6_0 -use-dxil
 //TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute
 
 // A lot of the setup is the same as for `interface-shader-param`,
@@ -51,7 +53,7 @@ RWStructuredBuffer<int> gOutputBuffer;
 // that it *will* contain uniform/ordinary data
 // after specialization.
 //
-//TEST_INPUT:cbuffer(data=[0]):
+//TEST_INPUT:cbuffer(data=[0 0 0 0 0 0 0 0]):
 cbuffer C
 {
    IRandomNumberGenerationStrategy gStrategy;
@@ -79,7 +81,7 @@ struct Stuff
 
 [numthreads(4, 1, 1)]
 void computeMain(
-//TEST_INPUT:root_constants(data=[256]):
+//TEST_INPUT:root_constants(data=[0 0 0 0 0 0 0 0 256]):
     uniform Stuff stuff,
 
     uint3       dispatchThreadID : SV_DispatchThreadID)
diff --git a/tests/compute/interface-shader-param-legalization.slang b/tests/compute/interface-shader-param-legalization.slang
index 26603bfaa..0c285a60f 100644
--- a/tests/compute/interface-shader-param-legalization.slang
+++ b/tests/compute/interface-shader-param-legalization.slang
@@ -10,7 +10,13 @@ interface IModifier
     int modify(int val);
 }
 
-IModifier gModifier;
+// Note: a constant buffer for the global scope gets introduced
+// at this point, since `gModifier` introduces uniform storage
+// for the RTTI, witness table, and "any value" parts of the
+// existential.
+//
+//TEST_INPUT:cbuffer(data=[1 2 3 4 5 6 7 8], stride=4):
+uniform IModifier gModifier;
 
 int test(
     int val)
diff --git a/tests/compute/interface-shader-param.slang b/tests/compute/interface-shader-param.slang
index 8979b66e0..9b1a747e7 100644
--- a/tests/compute/interface-shader-param.slang
+++ b/tests/compute/interface-shader-param.slang
@@ -3,8 +3,10 @@
 // Test using interface tops as top-level shader parameters
 // (whether global, or on an entry point).
 
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12
+// Note: D3D11 test disabled because of missing `uint64_t`; re-enable when `uint2` used instead.
+//DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
+
+//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12 -profile sm_6_0 -use-dxil
 //TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute
 //DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-cpu -compute
 
@@ -69,6 +71,20 @@ int test(
     return modifiedVal;
 }
 
+// The global-scope parameters for this example include
+// some `uniform` parameters, and that will trigger
+// the allocation of a global-scope constant buffer to
+// hold them. Slang's layout rules mean that the buffer
+// will be allocated before any other global-scope parameters.
+//
+// In this example, the buffer will not be needed after specialization,
+// but we need to declare/allocate it here so that the application
+// creates a descriptor table/set that matches what the shader
+// signature expects.
+//
+//TEST_INPUT:cbuffer(data=[0], stride=4):name=gStrategy
+
+
 
 // Now we'll define a shader entry point that will use
 // these interfaces to define its behavior.
@@ -81,7 +97,7 @@ RWStructuredBuffer<int> gOutputBuffer;
 // Now we'll define a global shader parameter for the
 // random number generation strategy.
 //
-IRandomNumberGenerationStrategy gStrategy;
+uniform IRandomNumberGenerationStrategy gStrategy;
 
 // The other parameter (for the modifier) will be attached
 // the entry point instead, so that we are testing both
@@ -89,6 +105,7 @@ IRandomNumberGenerationStrategy gStrategy;
 //
 [numthreads(4, 1, 1)]
 void computeMain(
+//TEST_INPUT:root_constants(data=[0], stride=4):
     uniform IModifier   modifier,
             uint3       dispatchThreadID : SV_DispatchThreadID)
 {
diff --git a/tests/compute/interface-shader-param2.slang b/tests/compute/interface-shader-param2.slang
index 6127b3618..b276dae99 100644
--- a/tests/compute/interface-shader-param2.slang
+++ b/tests/compute/interface-shader-param2.slang
@@ -4,8 +4,10 @@
 // concrete types that have data within them, instead of
 // just empty types.
 
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12
+// Note: D3D11 test disabled because of missing `uint64_t`; re-enable when `uint2` used instead.
+//DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
+
+//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12 -profile sm_6_0 -use-dxil
 //TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute
 
 // A lot of the setup is the same as for `interface-shader-param`,
@@ -45,13 +47,13 @@ int test(
 //TEST_INPUT:ubuffer(data=[0 0 0 0], stride=4):out
 RWStructuredBuffer<int> gOutputBuffer;
 
-//TEST_INPUT:cbuffer(data=[1 0 0 0], stride=4):
+//TEST_INPUT:cbuffer(data=[0 0 0 0 1 0 0 0], stride=4):
 ConstantBuffer<IRandomNumberGenerationStrategy> gStrategy;
 
 [numthreads(4, 1, 1)]
 void computeMain(
 
-//TEST_INPUT:root_constants(data=[8 0 0 0], stride=4):
+//TEST_INPUT:root_constants(data=[0 0 0 0 8 0 0 0], stride=4):
     uniform IModifier   modifier,
             uint3       dispatchThreadID : SV_DispatchThreadID)
 {
diff --git a/tests/compute/interface-shader-param3.slang b/tests/compute/interface-shader-param3.slang
index 17236642c..3e4f04ac0 100644
--- a/tests/compute/interface-shader-param3.slang
+++ b/tests/compute/interface-shader-param3.slang
@@ -4,8 +4,10 @@
 // interface types at more complicated places in the overall layout.
 //
 
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12
+// Note: D3D11 test disabled because of missing `uint64_t`; re-enable when `uint2` used instead.
+//DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
+
+//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12 -profile sm_6_0 -use-dxil
 //TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute
 
 // A lot of the setup is the same as for `interface-shader-param`,
@@ -56,7 +58,7 @@ ConstantBuffer<IRandomNumberGenerationStrategy> gStrategy;
 // type argument at the same time.
 //
 //TEST_INPUT: globalExistentialType MyStrategy
-//TEST_INPUT:cbuffer(data=[1 0 0 0], stride=4):
+//TEST_INPUT:cbuffer(data=[0 0 0 0 1 0 0 0], stride=4):
 
 [numthreads(4, 1, 1)]
 void computeMain(
@@ -83,7 +85,7 @@ void computeMain(
 //
 // Here's the incantation to make the test runner fill in the constant buffer:
 //
-//TEST_INPUT:root_constants(data=[256 0 0 0 16 0 0 0], stride=4):
+//TEST_INPUT:root_constants(data=[0 0 0 0 16 0 0 0 256], stride=4):
 //
 // So, the value `256` will be used for `extra` and the value `16`
 // will be written to the first four bytes of the concrete value
diff --git a/tests/compute/interface-shader-param4.slang b/tests/compute/interface-shader-param4.slang
index 6226fd4c6..f1ef76993 100644
--- a/tests/compute/interface-shader-param4.slang
+++ b/tests/compute/interface-shader-param4.slang
@@ -5,8 +5,10 @@
 // shader parameters.
 //
 
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
-//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12
+// Note: D3D11 test disabled because of missing `uint64_t`; re-enable when `uint2` used instead.
+//DISABLE_TEST(compute):COMPARE_COMPUTE_EX:-slang -compute
+
+//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -dx12 -profile sm_6_0 -use-dxil
 //TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute
 
 // A lot of the setup is the same as for `interface-shader-param`,
@@ -72,7 +74,7 @@ ConstantBuffer<IRandomNumberGenerationStrategy> gStrategy;
 void computeMain(
 
 // Similarly to the previous test, we are declaring two `uniform`
-// paameters on the entry point, where one will be plugged in
+// parameters on the entry point, where one will be plugged in
 // with a concrete type, and thus get laid out second.
 //
     uniform IModifier   modifier,
@@ -83,7 +85,11 @@ void computeMain(
 // the previous test, the concrete type plugged in for `modifier`
 // has no uniform/ordinary data, so we don't need to fill it in.
 //
-//TEST_INPUT:root_constants(data=[256]):
+// We *do* need to reserve space in the constant buffer for the
+// existential value (RTTI, witness table, any-value) and that
+// needs to come *before* the value for `extra`.
+//
+//TEST_INPUT:root_constants(data=[0 0 0 0 0 0 0 0 256]):
 
             uint3       dispatchThreadID : SV_DispatchThreadID)
 {