Change how buffers are emitted (#741)

* Change how buffers are emitted

This is a change with a lot of pieces, which can't always be separated out cleanly. I'm going to walk through them in what I hope is a logical order.

The main goal of this change was to allow arrays of structured buffers to translate to Vulkan. Consider two declarations of structured buffers in HLSL/Slang:

```hlsl
StructuredBuffer<X> single;
StructuredBuffer<Y> multiple[10];
```

The current translation logic was handling `single` by translating it into an *unnamed* GLSL `buffer` block like:

```glsl
layout(std430)
buffer _S1
{
	X single[];
};
```

That syntax allows an expression like `single[i]` in Slang to be translated simply as `single[i]` in GLSL.

But that naive translating doesn't work for `multiple`, since we need to declare a array of blocks in GLSL, which requires giving the whole thing a name:

```glsl
layout(std430)
buffer _S2
{
	Y _data[];
} multiple[10];
```

Now a reference to `multiple[i][j]` in Slang needs to become `multiple[i]._data[j]` in GLSL.

To avoid having way too many special cases around single structured buffers vs. arrays, it makes sense to allows emit things in the latter form, so that we instead lower `single` as:

```glsl
layout(std430)
buffer _S1
{
	X _data[];
} single;
```

So that now a reference to `single[i]` becomes `single._data[i]` in GLSL.
Most of that can be handled in the standard library translation of the structured buffer indexing operations.

The only wrinkle there is that there were some *old* special-case instructions in the IR intended to handle buffer load/store operations (these were added back when I was trying to keep the "VM" path working). These aren't really needed to have structured-buffer operations work; they can be handled as ordinary functions as far as the stdlib is concerned. I removed the old instructions.

Along the way, it became clear that a few other cases follow the same pattern. Byte-addressed buffers are an obvious case. We were lowering HLSL/Slang:

```hlsl
ByteAddressBuffer b;
...
uint x = b.Load(0);
```

to GLSL like:

```glsl
layout(std430)
buffer _S1
{
	uint b[];
};
...
uint x = b[0];
```

That logic would fail for arrays the same way that the structured buffer case was failing. The fix is the same: use named `buffer` blocks and then introduce an explicit `_data` field:

```glsl
layout(std430)
buffer _S1
{
	uint _data[];
} b;
...
uint x = b._data[0];
```

Just like with structured buffers, all of the VK translation for operations on byte-addressed buffers can be implemented directly in teh stdlib, so once the emit logic was changed it was just a matter of adding `._data` to a bunch of VK tranlsations.

It turns out that arrays of constant buffers have more or less the same problem, and furthermore we have some problems with any code that directly uses the modern HLSL `ConstantBuffer<T>` type.

Note: the emit logic around constant buffers sometimes refers to "parameter groups" because that is being used in the compiler as a catch-all term for constant buffers, texture buffers, and parameter blocks.

The existing code was going out of its way to reproduce the way that constant buffer declarations are implicitly referenced in HLSL:

```hlsl
cbuffer C { float f; }
...
float tmp = f; // No reference to `C` here
```

This can be seen in the emit logic with the `isDerefBaseImplicit` function, which is used to take the internal IR representation for a reference to `f` (which is closer to the expression `(*C).f` or `C->f`) and leave off any reference to `C` so that we emit just `f`.

That kind of logic just flat out doesn't work in some important cases. Arrays of constant buffers are a clear one:

```hlsl
ConstantBuffer<X> cbArray[3];
...
X x = cbArray[0];
```

There is no way to translate that to an ordinary `cbuffer` declaration at all. The same problem can be created without arrays, though:

```hlsl
ConstantBuffer<X> singleCB;
...
X x = singleCB;
```

The current strategy for translating constant buffers was translating `singleCB` into a `cbuffer` declaration that reproduced the fields of `X` as its members, which just wouldn't work:

```hlsl
cbuffer singleCB
{
	float f; // field of `X`
}
...
X x = singleCB; // ERROR: there is nothing named `singleCB` in this HLSL
```

The new strategy is more consistent. We still generate a `cbuffer` declaration for a single constant buffer, but we always give it a single field of the chosen element type:

```hlsl
cbuffer singleCB
{
	X singleCB;
}
...
X x = singleCB; // this works fine!
```

And in the array case we generate code that uses the explicit `ConstantBuffer<T>` type:

```hlsl
ConstantBuffer<X> cbArray[3];
...
X x = cbArray[0];
```

The GLSL output is more complicated because unlike with HLSL there is no implicit conversion from a uniform block to its element type (there is no notion of an element type). The array case thus needs a `_data` field similar to what we do for structured buffers:

```glsl
layout(std140)
uniform _S3
{
	X _data;
} cbArray[3];
...
X x = cbArray[0]._data;
```

And then the non-array case needs to have a similar `_data` field for consistency:

```glsl
layout(std140)
uniform _S1
{
	X _data;
} singleCB;
...
X x = singleCB._data;
```

This is handled by inserting the necessary reference to `_data` whenever we dereference a constant buffer, either as part of a load instruction (loading from the whole CB as a pointer), or an `IRFieldAddress` instruction which forms a pointer into the CB (e.g., `&(singleCB->f)` becomes `singleCB._data.f`).

The current emit logic handles `ParameterBlock<X>` differently from `ConstantBuffer<X>`, but really only to allow parameter blocks to be explicitly named in the output, while constant buffers were left implicit by default. Thus the only difference was a legacy one (from back when trying to exactly reproduce the HLSL text we got as input was considered an important goal), and the new approach to emitting constant buffers would get rid of it.

I removed the separate logic for emitting `ParameterBlock<X>` and just let the handling for constant buffers deal with it.

Note that any resource types inside of a `ParameterBlock<X>` would have been moved out as part of legalization, so that a parameter block is 100% equivalent to a constant buffer when it comes time to emit code.

Unsurprisingly, changing the way we generate HLSL and GLSL output for all these buffer types meant that any tests that were directly comparing the output of `slangc` against `fxc`, `dxc`, or `glslang` broke.

The basic approach to fixing the breakage in GLSL tests was to update the GLSL baseline to reflect the new output startegy. In some cases I used macros to name the various `_S<digits>` temporaries so that future renaming will hopefully be easier (it would be great if we auto-generated temporary names with a bit more context). There was one GLSL test (`tests/bugs/vk-structured-buffer-binding`) that was using raw GLSL expected output, and this was changed to use a GLSL baseline to generate SPIR-V for comparison.

For HLSL tests we were sometimes running the same input file through `slangc` and `fxc`/`dxc`, and in these cases I macro-ized the various `cbuffer` declarations to generate different declarations depending on the compiler.

I completely dropped the tests coming from the D3D SDK because they aren't providing much coverage, and updating them would change them so far from the original code that the purported benefit (using a body of existing shaders) would be lost.

I also dropped the explicit matrix layout qualifiers in the `matrix-layout` test because the new output strategy breaks those for GLSL (you can't put matrix layout qualifiers on `struct` fields, and now the body of every constant buffer is inside a `struct`). This isn't as big of a loss as it seems, because our handling of those qualifiers wasn't really right to begin with. Slang users should only be setting the matrix layout mode globally (and we should probably switch to error out on the explicit qualifiers for now).

The other thing that got dropped is tests involving `packoffset` modifiers.
Slang already warns that it doesn't support these, and the way they were used in the test cases is actually misleading. For the binding/layout-related tests, the goal was to show that Slang reproduces the same layout as fxc, in which case explicitly enforcing a layout via `packoffset` seems like cheating (are we sure we enforced the layout fxc would have produced?). The real reason was that Slang used to emit explicit `packoffset` on *every* field of a `cbuffer` it would output, because of an `fxc` bug where you couldn't use `register` on textures/samplers declared inside a `cbuffer` unless *every* field in the `cbuffer` used a `register` or `packoffset` modifier. Slang hasn't required that behavior in a while because it now splits textures and samplers, and the one test case where we needed `packoffset` to work around the `fxc` bug in the baseline HLSL has been macro-ified even more to work around the bug.

The amount of churn in the test cases is unfortunate, but it continues to point at the weakness of any testing strategy that checks for exact equivalent between Slang's output and that of other compilers. We need to keep working to replace these tests with better alternatives.

In `check.cpp` there is logic to perform implicit dereferencing, so that if you write `obj.f` where `obj` is a `ConstantBuffer<X>` (or some other "pointer-like" type) and `f` is a field in `X`, then this effectively translates as `(*obj).f`. That is, we dereference the value of type `ConstantBuffer<X>` to get a value of type `X`, and then refer to the field of the `X` value.

There was a problem where the logic to insert that kind of implicit dereference operation was using a reference (`auto& type = ...`) for the type of the expression being dereferenced, and then clobbering it. This would mean that an expression of type `ConstantBuffer<X>` would have its type overwritten to be just `X` and then codegen would break later on.

I'm not sure how we haven't run into that before.

The `array-of-buffers` test case was added to confirm that we now support arrays of constant, structured, and byte-address buffers for both DXIL and SPIR-V output.

Okay, so that was a lot of stuff, but hopefully it is clear how this all works to make the output of the compiler more consistent and explicit, while also supporting the required new functionality.

* fixup: review feedback

1 files changed, 0 insertions, 468 deletions

diff --git a/tests/hlsl/dxsdk/FixedFuncEMUFX11/FixedFuncEMU.fx b/tests/hlsl/dxsdk/FixedFuncEMUFX11/FixedFuncEMU.fx
deleted file mode 100644
index 699df8655..000000000
--- a/tests/hlsl/dxsdk/FixedFuncEMUFX11/FixedFuncEMU.fx
+++ /dev/null
@@ -1,468 +0,0 @@
-//TEST_IGNORE_FILE:
-// FixedFuncEMU.fx
-// Copyright (c) 2005 Microsoft Corporation. All rights reserved.
-//
-
-struct VSSceneIn
-{
-    float3 pos          : POSITION;         //position of the particle
-    float3 norm         : NORMAL;           //velocity of the particle
-    float2 tex          : TEXTURE0;         //tex coords
-};
-
-struct VSSceneOut
-{
-    float4 pos : SV_Position;               //position
-    float2 tex : TEXTURE0;                  //texture coordinate
-    float3 wPos : TEXTURE1;                 //world space pos
-    float3 wNorm : TEXTURE2;                //world space normal
-    float4 colorD : COLOR0;                 //color for gouraud and flat shading
-    float4 colorS : COLOR1;                 //color for specular
-    float  fogDist : FOGDISTANCE;           //distance used for fog calculations
-    float3 planeDist : SV_ClipDistance0;    //clip distance for 3 planes
-};
-
-struct PSSceneIn
-{
-    float4 pos : SV_Position;               //position
-    float2 tex : TEXTURE0;                  //texture coordinate
-    float3 wPos : TEXTURE1;                 //world space pos
-    float3 wNorm : TEXTURE2;                //world space normal
-    float4 colorD : COLOR0;                 //color for gouraud and flat shading
-    float4 colorS : COLOR1;                 //color for specular
-    float  fogDist : FOGDISTANCE;           //distance used for fog calculations
-};
-
-struct Light
-{
-    float4 Position;
-    float4 Diffuse;
-    float4 Specular;
-    float4 Ambient;
-    float4 Atten;
-};
-
-#define FOGMODE_NONE    0
-#define FOGMODE_LINEAR  1
-#define FOGMODE_EXP     2
-#define FOGMODE_EXP2    3
-#define E 2.71828
-
-cbuffer cbLights
-{
-    float4   g_clipplanes[3];
-    Light    g_lights[8];
-};
-
-cbuffer cbPerFrame
-{
-    float4x4 g_mWorld;
-    float4x4 g_mView;
-    float4x4 g_mProj;
-    float4x4 g_mInvProj;
-    float4x4 g_mLightViewProj;
-};
-
-cbuffer cbPerTechnique
-{
-    bool     g_bEnableLighting = true;
-    bool     g_bEnableClipping = true;
-    bool     g_bPointScaleEnable = false;
-    float    g_pointScaleA;
-    float    g_pointScaleB;
-    float    g_pointScaleC;
-    float    g_pointSize;
-    
-    //fog params
-    int      g_fogMode = FOGMODE_NONE;
-    float    g_fogStart;
-    float    g_fogEnd;
-    float    g_fogDensity;
-    float4   g_fogColor;
-};
-    
-cbuffer cbPerViewChange
-{
-    //viewport params
-    float    g_viewportHeight;
-    float    g_viewportWidth;
-    float    g_nearPlane;
-};
-
-cbuffer cbImmutable
-{
-    float3 g_positions[4] =
-    {
-        float3( -0.5, 0.5, 0 ),
-        float3( 0.5, 0.5, 0 ),
-        float3( -0.5, -0.5, 0 ),
-        float3( 0.5, -0.5, 0 ),
-    };
-};
-
-Texture2D g_txDiffuse;
-Texture2D g_txProjected;
-SamplerState g_samLinear
-{
-    Filter = MIN_MAG_MIP_LINEAR;
-    AddressU = Clamp;
-    AddressV = Clamp;
-};
-
-DepthStencilState DisableDepth
-{
-    DepthEnable = FALSE;
-    DepthWriteMask = ZERO;
-};
-
-DepthStencilState EnableDepth
-{
-    DepthEnable = TRUE;
-    DepthWriteMask = ALL;
-};
-
-struct ColorsOutput
-{
-    float4 Diffuse;
-    float4 Specular;
-};
-
-ColorsOutput CalcLighting( float3 worldNormal, float3 worldPos, float3 cameraPos )
-{
-    ColorsOutput output = (ColorsOutput)0.0;
-    
-    for(int i=0; i<8; i++)
-    {
-        float3 toLight = g_lights[i].Position.xyz - worldPos;
-        float lightDist = length( toLight );
-        float fAtten = 1.0/dot( g_lights[i].Atten, float4(1,lightDist,lightDist*lightDist,0) );
-        float3 lightDir = normalize( toLight );
-        float3 halfAngle = normalize( normalize(-cameraPos) + lightDir );
-        
-        output.Diffuse += max(0,dot( lightDir, worldNormal ) * g_lights[i].Diffuse * fAtten) + g_lights[i].Ambient;
-        output.Specular += max(0,pow( dot( halfAngle, worldNormal ), 64 ) * g_lights[i].Specular * fAtten );
-    }
-    
-    return output;
-}
-
-//
-// VS for emulating fixed function pipeline
-//
-VSSceneOut VSScenemain(VSSceneIn input)
-{
-    VSSceneOut output = (VSSceneOut)0.0;
-
-    //output our final position in clipspace
-    float4 worldPos = mul( float4( input.pos, 1 ), g_mWorld );
-    float4 cameraPos = mul( worldPos, g_mView ); //Save cameraPos for fog calculations
-    output.pos = mul( cameraPos, g_mProj );
-    
-    //save world pos for later
-    output.wPos = worldPos;
-    
-    //save the fog distance for later
-    output.fogDist = cameraPos.z;
-    
-    //find our clipping planes (fixed function clipping is done in world space)
-    if( g_bEnableClipping )
-    {
-        worldPos.w = 1;
-        
-        //calc the distance from the 3 clipping planes
-        output.planeDist.x = dot( worldPos, g_clipplanes[0] );
-        output.planeDist.y = dot( worldPos, g_clipplanes[1] );
-        output.planeDist.z = dot( worldPos, g_clipplanes[2] );
-    }
-    else
-    {
-        output.planeDist.x = 1;
-        output.planeDist.y = 1;
-        output.planeDist.z = 1;
-    }
-    
-    //do gouraud lighting
-    if( g_bEnableLighting )
-    {
-        float3 worldNormal = normalize( mul( input.norm, (float3x3)g_mWorld ) );
-        output.wNorm = worldNormal;
-        ColorsOutput cOut = CalcLighting( worldNormal, worldPos, cameraPos );
-        output.colorD = cOut.Diffuse;
-        output.colorS = cOut.Specular;
-    }
-    else
-    {
-        output.colorD = float4(1,1,1,1);
-    }
-    
-    //propogate texture coordinate
-    output.tex = input.tex;
-    
-    return output;
-}
-
-//
-// VS for rendering in screen space
-//
-PSSceneIn VSScreenSpacemain(VSSceneIn input)
-{
-    PSSceneIn output = (PSSceneIn)0.0;
-
-    //output our final position
-    output.pos.x = (input.pos.x / (g_viewportWidth/2.0)) -1;
-    output.pos.y = -(input.pos.y / (g_viewportHeight/2.0)) +1;
-    output.pos.z = input.pos.z;
-    output.pos.w = 1;
-    
-    //propogate texture coordinate
-    output.tex = input.tex;
-    output.colorD = float4(1,1,1,1);
-    
-    return output;
-}
-
-//
-// GS for flat shaded rendering
-//
-
-[maxvertexcount(3)]
-void GSFlatmain( triangle VSSceneOut input[3], inout TriangleStream<VSSceneOut> FlatTriStream )
-{
-    VSSceneOut output;
-    
-    //
-    // Calculate the face normal
-    //
-    float3 faceEdgeA = input[1].wPos - input[0].wPos;
-    float3 faceEdgeB = input[2].wPos - input[0].wPos;
-
-    //
-    // Cross product
-    //
-    float3 faceNormal = cross(faceEdgeA, faceEdgeB);
-    
-    //
-    //calculate the face center
-    //
-    float3 faceCenter = (input[0].wPos + input[1].wPos + input[2].wPos)/3.0;
-    
-    //find world pos and camera pos
-    float4 worldPos = float4( faceCenter, 1 );
-    float4 cameraPos = mul( worldPos, g_mView );
-    
-    //do shading
-    float3 worldNormal = normalize( faceNormal );
-    ColorsOutput cOut = CalcLighting( worldNormal, worldPos, cameraPos );
-    
-    for(int i=0; i<3; i++)
-    {
-        output = input[i];
-        output.colorD = cOut.Diffuse;
-        output.colorS = cOut.Specular;
-        
-        FlatTriStream.Append( output );
-    }
-    FlatTriStream.RestartStrip();
-}
-
-//
-// GS for point rendering
-//
-[maxvertexcount(12)]
-void GSPointmain( triangle VSSceneOut input[3], inout TriangleStream<VSSceneOut> PointTriStream )
-{
-    VSSceneOut output;
-    
-    //
-    // Calculate the point size
-    //
-    //float fSizeX = (g_pointSize/g_viewportWidth)/4.0;
-    float fSizeY = (g_pointSize/g_viewportHeight)/4.0;
-    float fSizeX = fSizeY;
-    
-    for(int i=0; i<3; i++)
-    {
-        output = input[i];
-    
-        //find world pos and camera pos
-        float4 worldPos = float4(input[i].wPos,1);
-        float4 cameraPos = mul( worldPos, g_mView );
-        
-        //find our size
-        if( g_bPointScaleEnable )
-        {   
-            float dEye = length( cameraPos.xyz );
-            fSizeX = fSizeY = g_viewportHeight * g_pointSize * 
-                    sqrt( 1.0f/( g_pointScaleA + g_pointScaleB*dEye + g_pointScaleC*(dEye*dEye) ) );
-        }
-        
-        //do shading
-        if(g_bEnableLighting)
-        {
-            float3 worldNormal = input[i].wNorm;
-            ColorsOutput cOut = CalcLighting( worldNormal, worldPos, cameraPos );
-        
-            output.colorD = cOut.Diffuse;
-            output.colorS = cOut.Specular;
-        }
-        else
-        {
-            output.colorD = float4(1,1,1,1);
-        }
-        
-        output.tex = input[i].tex;
-        
-        //
-        // Emit two new triangles
-        //
-        for(int i=0; i<4; i++)
-        {
-            float4 outPos = mul( worldPos, g_mView );
-            output.pos = mul( outPos, g_mProj );
-            float zoverNear = (outPos.z)/g_nearPlane;
-            float4 posSize = float4( g_positions[i].x*fSizeX*zoverNear,
-                                     g_positions[i].y*fSizeY*zoverNear,
-                                     0,
-                                     0 );
-            output.pos += posSize;
-            
-            PointTriStream.Append(output);
-        }
-        PointTriStream.RestartStrip();
-    }
-}
-
-//
-// Calculates fog factor based upon distance
-//
-float CalcFogFactor( float d )
-{
-    float fogCoeff = 1.0;
-    
-    if( FOGMODE_LINEAR == g_fogMode )
-    {
-        fogCoeff = (g_fogEnd - d)/(g_fogEnd - g_fogStart);
-    }
-    else if( FOGMODE_EXP == g_fogMode )
-    {
-        fogCoeff = 1.0 / pow( E, d*g_fogDensity );
-    }
-    else if( FOGMODE_EXP2 == g_fogMode )
-    {
-        fogCoeff = 1.0 / pow( E, d*d*g_fogDensity*g_fogDensity );
-    }
-    
-    return clamp( fogCoeff, 0, 1 );
-}
-
-//
-// PS for rendering with clip planes
-//
-float4 PSScenemain(PSSceneIn input) : SV_Target
-{   
-    //calculate the fog factor  
-    float fog = CalcFogFactor( input.fogDist );
-    
-    //calculate the color based off of the normal, textures, etc
-    float4 normalColor = g_txDiffuse.Sample( g_samLinear, input.tex ) * input.colorD + input.colorS;
-    
-    //calculate the color from the projected texture
-    float4 cookieCoord = mul( float4(input.wPos,1), g_mLightViewProj );
-    //since we don't have texldp, we must perform the w divide ourselves befor the texture lookup
-    cookieCoord.xy = 0.5 * cookieCoord.xy / cookieCoord.w + float2( 0.5, 0.5 ); 
-    float4 cookieColor = float4(0,0,0,0);
-    if( cookieCoord.z > 0 )
-        cookieColor = g_txProjected.Sample( g_samLinear, cookieCoord.xy );
-    
-    //for standard light-modulating effects just multiply normalcolor and coookiecolor
-    normalColor += cookieColor;
-    
-    return fog * normalColor + (1.0 - fog)*g_fogColor;
-}
-
-//
-// PS for rendering with alpha test
-//
-float4 PSAlphaTestmain(PSSceneIn input) : SV_Target
-{   
-    float4 color =  g_txDiffuse.Sample( g_samLinear, input.tex ) * input.colorD;
-    if( color.a < 0.5 )
-        discard;
-    return color;
-}
-
-//
-// RenderSceneGouraud - renders gouraud-shaded primitives
-//
-technique10 RenderSceneGouraud
-{
-    pass p0
-    {
-        SetVertexShader( CompileShader( vs_4_0, VSScenemain() ) );
-        SetGeometryShader( NULL );
-        SetPixelShader( CompileShader( ps_4_0, PSScenemain() ) );
-        
-        SetDepthStencilState( EnableDepth, 0 );
-    }  
-}
-
-//
-// RenderSceneFlat - renders flat-shaded primitives
-//
-technique10 RenderSceneFlat
-{
-    pass p0
-    {
-        SetVertexShader( CompileShader( vs_4_0, VSScenemain() ) );
-        SetGeometryShader( CompileShader( gs_4_0, GSFlatmain() ) );
-        SetPixelShader( CompileShader( ps_4_0, PSScenemain() ) );
-        
-        SetDepthStencilState( EnableDepth, 0 );
-    }  
-}
-
-//
-// RenderScenePoint - replaces d3dfill_point
-//
-technique10 RenderScenePoint
-{
-    pass p0
-    {
-        SetVertexShader( CompileShader( vs_4_0, VSScenemain() ) );
-        SetGeometryShader( CompileShader( gs_4_0, GSPointmain() ) );
-        SetPixelShader( CompileShader( ps_4_0, PSScenemain() ) );
-        
-        SetDepthStencilState( EnableDepth, 0 );
-    }  
-}
-
-//
-// RenderScreneSpace - shows how to render something in screenspace
-//
-technique10 RenderScreenSpaceAlphaTest
-{
-    pass p0
-    {
-        SetVertexShader( CompileShader( vs_4_0, VSScreenSpacemain() ) );
-        SetGeometryShader( NULL );
-        SetPixelShader( CompileShader( ps_4_0, PSAlphaTestmain() ) );
-        
-        SetDepthStencilState( DisableDepth, 0 );
-    }  
-}
-
-//
-// RenderScreneSpace - shows how to render something in screenspace
-//
-technique10 RenderTextureOnly
-{
-    pass p0
-    {
-        SetVertexShader( CompileShader( vs_4_0, VSScenemain() ) );
-        SetGeometryShader( NULL );
-        SetPixelShader( CompileShader( ps_4_0, PSScenemain() ) );
-        
-        SetDepthStencilState( EnableDepth, 0 );
-    }  
-}
-