Improve GLSL coverage of boolean binary ops (#1335)

* Improve GLSL coverage of boolean binary ops

This change ensures that the `&&`, `||`, `&`, `|`, and `^` apply correctly to vectors of `bool` values when targetting GLSL.

Most of the changes are in the GLSL emit path, where the IR instructions for these operators are bottlenecked through a small set of helper routines to cover the different cases. In general:

* The vector variants of the operations are implemented by casting to `uint` vectors, performing bitwise ops, then casting back
* The scalar variants are handled by conveting the bitwise operations to their equivalent logical operator (the one interesting case there is bitwise `^` where the equivalent logical operation on `bool` is `!=`)

This change makes it clear that our IR really shouldn't have distinct opcodes for logical vs. bitwise and/or/xor, and instead should just have a single family of operations where the behavior differs based on the type of the operand. That is already *de facto* the way things work (a user can always write `&`, `|` and `^` and expect them to work on `bool` and vectors of `bool`), so that the GLSL output path has to deal with the overlap. Having two sets of IR ops here actually makes for more code instead of less.

* Fixups: review feedback and test ! operator

2 files changed, 47 insertions, 0 deletions

diff --git a/tests/cross-compile/glsl-bool-ops.slang b/tests/cross-compile/glsl-bool-ops.slang
new file mode 100644
index 000000000..a7cf21f23
--- /dev/null
+++ b/tests/cross-compile/glsl-bool-ops.slang
@@ -0,0 +1,43 @@
+// glsl-bool-ops.slang
+
+// This test exists to ensure that binary operations on
+// vectors of `bool` produce correct GLSL/SPIR-V output.
+
+//TEST(compute):COMPARE_COMPUTE:-dx11 -compute
+//TEST(compute):COMPARE_COMPUTE:-vk -compute
+
+uint test(uint val)
+{
+	bool2 a = val > uint2(0);
+	bool2 b = (val & uint2(1)) == 0;
+	bool2 c = val < uint2(3);
+
+	bool2 d = a && b;
+	bool2 e = a || c;
+
+	bool2 f = !(a & b);
+	bool2 g = a | c;
+	bool2 h = a ^ c;
+
+	uint2 t = 0;
+	t = t*2 + d;
+	t = t*2 + e;
+	t = t*2 + f;
+	t = t*2 + g;
+	t = t*2 + h;
+
+	uint result = 0;
+	result = result*256 + t.x;
+	result = result*256 + t.y;
+	return result;
+}
+
+//TEST_INPUT:ubuffer(data=[0 0 0 0], stride=4):out,name outputBuffer
+RWStructuredBuffer<uint> outputBuffer;
+
+[numthreads(4, 1, 1)]
+void computeMain(uint3 dispatchThreadID : SV_DispatchThreadID)
+{
+    uint tid = dispatchThreadID.x;
+    outputBuffer[tid] = test(tid);
+}
diff --git a/tests/cross-compile/glsl-bool-ops.slang.expected.txt b/tests/cross-compile/glsl-bool-ops.slang.expected.txt
new file mode 100644
index 000000000..b511eeb33
--- /dev/null
+++ b/tests/cross-compile/glsl-bool-ops.slang.expected.txt
@@ -0,0 +1,4 @@
+F0F
+E0E
+1A1A
+F0F
\ No newline at end of file