Add IREnumType to distinguish enums from ints and each other (#6973)

* Add IREnumType to distinguish enums from ints and each other

* Add issue example as test

* format code

* Add expected test output

* Fix peephole optimization hanging

No idea why this PR triggered this, but there seems to have been a clear bug
here anyway, so may just as well fix it now.

* Move enum lowering later

* Add linkage decoration to enum type

* Use filecheck-buffer instead of expected.txt

* Fix comment

* Make enum casts actually use IR enum casts

They were all BuiltinCasts by accident

* Lower enum type before VM

* Deal with rate-qualified types in enum cast

* Allow any value marshalling for enum types

* Handle new enum instructions in a couple more switches

* Fix formatting

---------

Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>

1 files changed, 58 insertions, 0 deletions

diff --git a/tests/bugs/gh-6964.slang b/tests/bugs/gh-6964.slang
new file mode 100644
index 000000000..b283a17a8
--- /dev/null
+++ b/tests/bugs/gh-6964.slang
@@ -0,0 +1,58 @@
+//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK): -vk
+//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK): -dx12
+//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK): -cpu
+
+// CHECK: 1
+// CHECK-NEXT: 0
+// CHECK-NEXT: 2
+// CHECK-NEXT: 3
+
+//TEST_INPUT:ubuffer(data=[0 0 0 0], stride=4):out,name=outputBuffer
+RWStructuredBuffer<int> outputBuffer;
+
+interface IThing
+{
+    void writeInfo(int i);
+}
+
+enum FirstEnum
+{
+    A,
+    B
+};
+
+enum SecondEnum
+{
+    C,
+    D=3
+};
+
+extension FirstEnum: IThing
+{
+    void writeInfo(int i)
+    {
+        outputBuffer[i] = 1;
+        outputBuffer[i+1] = this;
+    }
+}
+
+extension SecondEnum: IThing
+{
+    void writeInfo(int i)
+    {
+        outputBuffer[i] = 2;
+        outputBuffer[i+1] = this;
+    }
+}
+
+void indirectionFunc<T: IThing>(T val, int i)
+{
+    val.writeInfo(i);
+}
+
+[numthreads(1, 1, 1)]
+void computeMain(int3 dispatchThreadID: SV_DispatchThreadID)
+{
+    indirectionFunc(FirstEnum.A, 0);
+    indirectionFunc(SecondEnum.D, 2);
+}