Fix compiler crash when enum is used as vertex output data (#7915)

* Initial plan

* Initial investigation and plan for enum vertex output fix

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Fix compiler crash when enum is used as vertex output data

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Fix compiler crash when enum is used as vertex output data

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Address reviewer feedback: use SLANG_ASSERT and improve CHECK directives

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

1 files changed, 10 insertions, 0 deletions

diff --git a/source/slang/slang-parameter-binding.cpp b/source/slang/slang-parameter-binding.cpp
index 06d2b1f34..89e633857 100644
--- a/source/slang/slang-parameter-binding.cpp
+++ b/source/slang/slang-parameter-binding.cpp
@@ -2554,6 +2554,16 @@ static RefPtr<TypeLayout> processEntryPointVaryingParameter(
                     globalGenericParamDecl.getDecl());
             }
         }
+        else if (auto enumDeclRef = declRef.as<EnumDecl>())
+        {
+            // We handle an enumeration type as its tag type for varying parameters.
+            // This allows enums to be used in vertex output/input similar to their
+            // underlying integer types.
+            //
+            auto tagType = enumDeclRef.getDecl()->tagType;
+            SLANG_ASSERT(tagType);
+            return processEntryPointVaryingParameter(context, tagType, state, varLayout);
+        }
         else if (auto associatedTypeParam = declRef.as<AssocTypeDecl>())
         {
             RefPtr<TypeLayout> assocTypeLayout = new TypeLayout();