Mesh shader support (#2464)

* Add gdb generated files to .gitignore

* Switch to c++17

TODO: Ellie update coding style doc

* WIP mesh shaders

* Add MeshOutputType and mesh output decorations

* Lift array type layout creation out of _createTypeLayout

in preparation for sharing it elsewhere

* Initial pass at GLSL legalization for mesh shaders

* Create output types for builtin mesh outputs

This should be rendered as an out paramter block

* Handle writes to member fields in mesh shader output

* Per primitive output from mesh shaders

* Add mesh shader tests

* Redeclare mesh output builtins

* Remove unused instruction

* Emit explicit mesh output max max size

* Add unimplemented warning for array members in mesh output

* Implement mesh output splitting for GLSL in terms of getSubscriptVal

* Allow HLSL syntax for mesh output modifiers

* Improve error messages for mesh output

* Add test for HLSL style mesh output syntax

* Emit explicit mesh output indices max size

* HLSL generation support for mesh shaders

* Better errors for mesh shader misuse

* Neaten comments

* Regenerate vs2019 project files

* Fix build on vs2019

* Retreat on c++17

Will make the change in a separate PR

* slang-glslang binary dep 11.10.0 -> 11.12.0-32

* Fixes for msvc compiler

* Update msvc project

1 files changed, 29 insertions, 0 deletions

diff --git a/source/slang/slang-ast-builder.cpp b/source/slang/slang-ast-builder.cpp
index 6249d7825..819fe7d25 100644
--- a/source/slang/slang-ast-builder.cpp
+++ b/source/slang/slang-ast-builder.cpp
@@ -345,6 +345,35 @@ bool ASTBuilder::isDifferentiableInterfaceAvailable()
     return (m_sharedASTBuilder->tryFindMagicDecl("DifferentiableType") != nullptr);
 }
 
+MeshOutputType* ASTBuilder::getMeshOutputTypeFromModifier(
+    HLSLMeshShaderOutputModifier* modifier,
+    Type* elementType,
+    IntVal* maxElementCount)
+{
+    SLANG_ASSERT(modifier);
+    SLANG_ASSERT(elementType);
+    SLANG_ASSERT(maxElementCount);
+
+    const char* declName
+        = as<HLSLVerticesModifier>(modifier) ? "VerticesType"
+        : as<HLSLIndicesModifier>(modifier) ? "IndicesType"
+        : as<HLSLPrimitivesModifier>(modifier) ? "PrimitivesType"
+        : (SLANG_UNEXPECTED("Unhandled mesh output modifier"), nullptr);
+    auto genericDecl = dynamicCast<GenericDecl>(m_sharedASTBuilder->findMagicDecl(declName));
+
+    auto typeDecl = genericDecl->inner;
+
+    auto substitutions = getOrCreate<GenericSubstitution>(
+        genericDecl,
+        elementType,
+        maxElementCount);
+
+    auto declRef = DeclRef<Decl>(typeDecl, substitutions);
+    auto rsType = DeclRefType::create(this, declRef);
+
+    return as<MeshOutputType>(rsType);
+}
+
 DeclRef<Decl> ASTBuilder::getBuiltinDeclRef(const char* builtinMagicTypeName, Val* genericArg)
 {
     DeclRef<Decl> declRef;