AST nodes using C++ Extractor (#1341)

* Extractor builds without any reference to syntax (as it will be helping to produce this!).

* Change macros to include the super class.

* WIP replacing defs files.

* Added indexOf(const UnownedSubString& in) to UnownedSubString.
Refactored extractor
* Output a macro for each type with the extracted info - can be used during injection in class
* Simplify the header file - as can get super type and last from macro now
* Store the 'origin' of a definition

* Some small tidy ups to the extractor.

* Improve comments on the extractor options.

* Made CPPExtractor own SourceOrigins

* Small fixes around SourceOrigin.

* Small tidy up around macroOrign

* WIP Visitor seems now to work correctly.
Split out types used by ast into slang-ast-support-types.h

* Fix remaining problems with C++ extractor being used with AST nodes.
Add CountOf to extractor type ids.
Added ReflectClassInfo::getInfo to turn an ASTNodeType into a ReflectClassInfo

* Fix compiling on linux.
Fix typo in memset.

* Small tidy up around comments/layout.
Moved NodeBase casting to NodeBase.

* Make premake generate project that builds with cpp-extractor for AST.

* Get the source directory from the filter in premake.

* Fix typo in source path

* Explicitly set the source path for  premake generation for AST.

* Special case handling of override to apease Clang.

* Use a more general way to find the slang-ast-reflect.h file to run the extractor.

* Appveyor is not triggering slang-cpp-extractor - try putting dependson together.

* Put building slang-cpp-extractor first.

* Disable some project options to stop MSBuild producing internal compiler errors.

* Try reordering the projects in premake5.lua

* Hack to try and make slang-cpp-extractor built on appveyor.

* Disable flags - not required for MSBuild on appveyor.

* Disable flags not required for build on AppVeyor.

* Updated Visual Studio projects with slang-cpp-extractor.

* Added Visual Studio slang-cpp-extractor project.

3 files changed, 281 insertions, 1 deletions

diff --git a/tools/slang-cpp-extractor/slang-cpp-extractor-main.cpp b/tools/slang-cpp-extractor/slang-cpp-extractor-main.cpp
index f5724b7b6..9dd05ac78 100644
--- a/tools/slang-cpp-extractor/slang-cpp-extractor-main.cpp
+++ b/tools/slang-cpp-extractor/slang-cpp-extractor-main.cpp
@@ -1889,6 +1889,9 @@ SlangResult CPPExtractorApp::calcHeader(CPPExtractor& extractor, StringBuilder&
                 typeIndex++;
             }
 
+            _indent(1, out);
+            out << "CountOf\n";
+
             out << "};\n\n";
         }
 
@@ -1968,7 +1971,7 @@ SlangResult CPPExtractorApp::calcHeader(CPPExtractor& extractor, StringBuilder&
             // Output all of the definitions for each type
             for (Node* node : nodes)
             {
-                out << "#define SLANG_" << reflectTypeName << "_" << node->m_name.Content << "(x, param) ";
+                out << "#define " << m_options.m_prefixMark <<  reflectTypeName << "_" << node->m_name.Content << "(x, param) ";
 
                 // Output the X macro part
                 _indent(1, out);
@@ -2027,6 +2030,7 @@ SlangResult CPPExtractorApp::calcHeader(CPPExtractor& extractor, StringBuilder&
             }
         }
 
+    
         // Now pop the scope in revers
         for (Index j = baseScopePath.getCount() - 1; j >= 0; j--)
         {
@@ -2035,6 +2039,27 @@ SlangResult CPPExtractorApp::calcHeader(CPPExtractor& extractor, StringBuilder&
         }
     }
 
+    // Do macros by origin
+
+    out << "// Origin macros\n\n";
+
+    for (SourceOrigin* origin : extractor.getSourceOrigins())
+    {
+        out << "#define " << m_options.m_prefixMark <<  "ORIGIN_" << origin->m_macroOrigin << "_" << reflectTypeName << "(x, param) \\\n";
+
+        for (Node* node : origin->m_nodes)
+        {
+            if (!(node->m_isReflected && node->isClassLike()))
+            {
+                continue;
+            }
+
+            _indent(1, out);
+            out << "x(" << node->m_name.Content << ", param) \\\n";
+        }
+        out << "/* */\n\n";
+    }
+
     return SLANG_OK;
 }
 
diff --git a/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj b/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj
new file mode 100644
index 000000000..319632499
--- /dev/null
+++ b/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj
@@ -0,0 +1,192 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project DefaultTargets="Build" ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <ItemGroup Label="ProjectConfigurations">
+    <ProjectConfiguration Include="Debug|Win32">
+      <Configuration>Debug</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
+    <ProjectConfiguration Include="Debug|x64">
+      <Configuration>Debug</Configuration>
+      <Platform>x64</Platform>
+    </ProjectConfiguration>
+    <ProjectConfiguration Include="Release|Win32">
+      <Configuration>Release</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
+    <ProjectConfiguration Include="Release|x64">
+      <Configuration>Release</Configuration>
+      <Platform>x64</Platform>
+    </ProjectConfiguration>
+  </ItemGroup>
+  <PropertyGroup Label="Globals">
+    <ProjectGuid>{CA8A30D1-8FA9-4330-B7F7-84709246D8DC}</ProjectGuid>
+    <IgnoreWarnCompileDuplicatedFilename>true</IgnoreWarnCompileDuplicatedFilename>
+    <Keyword>Win32Proj</Keyword>
+    <RootNamespace>slang-cpp-extractor</RootNamespace>
+  </PropertyGroup>
+  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <UseDebugLibraries>true</UseDebugLibraries>
+    <CharacterSet>Unicode</CharacterSet>
+    <PlatformToolset>v140</PlatformToolset>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <UseDebugLibraries>true</UseDebugLibraries>
+    <CharacterSet>Unicode</CharacterSet>
+    <PlatformToolset>v140</PlatformToolset>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <UseDebugLibraries>false</UseDebugLibraries>
+    <CharacterSet>Unicode</CharacterSet>
+    <PlatformToolset>v140</PlatformToolset>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <UseDebugLibraries>false</UseDebugLibraries>
+    <CharacterSet>Unicode</CharacterSet>
+    <PlatformToolset>v140</PlatformToolset>
+  </PropertyGroup>
+  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
+  <ImportGroup Label="ExtensionSettings">
+  </ImportGroup>
+  <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <PropertyGroup Label="UserMacros" />
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
+    <LinkIncremental>true</LinkIncremental>
+    <OutDir>..\..\bin\windows-x86\debug\</OutDir>
+    <IntDir>..\..\intermediate\windows-x86\debug\slang-cpp-extractor\</IntDir>
+    <TargetName>slang-cpp-extractor</TargetName>
+    <TargetExt>.exe</TargetExt>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
+    <LinkIncremental>true</LinkIncremental>
+    <OutDir>..\..\bin\windows-x64\debug\</OutDir>
+    <IntDir>..\..\intermediate\windows-x64\debug\slang-cpp-extractor\</IntDir>
+    <TargetName>slang-cpp-extractor</TargetName>
+    <TargetExt>.exe</TargetExt>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
+    <LinkIncremental>false</LinkIncremental>
+    <OutDir>..\..\bin\windows-x86\release\</OutDir>
+    <IntDir>..\..\intermediate\windows-x86\release\slang-cpp-extractor\</IntDir>
+    <TargetName>slang-cpp-extractor</TargetName>
+    <TargetExt>.exe</TargetExt>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
+    <LinkIncremental>false</LinkIncremental>
+    <OutDir>..\..\bin\windows-x64\release\</OutDir>
+    <IntDir>..\..\intermediate\windows-x64\release\slang-cpp-extractor\</IntDir>
+    <TargetName>slang-cpp-extractor</TargetName>
+    <TargetExt>.exe</TargetExt>
+  </PropertyGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
+    <ClCompile>
+      <PrecompiledHeader>NotUsing</PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <AdditionalIncludeDirectories>..\..;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <DebugInformationFormat>EditAndContinue</DebugInformationFormat>
+      <Optimization>Disabled</Optimization>
+      <RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
+    </ClCompile>
+    <Link>
+      <SubSystem>Console</SubSystem>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+    </Link>
+  </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
+    <ClCompile>
+      <PrecompiledHeader>NotUsing</PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <AdditionalIncludeDirectories>..\..;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <DebugInformationFormat>EditAndContinue</DebugInformationFormat>
+      <Optimization>Disabled</Optimization>
+      <RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
+    </ClCompile>
+    <Link>
+      <SubSystem>Console</SubSystem>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+    </Link>
+  </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
+    <ClCompile>
+      <PrecompiledHeader>NotUsing</PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <PreprocessorDefinitions>NDEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <AdditionalIncludeDirectories>..\..;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <Optimization>Full</Optimization>
+      <FunctionLevelLinking>true</FunctionLevelLinking>
+      <IntrinsicFunctions>true</IntrinsicFunctions>
+      <MinimalRebuild>false</MinimalRebuild>
+      <StringPooling>true</StringPooling>
+      <RuntimeLibrary>MultiThreaded</RuntimeLibrary>
+    </ClCompile>
+    <Link>
+      <SubSystem>Console</SubSystem>
+      <EnableCOMDATFolding>true</EnableCOMDATFolding>
+      <OptimizeReferences>true</OptimizeReferences>
+    </Link>
+  </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
+    <ClCompile>
+      <PrecompiledHeader>NotUsing</PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <PreprocessorDefinitions>NDEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <AdditionalIncludeDirectories>..\..;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <Optimization>Full</Optimization>
+      <FunctionLevelLinking>true</FunctionLevelLinking>
+      <IntrinsicFunctions>true</IntrinsicFunctions>
+      <MinimalRebuild>false</MinimalRebuild>
+      <StringPooling>true</StringPooling>
+      <RuntimeLibrary>MultiThreaded</RuntimeLibrary>
+    </ClCompile>
+    <Link>
+      <SubSystem>Console</SubSystem>
+      <EnableCOMDATFolding>true</EnableCOMDATFolding>
+      <OptimizeReferences>true</OptimizeReferences>
+    </Link>
+  </ItemDefinitionGroup>
+  <ItemGroup>
+    <ClInclude Include="..\..\source\slang\slang-diagnostics.h" />
+    <ClInclude Include="..\..\source\slang\slang-file-system.h" />
+    <ClInclude Include="..\..\source\slang\slang-lexer.h" />
+    <ClInclude Include="..\..\source\slang\slang-name.h" />
+    <ClInclude Include="..\..\source\slang\slang-source-loc.h" />
+    <ClInclude Include="..\..\source\slang\slang-token.h" />
+    <ClInclude Include="slang-cpp-extractor-diagnostic-defs.h" />
+    <ClInclude Include="slang-cpp-extractor-diagnostics.h" />
+  </ItemGroup>
+  <ItemGroup>
+    <ClCompile Include="..\..\source\slang\slang-diagnostics.cpp" />
+    <ClCompile Include="..\..\source\slang\slang-file-system.cpp" />
+    <ClCompile Include="..\..\source\slang\slang-lexer.cpp" />
+    <ClCompile Include="..\..\source\slang\slang-name.cpp" />
+    <ClCompile Include="..\..\source\slang\slang-source-loc.cpp" />
+    <ClCompile Include="..\..\source\slang\slang-token.cpp" />
+    <ClCompile Include="slang-cpp-extractor-diagnostics.cpp" />
+    <ClCompile Include="slang-cpp-extractor-main.cpp" />
+  </ItemGroup>
+  <ItemGroup>
+    <ProjectReference Include="..\..\source\core\core.vcxproj">
+      <Project>{F9BE7957-8399-899E-0C49-E714FDDD4B65}</Project>
+    </ProjectReference>
+  </ItemGroup>
+  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
+  <ImportGroup Label="ExtensionTargets">
+  </ImportGroup>
+</Project>
\ No newline at end of file
diff --git a/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj.filters b/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj.filters
new file mode 100644
index 000000000..e5097b221
--- /dev/null
+++ b/tools/slang-cpp-extractor/slang-cpp-extractor.vcxproj.filters
@@ -0,0 +1,63 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <ItemGroup>
+    <Filter Include="Header Files">
+      <UniqueIdentifier>{21EB8090-0D4E-1035-B6D3-48EBA215DCB7}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Source Files">
+      <UniqueIdentifier>{E9C7FDCE-D52A-8D73-7EB0-C5296AF258F6}</UniqueIdentifier>
+    </Filter>
+  </ItemGroup>
+  <ItemGroup>
+    <ClInclude Include="..\..\source\slang\slang-diagnostics.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\source\slang\slang-file-system.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\source\slang\slang-lexer.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\source\slang\slang-name.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\source\slang\slang-source-loc.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\source\slang\slang-token.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="slang-cpp-extractor-diagnostic-defs.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="slang-cpp-extractor-diagnostics.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+  </ItemGroup>
+  <ItemGroup>
+    <ClCompile Include="..\..\source\slang\slang-diagnostics.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\source\slang\slang-file-system.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\source\slang\slang-lexer.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\source\slang\slang-name.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\source\slang\slang-source-loc.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\source\slang\slang-token.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="slang-cpp-extractor-diagnostics.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="slang-cpp-extractor-main.cpp">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+  </ItemGroup>
+</Project>
\ No newline at end of file