AST serialize improvements (#1412)

* Try to fix problem with C++ extractor concating tokens producing an erroneous result.

* Improve naming/comments around C++ extractor fix.

* Another small improvement around space concating when outputing token list.

* Handle some more special cases for consecutive tokens for C++ extractor concat of tokens.

* WIP AST serialization.

* Comment out so compile works.

* More work on AST serialization.

* WIP AST serialize.

* WIP AST Serialization - handling more types.

* WIP: Compiles but not all types are converted, as not all List element types are handled.

* Compiles with array types.

* Finish off AST serialization of remaining types.

* Remove ComputedLayoutModifier and TupleVarModifier.

* Add fields to ASTSerialClass type.

* Construct AST type layout.

* AST Serialization working for writing to ASTSerialWriter.

* Removed call to ASTSerialization::selfTest in session creation.

* Fixes for gcc.

* Diagnostics handling - better handling of dashify.

* Improve comment around DiagnosticLookup.

* Updated VS project.

* Write out as a Stream, taking into account alignment.

* First pass at serializing in AST.

* Added support for deserializing arrays.

* Small bug fixes.

* Fix problem calculating layout.
Split out loading on entries.

* Fix typo in AST conversion.

* Add some flags to control AST dumping.

* Fix bug from a typo.

* Special case handling of Name* in AST serialization.

* Special case handling of Token lexemes, make Names on read.

* Documentation on AST serialization.

* ASTSerialTestUtil - put AST testing functions.
Fix typo that broke compilation.

* Fix typo.

1 files changed, 2 insertions, 14 deletions

diff --git a/source/slang/slang.cpp b/source/slang/slang.cpp
index 6d4a27733..b79cecb59 100644
--- a/source/slang/slang.cpp
+++ b/source/slang/slang.cpp
@@ -1064,7 +1064,7 @@ void FrontEndCompileRequest::parseTranslationUnit(
             StringBuilder buf;
             SourceWriter writer(linkage->getSourceManager(), LineDirectiveMode::None);
 
-            ASTDumpUtil::dump(translationUnit->getModuleDecl(), ASTDumpUtil::Style::Flat, &writer);
+            ASTDumpUtil::dump(translationUnit->getModuleDecl(), ASTDumpUtil::Style::Flat, 0, &writer);
 
             const String& path = sourceFile->getPathInfo().foundPath;
             if (path.getLength())
@@ -1079,19 +1079,7 @@ void FrontEndCompileRequest::parseTranslationUnit(
 #if 0
         // Test serialization
         {
-            RefPtr<ASTSerialClasses> classes = new ASTSerialClasses;
-
-            OwnedMemoryStream stream(FileAccess::ReadWrite);
-
-            {
-                ASTSerialWriter writer(classes);
-
-                // Lets serialize it all
-                writer.addPointer(translationUnit->getModuleDecl());
-                // Let's stick it all in a stream
-                writer.write(&stream);
-            }
-
+            ASTSerialTestUtil::testSerialize(translationUnit->getModuleDecl(), getSession()->getRootNamePool(), getLinkage()->getASTBuilder()->getSharedASTBuilder(), getSourceManager());
         }
 #endif