Documentation markup extraction (#1724)

* #include an absolute path didn't work - because paths were taken to always be relative.

* WIP extracting source documentation.

* WIP doc extraction.

* More stuff around doc markup extraction.

* More WIP around doc extraction.

* Fix some indexing issues.

* Initial doc extraction working.

* Renaming of types in markup extraction process.

* Extracting markup content.
Removing indenting.
Other fixes and improvements around document tools.

* WIP support for documentation system.

* Remove some commented out sections.

* Remove some comments that no longer apply.

* Improvements around SourceFile - such that more granularity around line ops.
Made some functionality explicitly work without source.
Improved Doc types nameing.

2 files changed, 25 insertions, 0 deletions

diff --git a/source/core/slang-string-util.cpp b/source/core/slang-string-util.cpp
index cac7c7cfa..6ce75f3f0 100644
--- a/source/core/slang-string-util.cpp
+++ b/source/core/slang-string-util.cpp
@@ -360,6 +360,28 @@ ComPtr<ISlangBlob> StringUtil::createStringBlob(const String& string)
     }
 }
 
+/* static */UnownedStringSlice StringUtil::trimEndOfLine(const UnownedStringSlice& line)
+{
+    // Strip CR/LF from end of line if present
+
+    const char* begin = line.begin();
+    const char* end = line.end();
+
+    if (end > begin)
+    {
+        const char c = end[-1];
+        // If last char is CR/LF move back a char
+        if (c == '\n' || c == '\r')
+        {
+            --end;
+            // If next char is a match for the CR/LF pair move back an extra char.
+            end -= Index((end > begin) && (c ^ end[-1]) == ('\r' ^ '\n'));
+        }
+    }
+
+    return line.head(Index(end - begin));
+}
+
 /* static */bool StringUtil::areLinesEqual(const UnownedStringSlice& inA, const UnownedStringSlice& inB)
 {
     UnownedStringSlice a(inA), b(inB), lineA, lineB;
diff --git a/source/core/slang-string-util.h b/source/core/slang-string-util.h
index dade8a61d..4672fa1d0 100644
--- a/source/core/slang-string-util.h
+++ b/source/core/slang-string-util.h
@@ -88,6 +88,9 @@ struct StringUtil
         /// Given text, splits into lines stored in outLines. NOTE! That lines is only valid as long as textIn remains valid
     static void calcLines(const UnownedStringSlice& textIn, List<UnownedStringSlice>& lines);
 
+        /// Given a line that may contain cr/lf, returns the the a slice that doesn't have trailing cr/lf
+    static UnownedStringSlice trimEndOfLine(const UnownedStringSlice& slice);
+
         /// Equal if the lines are equal (in effect a way to ignore differences in line breaks)
     static bool areLinesEqual(const UnownedStringSlice& a, const UnownedStringSlice& b);