RTTI/JSON (#2021)

* #include an absolute path didn't work - because paths were taken to always be relative.

* Use 'Process' to communicate with an command line tool.

* Remove slang-win-stream

* Tidy up windows ProcessUtil.

* First version of BufferedReadStream.

* Windows working IPC for steams.

* Test proxy count option.

* Split Process/ProcessUtil. Process is platform dependant. ProcessUtil are functions that are platform independent.

* First implementation of Unix Process interface.

* Unix process compiles on cygwin.

* Fix typo in unix process.

* Separate unix pipe stream error of invalid access, from pipe availability.

* Fix in standard line extraction.

* Make fd non blocking.

* Fix issues with Windows Process streams.

* Added UnixPipe.

* Some fixes around UnixPipeStream.

* Make a unix stream closed explicit.

* Hack to debug linux process/stream.

* Revert to old linux pipe handling.

* Pass executable path for unit tests.
Split out CommandLine into own source.

* Small improvements in process/command line.

* Check process behavior with crash.

* Make stderr and stdout unbuffered for crash testing.

* Only turn disable buffering in crash test.

* Disable crash test on CI.

* Fix crash on clang/linux.

* Enable crash test.
Remove _appendBuffer as can use StreamUtil functionality.

* Added inital processing for http headers.

* Small improvements to HttpHeader.

* First pass HTTPPacketConnection working on windows.

* Enable other Process communication tests.

* Update comments.

* WIP JSON RPC.

* Add terminate to Process.
Made JSONRPC a Util.

* Small tidy up around HTTPPacketConnection.

* Improve process termination options.

* WIP for test-server.

* Add diagnostics error handling to test-server.

* Improved JSON support.
Parsing/creating JSON-RPC messages.

* WIP JSONRPC parsing.

* First pass RttiInfo support.

* WIP converting between JSON/native types.

* Project files.

* Split out RttiUtil.
Made RttiInfo constuction thread safe.

* WIP RTTI<->JSON.

* Add diagnostics to JSON<->native conversions.

* Make RttiInfo for structs globals. Avoids problem around derived types (like pointers), being able to cause an abort.

* Add pointer support to RTTI.
Fixed some compilation issues on linux.

* Add fixed array support.

* Added Rtti unit test.

* Add rtti unit test.

* Split out quoted/unquoted key handling.
Fix bugs in JSON value/container.
Added JSON native test.

* Make default array allocator use malloc/free.
Remove the new[] handler (doesn't work on visuals studio).

* Fix for linux warning.

* Remove some test code.

* Fix issues on x86 win.

* Fix warning on aarch64.

1 files changed, 402 insertions, 0 deletions

diff --git a/source/compiler-core/slang-json-rpc.cpp b/source/compiler-core/slang-json-rpc.cpp
index 15da92ff0..1ca55e9eb 100644
--- a/source/compiler-core/slang-json-rpc.cpp
+++ b/source/compiler-core/slang-json-rpc.cpp
@@ -1,5 +1,7 @@
 #include "slang-json-rpc.h"
 
+#include "../../slang-com-helper.h"
+
 namespace Slang {
 
 // https://www.jsonrpc.org/specification
@@ -7,6 +9,406 @@ namespace Slang {
 //    m_sourceManager.initialize(nullptr, nullptr);
 //    m_diagnosticSink.init(&m_sourceManager, &JSONLexer::calcLexemeLocation);
 
+static const auto g_jsonRpc = UnownedStringSlice::fromLiteral("jsonrpc");
+static const auto g_jsonRpcVersion = UnownedStringSlice::fromLiteral("2.0");
+static const auto g_method = UnownedStringSlice::fromLiteral("method");
+static const auto g_id = UnownedStringSlice::fromLiteral("id");
+static const auto g_params = UnownedStringSlice::fromLiteral("params");
+static const auto g_code = UnownedStringSlice::fromLiteral("code");
+static const auto g_error = UnownedStringSlice::fromLiteral("error");
+static const auto g_message = UnownedStringSlice::fromLiteral("message");
+static const auto g_result = UnownedStringSlice::fromLiteral("result");
+static const auto g_data = UnownedStringSlice::fromLiteral("data");
+
+// Add the fields.
+// TODO(JS): This is a little verbose, and could be improved on with something like
+// * Tool that automatically generated from C++ (say via the C++ extractor)
+// * Macro magic to simplify the construction
+static const StructRttiInfo _makeJSONRPCErrorResponse_ErrorRtti()
+{
+    JSONRPCErrorResponse::Error obj;
+    StructRttiBuilder builder(&obj, "JSONRPCErrorResponse::Error", nullptr);
+    builder.addField("code", &obj.code);
+    builder.addField("message", &obj.message);
+    return builder.make();
+}
+/* static */const StructRttiInfo JSONRPCErrorResponse::Error::g_rttiInfo = _makeJSONRPCErrorResponse_ErrorRtti();
+
+static const StructRttiInfo _makeJSONRPCErrorResponseRtti()
+{
+    JSONRPCErrorResponse obj;
+    StructRttiBuilder builder(&obj, "JSONRPCErrorResponse", nullptr);
+
+    builder.addField("error", &obj.error);
+    builder.addField("data", &obj.data, StructRttiInfo::Flag::Optional);
+    builder.addField("id", &obj.id, combine(StructRttiInfo::Flag::Optional, RttiDefaultValue::MinusOne));
+
+    return builder.make();
+}
+/* static */const StructRttiInfo JSONRPCErrorResponse::g_rttiInfo = _makeJSONRPCErrorResponseRtti();
+
+static const StructRttiInfo _makeJSONRPCCallResponseRtti()
+{
+    JSONRPCCall obj;
+    StructRttiBuilder builder(&obj, "JSONRPCCall", nullptr);
+
+    builder.addField("method", &obj.method);
+    builder.addField("params", &obj.params, StructRttiInfo::Flag::Optional);
+    builder.addField("id", &obj.id, combine(StructRttiInfo::Flag::Optional, RttiDefaultValue::MinusOne));
+
+    return builder.make();
+}
+/* static */const StructRttiInfo JSONRPCCall::g_rttiInfo = _makeJSONRPCCallResponseRtti();
+
+static const StructRttiInfo _makeJSONResultResponseResponseRtti()
+{
+    JSONResultResponse obj;
+    StructRttiBuilder builder(&obj, "JSONResultResponse", nullptr);
+
+    builder.addField("result", &obj.result);
+    builder.addField("id", &obj.id, combine(StructRttiInfo::Flag::Optional, RttiDefaultValue::MinusOne));
+
+    return builder.make();
+}
+/* static */const StructRttiInfo JSONResultResponse::g_rttiInfo = _makeJSONResultResponseResponseRtti();
+
+
+/* static */JSONValue JSONRPCUtil::createCall(JSONContainer* container, const UnownedStringSlice& method, JSONValue params, Int id)
+{
+    const Index maxPairs = 4;
+    JSONKeyValue pairs[maxPairs];
+
+    Index i = 0;
+
+    // Version number is a string
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_jsonRpc), container->createString(g_jsonRpcVersion));
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_method), container->createString(method));
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_params), params);
+
+    if (id >= 0)
+    {
+        pairs[i++] = JSONKeyValue::make(container->getKey(g_id), JSONValue::makeInt(id));
+    }
+
+    return container->createObject(pairs, i);
+}
+
+/* static */JSONValue JSONRPCUtil::createCall(JSONContainer* container, const UnownedStringSlice& method, Int id)
+{
+    const Index maxPairs = 3;
+    JSONKeyValue pairs[maxPairs];
+    Index i = 0;
+    // Version number is a string
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_jsonRpc), container->createString(g_jsonRpcVersion));
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_method), container->createString(method));
+
+    if (id >= 0)
+    {
+        pairs[i++] = JSONKeyValue::make(container->getKey(g_id), JSONValue::makeInt(id));
+    }
+
+    return container->createObject(pairs, i);
+}
+
+/* static */JSONValue JSONRPCUtil::createErrorResponse(JSONContainer* container, Index code, const UnownedStringSlice& message, const JSONValue& data, Int id)
+{
+    // Set up the error value
+    JSONValue errorValue;
+    {
+        const Index maxPairs = 2;
+        JSONKeyValue pairs[maxPairs];
+        Index i = 0;
+
+        if (code != 0)
+        {
+            pairs[i++] = JSONKeyValue::make(container->getKey(g_code), JSONValue::makeInt(code));
+        }
+        if (message.getLength() > 0)
+        {
+            pairs[i++] = JSONKeyValue::make(container->getKey(g_message), container->createString(message));
+        }
+        errorValue = container->createObject(pairs, i);
+    }
+
+    const Index maxPairs = 4;
+    JSONKeyValue pairs[maxPairs];
+    Index i = 0;
+
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_jsonRpc), container->createString(g_jsonRpcVersion));
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_error), errorValue);
+
+    if (data.isValid())
+    {
+        pairs[i++] = JSONKeyValue::make(container->getKey(g_data), data);
+    }
+
+    if (id >= 0)
+    {
+        pairs[i++] = JSONKeyValue::make(container->getKey(g_id), JSONValue::makeInt(id));
+    }
+
+    return container->createObject(pairs, i);
+}
+
+
+/* static */JSONValue JSONRPCUtil::createErrorResponse(JSONContainer* container, ErrorCode code, const UnownedStringSlice& message, const JSONValue& data, Int id)
+{
+    return createErrorResponse(container, Index(code), message, data, id);
+}
+
+/* static */JSONValue JSONRPCUtil::createResultResponse(JSONContainer* container, const JSONValue& resultValue, Int id)
+{
+    const Index maxPairs = 3;
+    JSONKeyValue pairs[maxPairs];
+    Index i = 0;
+
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_jsonRpc), container->createString(g_jsonRpcVersion));
+    pairs[i++] = JSONKeyValue::make(container->getKey(g_result), resultValue);
+
+    if (id >= 0)
+    {
+        pairs[i++] = JSONKeyValue::make(container->getKey(g_id), JSONValue::makeInt(id));
+    }
+
+    return container->createObject(pairs, i);
+}
+
+/* static */ JSONRPCUtil::ResponseType JSONRPCUtil::getResponseType(JSONContainer* container, const JSONValue& response)
+{
+    if (response.getKind() == JSONValue::Kind::Object)
+    {
+        const JSONKey resultKey = container->findKey(g_result);
+        const JSONKey errorKey = container->findKey(g_error);
+
+        auto pairs = container->getObject(response);
+
+        for (const auto& pair : pairs)
+        {
+            if (pair.key == resultKey)
+            {
+                return ResponseType::Result;
+            }
+            else if (pair.key == errorKey)
+            {
+                return ResponseType::Error;
+            }
+        }
+    }
+
+    return ResponseType::Error;
+}
+
+static SlangResult _parseError(JSONContainer* container, const JSONValue& error, JSONRPCUtil::ErrorResponse& out)
+{
+    if (error.getKind() != JSONValue::Kind::Object)
+    {
+        return SLANG_FAIL;
+    }
+    const auto pairs = container->getObject(error);
+
+    const JSONKey messageKey = container->findKey(g_message);
+    const JSONKey codeKey = container->findKey(g_code);
+    const JSONKey dataKey = container->findKey(g_data);
+
+    Int fieldBits = 0;
+
+    for (auto const& pair : pairs)
+    {
+        if (pair.key == messageKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::String)
+            {
+                return SLANG_FAIL;
+            }
+            out.message = container->getString(pair.value);
+            fieldBits |= 0x1;
+        }
+        else if (pair.key == codeKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::Integer)
+            {
+                return SLANG_FAIL;
+            }
+            out.code = Index(pair.value.asInteger());
+            fieldBits |= 0x2;
+        }
+        else if (pair.key == dataKey)
+        {
+            out.data = pair.value;
+            fieldBits |= 0x4;
+        }
+        else
+        {
+            return SLANG_FAIL;
+        }
+    }
+
+    // Check all required fields are set
+    return (fieldBits & 0x3) == 0x3 ? SLANG_OK : SLANG_FAIL;
+}
+
+/* static */SlangResult JSONRPCUtil::parseError(JSONContainer* container, const JSONValue& response, ErrorResponse& out)
+{
+    if (response.getKind() != JSONValue::Kind::Object)
+    {
+        return SLANG_FAIL;
+    }
+
+    const auto pairs = container->getObject(response);
+
+    const JSONKey jsonRpcKey = container->findKey(g_jsonRpc);
+    const JSONKey errorKey = container->findKey(g_error);
+    const JSONKey idKey = container->findKey(g_id);
+
+    Int fieldBits = 0;
+
+    for (auto const& pair : pairs)
+    {
+        if (pair.key == jsonRpcKey)
+        {
+            if (!container->areEqual(pair.value, g_jsonRpcVersion))
+            {
+                return SLANG_FAIL;
+            }
+            fieldBits |= 0x1;
+        }
+        else if (pair.key == errorKey)
+        {
+            // We need to decode the error 
+            SLANG_RETURN_ON_FAIL(_parseError(container, pair.value, out));
+            fieldBits |= 0x2;
+        }
+        else if (pair.key == idKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::Integer)
+            {
+                return SLANG_FAIL;
+            }
+            out.id = Int(pair.value.asInteger());
+            fieldBits |= 0x4;
+        }
+        else
+        {
+            // Unknown key
+            return SLANG_FAIL;
+        }
+    }
+
+    // Check all the required bits are set
+    return ((fieldBits & 0x3) == 0x3) ? SLANG_OK : SLANG_FAIL;
+}
+
+/* static */SlangResult JSONRPCUtil::parseResult(JSONContainer* container, const JSONValue& response, ResultResponse& out)
+{
+    if (response.getKind() != JSONValue::Kind::Object)
+    {
+        return SLANG_FAIL;
+    }
+
+    const auto pairs = container->getObject(response);
+
+    const JSONKey jsonRpcKey = container->findKey(g_jsonRpc);
+    const JSONKey resultKey = container->findKey(g_result);
+    const JSONKey idKey = container->findKey(g_id);
+
+    Int fieldBits = 0;
+
+    for (auto const& pair : pairs)
+    {
+        if (pair.key == jsonRpcKey)
+        {
+            if (!container->areEqual(pair.value, g_jsonRpcVersion))
+            {
+                return SLANG_FAIL;
+            }
+            fieldBits |= 0x1;
+        }
+        else if (pair.key == resultKey)
+        {
+            out.result = pair.value;
+            fieldBits |= 0x2;
+        }
+        else if (pair.key == idKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::Integer)
+            {
+                return SLANG_FAIL;
+            }
+            out.id = Int(pair.value.asInteger());
+            fieldBits |= 0x4;
+        }
+        else
+        {
+            // Unknown key
+            return SLANG_FAIL;
+        }
+    }
+
+    // Check all the required bits are set
+    return ((fieldBits & 0x3) == 0x3) ? SLANG_OK : SLANG_FAIL;
+}
+
+/* static */SlangResult JSONRPCUtil::parseCall(JSONContainer* container, const JSONValue& value, Call& out)
+{
+    if (value.getKind() != JSONValue::Kind::Object)
+    {
+        return SLANG_FAIL;
+    }
+
+    const auto pairs = container->getObject(value);
+
+    const JSONKey jsonRpcKey = container->findKey(g_jsonRpc);
+    const JSONKey methodKey = container->findKey(g_method);
+    const JSONKey paramsKey = container->findKey(g_params);
+    const JSONKey idKey = container->findKey(g_id);
+
+    Int fieldBits = 0;
+
+    for (auto const& pair : pairs)
+    {
+        if (pair.key == jsonRpcKey)
+        {
+            if (!container->areEqual(pair.value, g_jsonRpcVersion))
+            {
+                return SLANG_FAIL;
+            }
+            fieldBits |= 0x1;
+        }
+        else if (pair.key == methodKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::String)
+            {
+                return SLANG_FAIL;
+            }
+            out.method = container->getString(pair.value);
+            fieldBits |= 0x2;
+        }
+        else if (pair.key == idKey)
+        {
+            if (pair.value.getKind() != JSONValue::Kind::Integer)
+            {
+                return SLANG_FAIL;
+            }
+            out.id = Int(pair.value.asInteger());
+            fieldBits |= 0x4;
+        }
+        else if (pair.key == paramsKey)
+        {
+            out.params = pair.value;
+            fieldBits |= 0x8;
+        }
+        else
+        {
+            // Unknown key
+            return SLANG_FAIL;
+        }
+    }
+
+    // Check all the required bits are set
+    return ((fieldBits & 0x3) == 0x3) ? SLANG_OK : SLANG_FAIL;
+}
+
+
 /* static */SlangResult JSONRPCUtil::parseJSON(const UnownedStringSlice& slice, JSONContainer* container, DiagnosticSink* sink, JSONValue& outValue)
 {
     SourceManager* sourceManager = sink->getSourceManager();