Ability to set Paths on Pass Through/Back End Compilers (#1010)

* Expanded prelude for some other resource types. Disable C++ output for ParameterGroup.

* WIP: Layout for CPU.

* Fixes to CPU layout.

* WIP: The uniform is output, but the variable definition is not.

* WIP: Entry point parameters to global scope in C++.
Handling of resource types (in so far as outputting)

* Some discussion of ABI and different input types.

* WIP: More C++ support around resource types.

* WIP: Split up variables into different structures on emit.

* WIP: Emitting C++ with wrapping up of 'Context'

* WIP: C++ code has access to semantic values.
Wrap in struct so can use method calls to pass shared state.
Disable  legalizeResourceTypes and legalizeExistentialTypeLayout

* Fix structured buffer layout for CPU.

* Remove testing/handling of global uniforms on CPU path.
Typo fix.
Changed CPU tests to use new CPU calling convention.

* Check globals are working. Initalize context to zero globals.

* Order the global parameters for C++ ouput by their layout.
Note - that layout isn't quite working correctly because the StructuredBuffer<int> the int seems to be consuming uniform space.

* Work around for reflection not having all data needed for layout ordering for C++ code.

* Output constant buffers as pointers.

* Entry point parameters accessed through pointer to struct.

* WIP: Layout for CPU is reasonable for test case.

* Only output 'f' after float literal if type marks as a float.

* Cast construction works on C++.

* Made IntrinsicOp::ConvertConstruct to make intent clearer.

* C++ handling construction from scalar.
Handle access of a scalar with .x.
Check default initialization.

* Comment about need for split of kIROp_construct.
Release build works.

* Added support from constructVectorFromScalar to C/C++ target.

* Handling of in/out in C/C++.

* First pass documentation CPU support.

* Improvements to C++/C slang code generation documentation.

* Small doc change to include need for mechansim to specify cpp compiler path.

* WIP: Being able to set path for backends.

* Better handling of swizzling - allow swizzling a scalar into a vector.

* Fix missing/broken headers for path setting on session.

* Fix for compiling using clang on Windows.

* Remove Clang test code.

* * Removed spSessionGetGlobalSession - no longer needed because SlangSession is slang::IGlobalSession alias.
* Gave Session a ref count on spCreateSession, and have it checked on spDestroySession, so behaves correctly as ISlangUnknown

Note that spDestroySession does a release (and checks the ref count on debug builds). It's behaviour could be the same as just release, but this seems closer to the original intention.

6 files changed, 171 insertions, 23 deletions

diff --git a/source/core/slang-cpp-compiler.cpp b/source/core/slang-cpp-compiler.cpp
index 7ac4e96b4..153018527 100644
--- a/source/core/slang-cpp-compiler.cpp
+++ b/source/core/slang-cpp-compiler.cpp
@@ -327,10 +327,14 @@ const CPPCompiler::Desc& CPPCompilerUtil::getCompiledWithDesc()
     return nullptr;
 }
 
-// Have to do this conditionally because unreferenced static functions are a warning on VC, and warnings are errors.
-#if !SLANG_WINDOWS_FAMILY
-static void _addGCCFamilyCompiler(const String& exeName, CPPCompilerSet* compilerSet)
+static void _addGCCFamilyCompiler(const String& path, const String& inExeName, CPPCompilerSet* compilerSet)
 {
+    String exeName(inExeName);
+    if (path.getLength() > 0)
+    {
+        exeName = Path::combine(path, inExeName);
+    }
+
     CPPCompiler::Desc desc;
     if (SLANG_SUCCEEDED(GCCCompilerUtil::calcVersion(exeName, desc)))
     {
@@ -339,7 +343,6 @@ static void _addGCCFamilyCompiler(const String& exeName, CPPCompilerSet* compile
         compilerSet->addCompiler(compiler);
     }
 }
-#endif
 
 /* static */CPPCompiler* CPPCompilerUtil::findClosestCompiler(const CPPCompilerSet* set, const CPPCompiler::Desc& desc)
 {
@@ -353,15 +356,15 @@ static void _addGCCFamilyCompiler(const String& exeName, CPPCompilerSet* compile
     return findClosestCompiler(compilers, desc);
 }
 
-/* static */SlangResult CPPCompilerUtil::initializeSet(CPPCompilerSet* set)
+/* static */SlangResult CPPCompilerUtil::initializeSet(const InitializeSetDesc& desc, CPPCompilerSet* set)
 {
 #if SLANG_WINDOWS_FAMILY
     WinVisualStudioUtil::find(set);
-#else
-    _addGCCFamilyCompiler("clang", set);
-    _addGCCFamilyCompiler("g++", set);
 #endif
 
+    _addGCCFamilyCompiler(desc.getPath(CompilerType::Clang), "clang", set);
+    _addGCCFamilyCompiler(desc.getPath(CompilerType::GCC), "g++", set);
+
     // Set the default to the compiler closest to how this source was compiled
     set->setDefaultCompiler(findClosestCompiler(set, getCompiledWithDesc()));
     return SLANG_OK;
diff --git a/source/core/slang-cpp-compiler.h b/source/core/slang-cpp-compiler.h
index 185e960f3..d87ed63dc 100644
--- a/source/core/slang-cpp-compiler.h
+++ b/source/core/slang-cpp-compiler.h
@@ -6,6 +6,8 @@
 
 #include "slang-process-util.h"
 
+#include "slang-platform.h"
+
 namespace Slang
 {
 
@@ -93,6 +95,7 @@ public:
             enum Enum : Flags
             {
                 EnableExceptionHandling = 0x01,
+                Verbose                 = 0x02, 
             };
         };
 
@@ -104,6 +107,8 @@ public:
 
         Flags flags = Flag::EnableExceptionHandling;
 
+        PlatformKind platform = PlatformKind::Unknown;
+
         String modulePath;      ///< The path/name of the output module. Should not have the extension, as that will be added for each of the target types
 
         List<Define> defines;
@@ -171,7 +176,7 @@ public:
     struct Output
     {
             /// Reset to an initial empty state
-        void reset() { messages.clear(); result = SLANG_OK; }
+        void reset() { messages.clear(); rawMessages = String(); result = SLANG_OK; }
         
             /// Get the number of messages by type
         Index getCountByType(OutputMessage::Type type) const;
@@ -189,6 +194,8 @@ public:
             /// Remove all messages of the type
         void removeByType(OutputMessage::Type type);
 
+        String rawMessages;
+
         SlangResult result;
         List<OutputMessage> messages;
     };
@@ -287,6 +294,7 @@ struct CPPCompilerBaseUtil
     typedef CPPCompiler::TargetType TargetType;
     typedef CPPCompiler::DebugInfoType DebugInfoType;
     typedef CPPCompiler::SourceType SourceType;
+    typedef CPPCompiler::CompilerType CompilerType;
 
     typedef CPPCompiler::OutputMessage OutputMessage;
     typedef CPPCompiler::FloatingPointMode FloatingPointMode;
@@ -303,6 +311,14 @@ struct CPPCompilerUtil: public CPPCompilerBaseUtil
         Newest,
     };
 
+    struct InitializeSetDesc
+    {
+        const String& getPath(CompilerType type) const { return paths[int(type)]; }
+        void setPath(CompilerType type, const String& path) { paths[int(type)] = path; }
+
+        String paths[int(CPPCompiler::CompilerType::CountOf)];
+    };
+
         /// Find a compiler
     static CPPCompiler* findCompiler(const CPPCompilerSet* set, MatchType matchType, const CPPCompiler::Desc& desc);
     static CPPCompiler* findCompiler(const List<CPPCompiler*>& compilers, MatchType matchType, const CPPCompiler::Desc& desc);
@@ -315,7 +331,7 @@ struct CPPCompilerUtil: public CPPCompilerBaseUtil
     static const CPPCompiler::Desc& getCompiledWithDesc();
 
         /// Given a set, registers compilers found through standard means and determines a reasonable default compiler if possible
-    static SlangResult initializeSet(CPPCompilerSet* set);
+    static SlangResult initializeSet(const InitializeSetDesc& desc, CPPCompilerSet* set);    
 };
 
 }
diff --git a/source/core/slang-gcc-compiler-util.cpp b/source/core/slang-gcc-compiler-util.cpp
index d42076039..c48e779d1 100644
--- a/source/core/slang-gcc-compiler-util.cpp
+++ b/source/core/slang-gcc-compiler-util.cpp
@@ -279,10 +279,13 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
     LineParseResult prevLineResult = LineParseResult::Ignore;
     
     outOutput.reset();
+    outOutput.rawMessages = exeRes.standardError;
 
     for (auto line : LineParser(exeRes.standardError.getUnownedSlice()))
     {
         CPPCompiler::OutputMessage msg;
+        msg.reset();
+
         LineParseResult lineRes;
         
         SLANG_RETURN_ON_FAIL(_parseGCCFamilyLine(line, lineRes, msg));
@@ -383,6 +386,8 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
 
 /* static */SlangResult GCCCompilerUtil::calcArgs(const CompileOptions& options, CommandLine& cmdLine)
 {
+    PlatformKind platformKind = (options.platform == PlatformKind::Unknown) ? PlatformUtil::getPlatformKind() : options.platform;
+    
     cmdLine.addArg("-fvisibility=hidden");
 
     if (options.sourceType == SourceType::CPP)
@@ -391,9 +396,6 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
         cmdLine.addArg("-std=c++14");
     }
 
-    // Use shared libraries
-    //cmdLine.addArg("-shared");
-
     switch (options.optimizationLevel)
     {
         case OptimizationLevel::None:
@@ -425,6 +427,11 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
         cmdLine.addArg("-g");
     }
 
+    if (options.flags & CompileOptions::Flag::Verbose)
+    {
+        cmdLine.addArg("-v");
+    }
+
     switch (options.floatingPointMode)
     {
         case FloatingPointMode::Default: break;
@@ -454,8 +461,12 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
         {
             // Shared library
             cmdLine.addArg("-shared");
-            // Position independent
-            cmdLine.addArg("-fPIC");
+
+            if (PlatformUtil::isFamily(PlatformFamily::Unix, platformKind))
+            {
+                // Position independent
+                cmdLine.addArg("-fPIC");
+            }
             break;
         }
         case TargetType::Executable:
@@ -500,13 +511,14 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
 
     if (options.targetType == TargetType::SharedLibrary)
     {
-#if !SLANG_APPLE_FAMILY
-        // On MacOS, this linker option is not supported. That's ok though in
-        // so far as on MacOS it does report any unfound symbols without the option.
+        if (!PlatformUtil::isFamily(PlatformFamily::Apple, platformKind))
+        {
+            // On MacOS, this linker option is not supported. That's ok though in
+            // so far as on MacOS it does report any unfound symbols without the option.
 
-        // Linker flag to report any undefined symbols as a link error
-        cmdLine.addArg("-Wl,--no-undefined");
-#endif
+            // Linker flag to report any undefined symbols as a link error
+            cmdLine.addArg("-Wl,--no-undefined");
+        }
     }
 
     // Files to compile
@@ -524,11 +536,11 @@ static SlangResult _parseGCCFamilyLine(const UnownedStringSlice& line, LineParse
         cmdLine.addArg(libPath);
     }
 
-    if (options.sourceType == SourceType::CPP)
+    if (options.sourceType == SourceType::CPP && !PlatformUtil::isFamily(PlatformFamily::Windows, platformKind))
     {
         // Make STD libs available
         cmdLine.addArg("-lstdc++");
-	// Make maths lib available
+	    // Make maths lib available
         cmdLine.addArg("-lm");
     }
 
diff --git a/source/core/slang-platform.cpp b/source/core/slang-platform.cpp
index 4f4c805ac..e735216f4 100644
--- a/source/core/slang-platform.cpp
+++ b/source/core/slang-platform.cpp
@@ -197,4 +197,52 @@ SLANG_COMPILE_TIME_ASSERT(E_OUTOFMEMORY == SLANG_E_OUT_OF_MEMORY);
 
 #endif // _WIN32
 
+/* static */PlatformKind PlatformUtil::getPlatformKind()
+{
+#if SLANG_WINRT
+    return PlatformKind::WinRT;
+#elif SLANG_XBOXONE
+    return PlatformKind::XBoxOne;
+#elif SLANG_WIN64
+    return PlatformKind::Win64;
+#elif SLANG_X360
+    return PlatformKind::X360;
+#elif SLANG_WIN32
+    return PlatformKind::Win32;
+#elif SLANG_ANDROID
+    return PlatformKind::Android;
+#elif SLANG_LINUX
+    return PlatformKind::Linux;
+#elif SLANG_IOS
+    return PlatformKind::IOS;
+#elif SLANG_OSX
+    return PlatformKind::OSX;
+#elif SLANG_PS3
+    return PlatformKind::PS3;
+#elif SLANG_SLANG_PS4
+    return PlatformKind::PS4;
+#elif SLANG_PSP2
+    return PlatformKind::PSP2;
+#elif SLANG_WIIU
+    return PlatformKind::WIIU;
+#else
+    return PlatformKind::Unknown;
+#endif
+}
+
+static const PlatformFlags s_familyFlags[int(PlatformFamily::CountOf)] =
+{
+    0,                                                                                                                  // Unknown
+    PlatformFlag::WinRT | PlatformFlag::Win32 | PlatformFlag::Win64,                                                    // Windows
+    PlatformFlag::WinRT | PlatformFlag::Win32 | PlatformFlag::Win64 | PlatformFlag::X360 | PlatformFlag::XBoxOne,       // Microsoft
+    PlatformFlag::Linux | PlatformFlag::Android,                                                                        // Linux
+    PlatformFlag::IOS | PlatformFlag::OSX,                                                                              // Apple
+    PlatformFlag::Linux | PlatformFlag::Android | PlatformFlag::IOS | PlatformFlag::OSX,                                // Unix
+};
+
+/* static */PlatformFlags PlatformUtil::getPlatformFlags(PlatformFamily family)
+{
+    return s_familyFlags[int(family)];
+}
+
 }
diff --git a/source/core/slang-platform.h b/source/core/slang-platform.h
index 0e6d12cb6..c3ad1c486 100644
--- a/source/core/slang-platform.h
+++ b/source/core/slang-platform.h
@@ -7,6 +7,58 @@
 
 namespace Slang
 {
+    enum class PlatformKind
+    {
+        Unknown,
+        WinRT,
+        XBoxOne,
+        Win64,
+        Win32,
+        X360,
+        Android,
+        Linux,
+        IOS,
+        OSX,
+        PS3,
+        PS4,
+        PSP2,
+        WIIU,
+        CountOf,
+    };
+
+    typedef uint32_t PlatformFlags;
+    struct PlatformFlag
+    {
+        enum Enum
+        {
+            Unknown = 1 << int(PlatformKind::Unknown),
+            WinRT = 1 << int(PlatformKind::WinRT),
+            XBoxOne = 1 << int(PlatformKind::XBoxOne),
+            Win64 = 1 << int(PlatformKind::Win64),
+            Win32 = 1 << int(PlatformKind::Win32),
+            X360 = 1 << int(PlatformKind::X360),
+            Android = 1 << int(PlatformKind::Android),
+            Linux = 1 << int(PlatformKind::Linux),
+            IOS = 1 << int(PlatformKind::IOS),
+            OSX = 1 << int(PlatformKind::OSX),
+            PS3 = 1 << int(PlatformKind::PS3),
+            PS4 = 1 << int(PlatformKind::PS4),
+            PSP2 = 1 << int(PlatformKind::PSP2),
+            WIIU = 1 << int(PlatformKind::WIIU),
+        };
+    };
+
+    enum class PlatformFamily
+    {
+        Unknown,
+        Windows,
+        Microsoft,
+        Linux,
+        Apple,
+        Unix,
+        CountOf,
+    };
+
 	// Interface for working with shared libraries
 	// in a platform-independent fashion.
 	struct SharedLibrary
@@ -66,6 +118,16 @@ namespace Slang
             /// @param builderOut Append the string produced to builderOut
             /// @return SLANG_OK if string is found and appended. Fail otherwise. SLANG_E_NOT_IMPLEMENTED if there is no impl for this platform.
         static SlangResult appendResult(SlangResult res, StringBuilder& builderOut);
+
+            /// Get the platform kind as determined at compile time
+        static PlatformKind getPlatformKind();
+
+            /// Get the platforms that make up a family
+        static PlatformFlags getPlatformFlags(PlatformFamily family);
+
+            /// True if the kind is part of the family
+        static bool isFamily(PlatformFamily family, PlatformKind kind) { return (getPlatformFlags(family) & (PlatformFlags(1) << int(kind))) != 0; }
+
     };
 
 #ifndef _MSC_VER
diff --git a/source/core/slang-visual-studio-compiler-util.cpp b/source/core/slang-visual-studio-compiler-util.cpp
index 1c79f9004..4b1e08026 100644
--- a/source/core/slang-visual-studio-compiler-util.cpp
+++ b/source/core/slang-visual-studio-compiler-util.cpp
@@ -90,6 +90,11 @@ namespace Slang
         }
     }
 
+    if (options.flags & CompileOptions::Flag::Verbose)
+    {
+        // Doesn't appear to be a VS equivalent
+    }
+
     switch (options.debugInfoType)
     {
         default:
@@ -378,6 +383,8 @@ static SlangResult _parseVisualStudioLine(const UnownedStringSlice& line, CPPCom
 {
     outOutput.reset();
 
+    outOutput.rawMessages = exeRes.standardOutput;
+
     for (auto line : LineParser(exeRes.standardOutput.getUnownedSlice()))
     {
 #if 0