Search for multiple NVRTC versions (#1543)

* Search for multiple NVRTC versions

The main change here is that when locating the NVRTC compiler we try multiple library names and take the first one that loads successfully (with an ordering that means we try newer versions before older ones).

In order to support this change, I needed to fix the wrapping logic that invokes the downstream compiler "locator" function, so that it does not report every failed dynamic library load as an error diagnostic (leading to compilation failure), but instead only reports such failures once the locator has reported failure.

The form of the diagnostic output for failures is also changed, in that we now report a single umbrella error about failing to load a downstream compiler, and then report the actuall dynamic library load failures as notes on that diagnostic instead of errors of their own. This choice seems appropriate since for cases like NVRTC it is *not* the case that each failed library load is a compilation error. We only need one of the listed libraries to be loadable, so that reporting them all as errors risks confusing users.

One wrinkle that arose during testing is that the 11.0 release of NVRTC dropped support for the `compute_30` target, which had previously been the minimum and default. I had to add logic to check for versions of 11 or greater and switch to `compute_35` as the default. Similar changes may be required as part of supporting newer NVRTC versions if support for more architectures gets deprecated and removed.

A more complete implementation of this logic might try to load multiple NVRTC versions such that the Slang compiler can identify a suitable compiler based on the minimum feature level that code actually requires. That kind of cleanup is left as future work, since for most users the current approach will be sufficient.

* testing: use verbose mode for running tests by default

* fixup: guard against null diagnostic sink

5 files changed, 114 insertions, 6 deletions

diff --git a/source/core/slang-nvrtc-compiler.cpp b/source/core/slang-nvrtc-compiler.cpp
index 6b01ef070..a5720ddc8 100644
--- a/source/core/slang-nvrtc-compiler.cpp
+++ b/source/core/slang-nvrtc-compiler.cpp
@@ -311,8 +311,24 @@ SlangResult NVRTCDownstreamCompiler::compile(const CompileOptions& options, RefP
     }
 
     {
-        // Lowest supported is 3.0
+        // The lowest supported CUDA architecture version supported
+        // by NVRTC is `compute_30`.
+        //
         SemanticVersion version(3);
+
+        // Newer releases of NVRTC only support `compute_35` and up
+        // (with everything before `compute_52` being deprecated).
+        //
+        if( m_desc.majorVersion >= 11 )
+        {
+            version = SemanticVersion(3, 5);
+        }
+
+        // If constructs used in the code to be compield require
+        // a higher architecture version than the minimum, then
+        // we will set the version to the highest version listed
+        // among the requirements.
+        //
         for (const auto& capabilityVersion : options.requiredCapabilityVersions)
         {
             if (capabilityVersion.kind == DownstreamCompiler::CapabilityVersion::Kind::CUDASM)
@@ -478,8 +494,55 @@ SlangResult NVRTCDownstreamCompiler::compile(const CompileOptions& options, RefP
     }
     else
     {
-        const char* libraryName = "nvrtc64_102_0";
-        SLANG_RETURN_ON_FAIL(loader->loadSharedLibrary(libraryName, library.writeRef()));
+        // If the user doesn't supply a path to their preferred version of NVRTC,
+        // we will search for a suitable library version, proceeding from more
+        // recent versions to less recent ones.
+        //
+        // TODO: The list here was cobbled together from what NRTC releases I
+        // could easily identify. It would be good to ver this against some
+        // kind of official list.
+        //
+        // It would probably be good to support 32- and 64-bit here, and also
+        // to deal with any variation in the shared library name across platforms
+        // (
+        //
+        static const char* kNVRTCLibraryNames[]
+        {
+            // As a catch-all for non-Windows platforms, we search for
+            // a library simply named `nvrtc` (well, `libnvrtc`) which
+            // is expected to match whatever the user has installed.
+            //
+            "nvrtc",
+
+            "nvrtc64_110_0",
+            "nvrtc64_102_0",
+            "nvrtc64_101_0",
+            "nvrtc64_100_0",
+            "nvrtc64_92",
+            "nvrtc64_91",
+            "nvrtc64_90",
+            "nvrtc64_80",
+            "nvrtc64_75",
+        };
+
+        SlangResult result = SLANG_FAIL;
+        for( auto libraryName : kNVRTCLibraryNames )
+        {
+            // If we succeed at loading one of the library versions
+            // from our list, we will not continue to search; this
+            // approach assumes that the `kNVRTCLibraryNames` array
+            // has been sorted so that earlier entries are preferable.
+            //
+            result = loader->loadSharedLibrary(libraryName, library.writeRef());
+            if(!SLANG_FAILED(result))
+                break;
+        }
+
+        // If we tried to load all of the candidate versions and none
+        // was successful, then we report back a failure.
+        //
+        if(SLANG_FAILED(result))
+            return result;
     }
 
     RefPtr<NVRTCDownstreamCompiler> compiler(new NVRTCDownstreamCompiler);
diff --git a/source/slang/slang-check.cpp b/source/slang/slang-check.cpp
index e97cb0b61..bf004817b 100644
--- a/source/slang/slang-check.cpp
+++ b/source/slang/slang-check.cpp
@@ -42,7 +42,7 @@ namespace Slang
                 }
                 else
                 {
-                    m_sink->diagnose(SourceLoc(), Diagnostics::failedToLoadDynamicLibrary, path);
+                    m_sink->diagnose(SourceLoc(), Diagnostics::noteFailedToLoadDynamicLibrary, path);
                 }
             }
             return res;
@@ -133,8 +133,37 @@ namespace Slang
         {
             m_downstreamCompilerSet->remove(SlangPassThrough(type));
 
-            SinkSharedLibraryLoader loader(m_sharedLibraryLoader, sink);
-            locator(m_downstreamCompilerPaths[int(type)], &loader, m_downstreamCompilerSet);
+            // We want to be able to report a diagnostic to the user if a loader
+            // was unable to locate the desired downstream compiler, but we
+            // also need to deal with the fact that the locator might "probe"
+            // multiple possible library versions/names, and failing to load
+            // one library should not be taken as a hard error.
+            //
+            // The approach we use here is to first apply the `locator` directly
+            // with our `m_sharedLibraryLoader` and see if it succeeds. If
+            // it does, then we will move along.
+            //
+            if (SLANG_FAILED(locator(m_downstreamCompilerPaths[int(type)], m_sharedLibraryLoader, m_downstreamCompilerSet)))
+            {
+                // If the locator reported a failure the first time we invoked
+                // it, then we will invoke it against with a wrapper shared librar
+                // loader that reported library load failures to our diagnost `sink`.
+                //
+                // This means that in the case of failure the user will see a listing
+                // of all the libraries that the locator attempted to load but failed
+                // to find. The user will know that making one or more of these libraries
+                // available could fix the issue, but we cannot communicate precise
+                // information to them with this approach (e.g., the difference between
+                // "I need all of these libraries" vs. "I need at least one of these
+                // libraries").
+                //
+                if( sink )
+                {
+                    sink->diagnose(SourceLoc(), Diagnostics::failedToLoadDownstreamCompiler, type);
+                }
+                SinkSharedLibraryLoader loader(m_sharedLibraryLoader, sink);
+                locator(m_downstreamCompilerPaths[int(type)], &loader, m_downstreamCompilerSet);
+            }
 
             DownstreamCompilerUtil::updateDefaults(m_downstreamCompilerSet);
         }
diff --git a/source/slang/slang-compiler.cpp b/source/slang/slang-compiler.cpp
index 71582f60d..37a79e6c4 100755
--- a/source/slang/slang-compiler.cpp
+++ b/source/slang/slang-compiler.cpp
@@ -105,6 +105,12 @@ namespace Slang
         }
     }
 
+    void printDiagnosticArg(StringBuilder& sb, PassThroughMode val)
+    {
+        sb << TypeTextUtil::getPassThroughName(SlangPassThrough(val));
+    }
+
+
     // !!!!!!!!!!!!!!!!!!!!!!!!!!!! CompileResult !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 
     SlangResult CompileResult::getSharedLibrary(ComPtr<ISlangSharedLibrary>& outSharedLibrary)
diff --git a/source/slang/slang-compiler.h b/source/slang/slang-compiler.h
index 4ced85ec8..cbd62d2bc 100755
--- a/source/slang/slang-compiler.h
+++ b/source/slang/slang-compiler.h
@@ -845,6 +845,7 @@ namespace Slang
         NVRTC = SLANG_PASS_THROUGH_NVRTC,
         CountOf = SLANG_PASS_THROUGH_COUNT_OF,              
     };
+    void printDiagnosticArg(StringBuilder& sb, PassThroughMode val);
 
     class SourceFile;
 
diff --git a/source/slang/slang-diagnostic-defs.h b/source/slang/slang-diagnostic-defs.h
index c268fd241..044ce0830 100644
--- a/source/slang/slang-diagnostic-defs.h
+++ b/source/slang/slang-diagnostic-defs.h
@@ -118,6 +118,15 @@ DIAGNOSTIC(    86, Error, unableToCreateModuleContainer, "unable to create modul
 
 DIAGNOSTIC(    87, Error, unableToSetDefaultDownstreamCompiler, "unable to set default downstream compiler for source language '%0' to '%1'")
 
+
+//
+// 001xx - Downstream Compilers
+//
+
+DIAGNOSTIC(  100, Error, failedToLoadDownstreamCompiler, "failed to load downstream compiler '$0'")
+DIAGNOSTIC(99999, Note, noteFailedToLoadDynamicLibrary, "failed to load dynamic library '$0'")
+
+
 //
 // 1xxxx - Lexical analysis
 //