Generate IR per-module for loaded modules (#299)

The basic idea here is that for each module that gets loaded via `import`, we should also generate the initial IR for the declarations in that module at the time it gets loaded.

Furthermore, when we generate initial IR for a module, we will only generate IR *declarations* (not *definitions*) for any functions/variables in modules it imports.
Later, when cloning IR to begin code generation for an entry point, we will effectively "link" all of the loadedm modules together, so that a given global value can get its definition from any of the IR modules present.

- Change the `loadedModulesList` and related data structures to hold a new `LoadedModule` type, instead of just the AST (and then have a `LoadedModule` own both the AST and the IR module)

- Share some logic between the `import` and `#import` cases, so that we always try to generate IR for modules we load.

- Make sure that IR generation always gets skipped if the command-line flags tell us not to use the IR.

- A few small fixups for cases that didn't arise in IR lowering so far, but come up when we try to actually generate IR for things like the stdlib.

There are some notable gaps in this work right now:

- The stdlib modules are exempted from this behavior; we always generate IR for stdlib functions in any user module that calls them. This is just a workaround for the fact that the stdlib modules don't show up in the list of imported modules right now.

- We don't currently have logic that does the "linking" step for global variables like we do for functions. We really need to look up the symbols with the same mangled name, and favor any one of them that has a definition (if there is one)

- Similarly, the handling of witness tables is incomplete. During initial IR generation, we should probably be generating empty witness tables for any conformances that were declared in other modules (but are being used locally in this module), and then the "linking" step should favor non-empty witness tables over empty ones.

Still, all the test cases pass with the code like this, and this seems like an important step in the right direction.

1 files changed, 87 insertions, 11 deletions

diff --git a/source/slang/lower-to-ir.cpp b/source/slang/lower-to-ir.cpp
index c01059c61..5ec175668 100644
--- a/source/slang/lower-to-ir.cpp
+++ b/source/slang/lower-to-ir.cpp
@@ -269,6 +269,7 @@ LoweredValInfo LoweredValInfo::swizzledLValue(
 struct SharedIRGenContext
 {
     CompileRequest* compileRequest;
+    ModuleDecl*     mainModuleDecl;
 
     Dictionary<Decl*, LoweredValInfo> declValues;
 
@@ -2534,6 +2535,11 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
         SLANG_UNIMPLEMENTED_X("decl catch-all");
     }
 
+    LoweredValInfo visitEmptyDecl(EmptyDecl* /*decl*/)
+    {
+        return LoweredValInfo();
+    }
+
     LoweredValInfo visitTypeDefDecl(TypeDefDecl * decl)
     {
         return LoweredValInfo::simple(context->irBuilder->getTypeVal(decl->type.type));
@@ -2691,7 +2697,12 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
         context->shared->declValues[
             DeclRef<VarDeclBase>(decl, nullptr)] = globalVal;
 
-        if( auto initExpr = decl->initExpr )
+        if (isImportedDecl(decl))
+        {
+            // Always emit imported declarations as declarations,
+            // and not definitions.
+        }
+        else if( auto initExpr = decl->initExpr )
         {
             IRBuilder subBuilderStorage = *getBuilder();
             IRBuilder* subBuilder = &subBuilderStorage;
@@ -3131,6 +3142,49 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
         irFunc->mangledName = mangledName;
     }
 
+    ModuleDecl* findModuleDecl(Decl* decl)
+    {
+        for (auto dd = decl; dd; dd = dd->ParentDecl)
+        {
+            if (auto moduleDecl = dynamic_cast<ModuleDecl*>(dd))
+                return moduleDecl;
+        }
+        return nullptr;
+    }
+
+    bool isFromStdLib(Decl* decl)
+    {
+        for (auto dd = decl; dd; dd = dd->ParentDecl)
+        {
+            if (dd->HasModifier<FromStdLibModifier>())
+                return true;
+        }
+        return false;
+    }
+
+    bool isImportedDecl(Decl* decl)
+    {
+        ModuleDecl* moduleDecl = findModuleDecl(decl);
+        if (!moduleDecl)
+            return false;
+
+        // HACK: don't treat standard library code as
+        // being imported for right now, just because
+        // we don't load its IR in the same way as
+        // for other imports.
+        //
+        // TODO: Fix this the right way, by having standard
+        // library declarations have IR modules that we link
+        // in via the normal means.
+        if (isFromStdLib(decl))
+            return false;
+
+        if (moduleDecl != this->context->shared->mainModuleDecl)
+            return true;
+
+        return false;
+    }
+
     LoweredValInfo lowerFuncDecl(FunctionDeclBase* decl)
     {
         // Collect the parameter lists we will use for our new function.
@@ -3248,18 +3302,17 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
             irResultType);
         irFunc->type = irFuncType;
 
-        if (!decl->Body)
+        if (isImportedDecl(decl))
+        {
+            // Always emit imported declarations as declarations,
+            // and not definitions.
+        }
+        else if (!decl->Body)
         {
             // This is a function declaration without a body.
             // In Slang we currently try not to support forward declarations
-            // (although we might have to give in eventually), so the
-            // only case where this arises is for a function that
-            // needs to be imported from another module.
-
-            // TODO: we may need to attach something to the declaration,
-            // so that later passes don't get confused by it not having
-            // a body.
-
+            // (although we might have to give in eventually), so
+            // this case should really only occur for builtin declarations.
         }
         else
         {
@@ -3594,6 +3647,10 @@ LoweredValInfo maybeEmitSpecializeInst(IRGenContext*   context,
     if (!hasGenericSubstitutions(declRef.substitutions))
         return loweredDecl;
 
+    // There's no reason to specialize something that maps to a NULL pointer.
+    if (loweredDecl.flavor == LoweredValInfo::Flavor::None)
+        return loweredDecl;
+
     auto val = getSimpleVal(context, loweredDecl);
 
     // We have the "raw" substitutions from the AST, but we may
@@ -3635,7 +3692,7 @@ static void lowerEntryPointToIR(
     // we need to lower all global type arguments as well
     for (auto arg : entryPointRequest->genericParameterTypes)
         lowerType(context, arg);
-    auto loweredEntryPointFunc = lowerDecl(context, entryPointFuncDecl);
+    auto loweredEntryPointFunc = ensureDecl(context, entryPointFuncDecl);
 }
 
 #if 0
@@ -3682,12 +3739,18 @@ IRModule* lowerEntryPointToIR(
 IRModule* generateIRForTranslationUnit(
     TranslationUnitRequest* translationUnit)
 {
+    // If the user did not opt into IR usage, then don't compile IR
+    // for the translation unit.
+    if (!(translationUnit->compileFlags & SLANG_COMPILE_FLAG_USE_IR))
+        return nullptr;
+
     auto compileRequest = translationUnit->compileRequest;
 
     SharedIRGenContext sharedContextStorage;
     SharedIRGenContext* sharedContext = &sharedContextStorage;
 
     sharedContext->compileRequest = compileRequest;
+    sharedContext->mainModuleDecl = translationUnit->SyntaxNode;
 
     IRGenContext contextStorage;
     IRGenContext* context = &contextStorage;
@@ -3710,10 +3773,23 @@ IRModule* generateIRForTranslationUnit(
 
     // We need to emit IR for all public/exported symbols
     // in the translation unit.
+    //
+    // For now, we will assume that *all* global-scope declarations
+    // represent public/exported symbols.
+
+    // First, ensure that all entry points have been emitted,
+    // in case they require special handling.
     for (auto entryPoint : translationUnit->entryPoints)
     {
         lowerEntryPointToIR(context, entryPoint);
     }
+    //
+    // Next, ensure that all other global declarations have
+    // been emitted.
+    for (auto decl : translationUnit->SyntaxNode->Members)
+    {
+        ensureDecl(context, decl);
+    }
 
     // If we are being sked to dump IR during compilation,
     // then we can dump the initial IR for the module here.