"Shader Toy" example and related fixes (#1629)

* "Shader Toy" example and related fixes

This change introduces a new `shader-toy` example program that is primarily designed to show how Slang's features for type-based encapsulation and modularity can be applied to modularity for effects along the lines of those from `shadertoy.com`.

The Example
-----------

The example is being checked in with an example "toy" effect that I hastily put together, so that it would not be encumbered with any IP concerns. I wrote the effect using the shadertoy.com editor, so I can be sure it is valid GLSL. During bringup of the application I used a pre-existing and larger effect for testing, so some of the support code that was added is not being used at present.

The big-picture idea here is to have an exmaple that shows how to modularize things using Slang interfaces and generics, and then to use the Slang compiler API to manage the compilation, composition, specialization, and linking steps. For better or worse this leads to the sequence of API calls involved being much longer than what was in something like the `hello-world` example.

Future Work (Example)
---------------------

There is a lot of room for improvement and expansion here, so this should be viewed as a checkpoint of work in progress rather than something I'm claiming as a finalized demonstration of all we'd like to achieve. Areas for future work include:

* We need to copy the integration of "Dear, IMGUI" that was already done for the `model-viewer` example so that this example can have a UI.

* Now that the compilation flow is broken into all these additional steps, it should be possible to have the application load multiple effects as distinct modules, and then provide a UI for switching between them. The chosen effect module would be used to specialize the top-level shader(s) before kernel generation.

* The checked-in logic includes a compute shader that can execute an effect, but that hasn't been tested nor has it been wired up to any kind of UI. We should have a way to switch between multiple execution methods, with a goal of eventually including CPU execution.

* The "GLSL compatibility" code needs a lot of improvements before it is likely to be usable for a nontrivial number of shaders. Some of that work is waiting on Slang compiler fixes, though.

* We should consider allowing the individual "toy" effects to define their own uniform parameters and expose those via a UI and reflection. The catch in this case is not that this would be difficult to do, but that it would be a semantic change to how shader toy effects currently work.

The Compiler Fixes
------------------

Doing this work exposed a few bugs in Slang, and this change includes fixes for the ones that were quick to address.

We already had logic in `slang-check-shader.cpp` that was validating the entry points in a compile request - either by checking the explicitly-listed entry points, or by scanning for `[shader("...")]` attributes. The problem is that the routine that did that checking was not being invoked on all compiles. The logic that handled entry points was only being run for manual compiles using `SlangCompileRequest`, while anything using `import` or `loadModule` would ignore entry points. I refactored the relevant code into a subroutine that will be invoked in all compilation scenarios.

There were already `TODO` comments in `SpecializedComponentType` which made the point about how a specialized entry point like `myShader<YourType>` would need to properly show that it has dependencies on both the module that defines `myShader` *and* the module that defines `YourType`, while only the former was being handled at present. I went ahead and implemented the logic to scan the generic arguments for a specialized compoment type in order to determine what module(s) the arguments depend on (both type arguments and witness tables). With that change, using `IComponentType::link` on a specialized component will properly pull in the module(s) that the generic arguments come from.

In `slang-ir-legalize-types.cpp` we could run into assertion failures in debug builds because of code trying to legalize layout `IRAttr`s for fields or parameters with types that need legalization. In practice it is safe to skip these layout attributes, because legalization of the fields/parameters they pertain to would result in creation of entirely new layout attributes, and the old ones would then be unreferenced.

Future Work (Fixes)
-------------------

There are other compiler bugs that this work exposed, but which this change does not address. These will need to be resolved as part of subsequent changes:

* Slang allows for default-initialization of variables of a generic type. That is, given `<T : ISomething>` a user is allowed to declare `T x = {};` and the Slang front-end does not complain. Instead, this leads to an internal compiler error during IR lowering.

* The Slang `__init()` feature probably needs to be upgraded to a properly supported feature, and we probably need a way to make implementing default-initialization an easy thing (e.g., any `struct` type that has initial-value expressions for all its fields should automatically and implicitly satsify an `init();` requirement declared in an interface)

* Iniside an `__init()` definition, code has mutable access to members of the enclosing type, but for some reason the front-end is incorrectly treating `this` as immutable in those contexts. As a result you can write to `someField` but not `this.someField`.

* User-defined operator overloads flat out don't work (which isn't surprising given that no clients have decided to use them yet, and we have no test coverage for them). This is actually due to the shadowing rules being used for lookup right now, so a fix for this issue is going to have far-reaching consequences around what overloads are visible where (and anything that impacts overload resolution is a big can of worms, including around performance).

* fixup: test case had missing main function

1 files changed, 283 insertions, 12 deletions

diff --git a/source/slang/slang.cpp b/source/slang/slang.cpp
index 254a37c24..4a25c2392 100644
--- a/source/slang/slang.cpp
+++ b/source/slang/slang.cpp
@@ -1552,6 +1552,7 @@ void FrontEndCompileRequest::checkAllTranslationUnits()
     {
         checkTranslationUnit(translationUnit.Ptr());
     }
+    checkEntryPoints();
 }
 
 void FrontEndCompileRequest::generateIR()
@@ -1676,7 +1677,6 @@ SlangResult FrontEndCompileRequest::executeActionsInner()
     if (getSink()->getErrorCount() != 0)
         return SLANG_FAIL;
 
-
     // Look up all the entry points that are expected,
     // and use them to populate the `program` member.
     //
@@ -1905,13 +1905,11 @@ SlangResult EndToEndCompileRequest::executeActions()
 
 int FrontEndCompileRequest::addTranslationUnit(SourceLanguage language, Name* moduleName)
 {
-    Index result = translationUnits.getCount();
-
     if (!moduleName)
     {
         // We want to ensure that symbols defined in different translation
         // units get unique mangled names, so that we can, e.g., tell apart
-        // a `main()` function in `vertex.slang` and a `main()` in `fragment.slang`,
+        // a `main()` function in `vertex.hlsl` and a `main()` in `fragment.hlsl`,
         // even when they are being compiled together.
         //
         String generatedName = "tu";
@@ -1925,11 +1923,17 @@ int FrontEndCompileRequest::addTranslationUnit(SourceLanguage language, Name* mo
 
     translationUnit->moduleName = moduleName;
 
-    translationUnits.add(translationUnit);
+    return addTranslationUnit(translationUnit);
+}
 
+int FrontEndCompileRequest::addTranslationUnit(TranslationUnitRequest* translationUnit)
+{
+    Index result = translationUnits.getCount();
+    translationUnits.add(translationUnit);
     return (int) result;
 }
 
+
 void FrontEndCompileRequest::addTranslationUnitSourceFile(
     int             translationUnitIndex,
     SourceFile*     sourceFile)
@@ -2041,6 +2045,7 @@ UInt Linkage::addTarget(
 }
 
 void Linkage::loadParsedModule(
+    RefPtr<FrontEndCompileRequest>  compileRequest,
     RefPtr<TranslationUnitRequest>  translationUnit,
     Name*                           name,
     const PathInfo&                 pathInfo)
@@ -2061,7 +2066,7 @@ void Linkage::loadParsedModule(
     auto sink = translationUnit->compileRequest->getSink();
 
     int errorCountBefore = sink->getErrorCount();
-    checkTranslationUnit(translationUnit.Ptr());
+    compileRequest->checkAllTranslationUnits();
     int errorCountAfter = sink->getErrorCount();
 
     if (errorCountAfter != errorCountBefore)
@@ -2106,6 +2111,8 @@ RefPtr<Module> Linkage::loadModule(
     translationUnit->moduleName = name;
     translationUnit->sourceLanguage = SourceLanguage::Slang;
 
+    frontEndReq->addTranslationUnit(translationUnit);
+
     auto module = translationUnit->getModule();
 
     ModuleBeingImportedRAII moduleBeingImported(
@@ -2132,6 +2139,7 @@ RefPtr<Module> Linkage::loadModule(
     }
 
     loadParsedModule(
+        frontEndReq,
         translationUnit,
         name,
         filePathInfo);
@@ -2902,6 +2910,134 @@ RefPtr<ComponentType::SpecializationInfo> CompositeComponentType::_validateSpeci
 // SpecializedComponentType
 //
 
+/// Utility type for collecting modules references by types/declarations
+struct SpecializationArgModuleCollector : ComponentTypeVisitor
+{
+    HashSet<Module*> m_modulesSet;
+    List<Module*> m_modulesList;
+
+    void addModule(Module* module)
+    {
+        m_modulesList.add(module);
+        m_modulesSet.Add(module);
+    }
+
+    void maybeAddModule(Module* module)
+    {
+        if(!module)
+            return;
+        if(m_modulesSet.Contains(module))
+            return;
+
+        addModule(module);
+    }
+
+    void collectReferencedModules(Decl* decl)
+    {
+        auto module = getModule(decl);
+        maybeAddModule(module);
+    }
+
+    void collectReferencedModules(Substitutions* substitution)
+    {
+        if(auto genericSubst = as<GenericSubstitution>(substitution))
+        {
+            for(auto arg : genericSubst->args)
+            {
+                collectReferencedModules(arg);
+            }
+        }
+    }
+
+    void collectReferencedModules(SubstitutionSet const& substitutions)
+    {
+        for(auto subst = substitutions.substitutions; subst; subst = subst->outer)
+        {
+            collectReferencedModules(subst);
+        }
+    }
+
+    void collectReferencedModules(DeclRefBase const& declRef)
+    {
+        collectReferencedModules(declRef.decl);
+        collectReferencedModules(declRef.substitutions);
+    }
+
+    void collectReferencedModules(Type* type)
+    {
+        if(auto declRefType = as<DeclRefType>(type))
+        {
+            collectReferencedModules(declRefType->declRef);
+        }
+
+        // TODO: Handle non-decl-ref composite type cases
+        // (e.g., function types).
+    }
+
+    void collectReferencedModules(Val* val)
+    {
+        if(auto type = as<Type>(val))
+        {
+            collectReferencedModules(type);
+        }
+        else if (auto declRefVal = as<GenericParamIntVal>(val))
+        {
+            collectReferencedModules(declRefVal->declRef);
+        }
+
+        // TODO: other cases of values that could reference
+        // a declaration.
+    }
+
+    void collectReferencedModules(List<ExpandedSpecializationArg> const& args)
+    {
+        for(auto arg : args)
+        {
+            collectReferencedModules(arg.val);
+            collectReferencedModules(arg.witness);
+        }
+    }
+
+    //
+    // ComponentTypeVisitor methods
+    //
+
+    void visitEntryPoint(EntryPoint* entryPoint, EntryPoint::EntryPointSpecializationInfo* specializationInfo) SLANG_OVERRIDE
+    {
+        SLANG_UNUSED(entryPoint);
+
+        if(!specializationInfo)
+            return;
+
+        collectReferencedModules(specializationInfo->specializedFuncDeclRef);
+        collectReferencedModules(specializationInfo->existentialSpecializationArgs);
+    }
+
+    void visitModule(Module* module, Module::ModuleSpecializationInfo* specializationInfo) SLANG_OVERRIDE
+    {
+        SLANG_UNUSED(module);
+
+        if(!specializationInfo)
+            return;
+
+        for(auto arg : specializationInfo->genericArgs)
+        {
+            collectReferencedModules(arg.argVal);
+        }
+        collectReferencedModules(specializationInfo->existentialArgs);
+    }
+
+    void visitComposite(CompositeComponentType* composite, CompositeComponentType::CompositeSpecializationInfo* specializationInfo) SLANG_OVERRIDE
+    {
+        visitChildren(composite, specializationInfo);
+    }
+
+    void visitSpecialized(SpecializedComponentType* specialized) SLANG_OVERRIDE
+    {
+        visitChildren(specialized);
+    }
+};
+
 SpecializedComponentType::SpecializedComponentType(
     ComponentType*                      base,
     ComponentType::SpecializationInfo*  specializationInfo,
@@ -2914,8 +3050,147 @@ SpecializedComponentType::SpecializedComponentType(
 {
     m_irModule = generateIRForSpecializedComponentType(this, sink);
 
+    // We need to account for the fact that a specialized
+    // entity like `myShader<SomeType>` needs to not only
+    // depend on the module(s) that `myShader` depends on,
+    // but also on any modules that `SomeType` depends on.
+    //
+    // We will set up a "collector" type that will be
+    // used to build a list of these additional modules.
+    //
+    SpecializationArgModuleCollector moduleCollector;
+
+    // We don't want to go adding additional requirements for
+    // modules that the base component type already includes,
+    // so we will add those to the set of modules in
+    // the collector before we starting trying to add others.
+    //
+    base->enumerateModules([&](Module* module)
+    {
+        moduleCollector.m_modulesSet.Add(module);
+    });
+
+    // In order to collect the additional modules, we need
+    // to inspect the specialization arguments and see what
+    // they depend on.
+    //
+    // Naively, it seems like we'd just want to iterate
+    // over `specializationArgs`, which gives the specialization
+    // arguments as the user supplied them. However, such
+    // an approach would have a subtle problem.
+    //
+    // If we have a generic entry point like:
+    //
+    //      // In module A
+    //      myShader<T : IThing>
+    //
+    //
+    // And the type `SomeType` that is being used as an argument doesn't
+    // directly conform to `IThing`:
+    //
+    //      // In module B
+    //      struct SomeType { ... }
+    //
+    // and the conformance of `SomeType` to `IThing` is
+    // coming from yet another module:
+    //
+    //      // In module C
+    //      import B;
+    //      extension SomeType : IThing { ... }
+    //
+    // In this case, the specialized component for `myShader<SomeType>`
+    // needs to depend on all of:
+    //
+    // * Module A, because it defines `myShader`
+    // * Module B, because it defines `SomeType`
+    // * Module C, because it defines the conformance `SomeType : IThing`
+    //
+    // We thus need to iterate over a form of the specialization
+    // arguments that includes the "expanded" arguments like
+    // interface conformance witnesses that got added during
+    // semantic checking.
+    //
+    // The expanded arguments are being stored in the `specializationInfo`
+    // today (for use by downstream code generation), and the easiest
+    // way to walk that information and get to the leaf nodes where
+    // the expanded arguments are stored is to apply a visitor to
+    // the specialized component type we are in the middle of constructing.
+    //
+    moduleCollector.visitSpecialized(this);
+
+    // Now that we've collected our additional information, we can
+    // start to build up the final lists for the specialized component type.
+    //
+    // The starting point for our lists comes from the base component type.
+    //
+    m_moduleDependencies = base->getModuleDependencies();
+    m_filePathDependencies = base->getFilePathDependencies();
+
+    Index baseRequirementCount = base->getRequirementCount();
+    for( Index r = 0; r < baseRequirementCount; r++ )
+    {
+        m_requirements.add(base->getRequirement(r));
+    }
+
+    // The specialized component type will need to have additional
+    // dependencies and requirements based on the modules that
+    // were collected when looking at the specialization arguments.
+
+    // We want to avoid adding the same file path dependency more than once.
+    //
+    HashSet<String> filePathDependencySet;
+    for(auto path : m_filePathDependencies)
+        filePathDependencySet.Add(path);
+
+    for(auto module : moduleCollector.m_modulesList)
+    {
+        // The specialized component type will have an open (unsatisfied)
+        // requirement for each of the modules that its specialization
+        // arguments need.
+        //
+        // Note: what this means in practice is that the component type
+        // records that the given module(s) will need to be linked in
+        // before final code can be generated, but it importantly
+        // does not dictate the final placement of the parameters from
+        // those modules in the layout.
+        //
+        m_requirements.add(module);
+
+        // The speciialized component type will also have a dependency
+        // on all the file paths that any of the modules involved in
+        // it depend on (including those that are required but not
+        // yet linked in).
+        //
+        // The file path information is what a client would need to
+        // use to decide if kernel code is out of date compared to
+        // source files, so we want to include anything that could
+        // affect the validity of generated code.
+        //
+        for(auto path : module->getFilePathDependencies())
+        {
+            if(filePathDependencySet.Contains(path))
+                continue;
+            filePathDependencySet.Add(path);
+            m_filePathDependencies.add(path);
+        }
+
+        // Finalyl we also add the module for the specialization arguments
+        // to the list of modules that would be used for legacy lookup
+        // operations where we need an implicit/default scope to use
+        // and want it to be expansive.
+        //
+        // TODO: This stuff really isn't worth keeping around long
+        // term, and we should ditch the entire "legacy lookup" idea.
+        //
+        m_moduleDependencies.add(module);
+    }
+
     // The following is a bit of a hack.
     //
+    // TODO: We should not need this hack any longer, since the
+    // new approach to `switch`-based dynamic dispatch has made
+    // the existing tagged-union support obsolete.
+    //
     // Back-end code generation relies on us having computed layouts for all tagged
     // unions that end up being used in the code, which means we need a way to find
     // all such types that get used in a program (and the stuff it imports).
@@ -3001,16 +3276,12 @@ void SpecializedComponentType::acceptVisitor(ComponentTypeVisitor* visitor, Spec
 
 Index SpecializedComponentType::getRequirementCount()
 {
-    // TODO: A specialized component type may have *more* requirements
-    // than the original, because it also needs to include the module(s)
-    // that define the types used for specialization arguments.
-
-    return m_base->getRequirementCount();
+    return m_requirements.getCount();
 }
 
 RefPtr<ComponentType> SpecializedComponentType::getRequirement(Index index)
 {
-    return m_base->getRequirement(index);
+    return m_requirements[index];
 }
 
 String SpecializedComponentType::getEntryPointMangledName(Index index)