Shader caching (#2432)

* Changed all getEntryPointCode calls to use RendererBase::getEntryPointCodeFromShaderCache

* Hashing hooked up, tests pass but need to add more to fully test functionality

* checkpoint

* Checkpoint: File system creation seems functional, saving is broken

* checkpoint: Fixed filename generation from MD5 hash, shader blob might be going missing ahead of pipeline state creation

* Fixed a lot of bugs related to hash code generation, shader cache is likely working but needs further testing

* Added workaround for module loading by re-creating the test device, shader cache test functional

* Vulkan shader caching bug fixed, checkpoint commit before more refinement

* pre-ToT merge checkpoint

* checkpoint commit, improving cache keys

* Significantly expanded items included in the dependency hash for Module; Added dependency hash functions to SpecializedComponentType and RenamedEntryPointComponentType

* Temporarily disable shader cache test

* Mid cleanup changes, solution successfully builds

* Added several helper update functions to slang-md5 to help simplify usage; Added a function under ISession to compute a hash for all linkage-related items; Function renames and cleaned up some comments

* Ran premake.bat; Renamed getASTBasedHashCode to computeASTBasedHash

* Added slang unit tests for Checksum and MD5; Extended gfx shader cache test to test with multiple shader files and one shader file with multiple entry points

* Solution builds and shader cache tests pass, but at least a couple other tests now failing

* ran premake.bat

* More cleanup changes

* Added shaderCachePath field to IDevice desc in gfx.slang, gfx-smoke.slang should be functional

* ran premake

* cleanup changes; Adding test printf to getEntryPointCodeFromShaderCache to see if output can be seen in CI

* Removed debugging printfs; Added handling for getEntryPointCode() failing

* Cleanup changes; Jonathan's fixes to SerialWriter to zero initialize otherwise uninitialized memory; Change to SwizzleExpr creation to zero initialize elementCount

* Changed enable_if_t to enable_if

* Fixed enable_if

* Added test for import vs include and changes to included and imported files; Fixed build errors in CUDA; Renamed shader cache statistics fields

* cleanup changes

* Readd removed file

* Restructured computeDependencyBasedHash calls, added computeDependencyBasedHashImpl to all classes dervied from ComponentType

* Applied same restructuring to the AST hash functions

* Cleanup changes; Moved HashBuilder out to slang-digest.h and added some helper functions to streamline the process of adding items to a hash

* Cleanup; Fixed incorrect expected results for shader import and include test

1 files changed, 38 insertions, 0 deletions

diff --git a/slang.h b/slang.h
index 42825f5c4..08279a1de 100644
--- a/slang.h
+++ b/slang.h
@@ -4134,6 +4134,13 @@ namespace slang
         None, UnsizedArray, StructuredBuffer, ConstantBuffer, ParameterBlock
     };
 
+    // A struct storing a single hash represented as a four element uint32_t array.
+    // This is intended to be used with the current MD5 hashing implementation.
+    struct Digest
+    {
+        uint32_t values[4] = { 0 };
+    };
+
         /** A session provides a scope for code that is loaded.
 
         A session can be used to load modules of Slang source code,
@@ -4421,6 +4428,37 @@ namespace slang
             IBlob**     outCode,
             IBlob**     outDiagnostics = nullptr) = 0;
 
+            /** Compute the hash code of all dependencies for this component type. This generally means file path
+                dependencies but can also include the component's name or sub-components. The dependency-based
+                hash effectively represents all the files that may be included/imported by a component type along with
+                any non-code-specific that helps define a component. This can be useful to simply check for a component
+                type without needing to inspect the code. For example, a shader cache might key its entries using the
+                dependency-based hash in order to determine at a glance if a particular shader is present, with no
+                regard for the shader's contents.
+
+                This function should only have a meaningful implementation in ComponentType. All other types derived from
+                ComponentType that also inherit from IComponentType should do nothing.
+            */
+        virtual SLANG_NO_THROW void SLANG_MCALL computeDependencyBasedHash(
+            SlangInt entryPointIndex,
+            SlangInt targetIndex,
+            Digest* outHash) = 0;
+
+            /** Compute the hash code of this component type's AST. This hash effectively represents
+                the contents of the code covered by this component type, making its use ideal when we need
+                to confirm whether shader code changes have occurred. For example, a shader cache needs to be
+                able to check when a cache entry contains out-of-date code, which can be easily detected by
+                comparing the AST-based hashes since any change to the shader's code will be reflected in the
+                AST, and subsequently, the AST-based hash.
+
+                Not all component types will store an AST, and consequently, not all component types will have a
+                meaningful implementation for this function.
+
+                This function should only have a meaningful implementation in ComponentType. All other types derived
+                from ComponentType that also inherit from IComponentType should do nothing.
+            */
+        virtual SLANG_NO_THROW void SLANG_MCALL computeASTBasedHash(Digest* outHash) = 0;
+
             /** Specialize the component by binding its specialization parameters to concrete arguments.
 
             The `specializationArgs` array must have `specializationArgCount` entries, and