Shader caching (#2432)

* Changed all getEntryPointCode calls to use RendererBase::getEntryPointCodeFromShaderCache

* Hashing hooked up, tests pass but need to add more to fully test functionality

* checkpoint

* Checkpoint: File system creation seems functional, saving is broken

* checkpoint: Fixed filename generation from MD5 hash, shader blob might be going missing ahead of pipeline state creation

* Fixed a lot of bugs related to hash code generation, shader cache is likely working but needs further testing

* Added workaround for module loading by re-creating the test device, shader cache test functional

* Vulkan shader caching bug fixed, checkpoint commit before more refinement

* pre-ToT merge checkpoint

* checkpoint commit, improving cache keys

* Significantly expanded items included in the dependency hash for Module; Added dependency hash functions to SpecializedComponentType and RenamedEntryPointComponentType

* Temporarily disable shader cache test

* Mid cleanup changes, solution successfully builds

* Added several helper update functions to slang-md5 to help simplify usage; Added a function under ISession to compute a hash for all linkage-related items; Function renames and cleaned up some comments

* Ran premake.bat; Renamed getASTBasedHashCode to computeASTBasedHash

* Added slang unit tests for Checksum and MD5; Extended gfx shader cache test to test with multiple shader files and one shader file with multiple entry points

* Solution builds and shader cache tests pass, but at least a couple other tests now failing

* ran premake.bat

* More cleanup changes

* Added shaderCachePath field to IDevice desc in gfx.slang, gfx-smoke.slang should be functional

* ran premake

* cleanup changes; Adding test printf to getEntryPointCodeFromShaderCache to see if output can be seen in CI

* Removed debugging printfs; Added handling for getEntryPointCode() failing

* Cleanup changes; Jonathan's fixes to SerialWriter to zero initialize otherwise uninitialized memory; Change to SwizzleExpr creation to zero initialize elementCount

* Changed enable_if_t to enable_if

* Fixed enable_if

* Added test for import vs include and changes to included and imported files; Fixed build errors in CUDA; Renamed shader cache statistics fields

* cleanup changes

* Readd removed file

* Restructured computeDependencyBasedHash calls, added computeDependencyBasedHashImpl to all classes dervied from ComponentType

* Applied same restructuring to the AST hash functions

* Cleanup changes; Moved HashBuilder out to slang-digest.h and added some helper functions to streamline the process of adding items to a hash

* Cleanup; Fixed incorrect expected results for shader import and include test

1 files changed, 55 insertions, 0 deletions

diff --git a/source/slang/slang-hash-utils.h b/source/slang/slang-hash-utils.h
new file mode 100644
index 000000000..d0d610cb8
--- /dev/null
+++ b/source/slang/slang-hash-utils.h
@@ -0,0 +1,55 @@
+// slang-hash-utils.h - Utility functions specifically designed to be used with slang::Digest
+#pragma once
+#include "../../slang.h"
+#include "../core/slang-basic.h"
+#include "../core/slang-md5.h"
+#include "../core/slang-digest.h"
+
+namespace Slang
+{
+    // Compute the hash for an UnownedStringSlice
+    inline slang::Digest computeHashForStringSlice(UnownedStringSlice text)
+    {
+        DigestBuilder builder;
+        builder.addToDigest(text);
+
+        slang::Digest textHash;
+        builder.finalize(&textHash);
+
+        return textHash;
+    }
+
+    // Combines the two provided hashes.
+    inline slang::Digest combineHashes(const slang::Digest& hashA, const slang::Digest& hashB)
+    {
+        DigestBuilder builder;
+        builder.addToDigest(hashA);
+        builder.addToDigest(hashB);
+
+        slang::Digest combined;
+        builder.finalize(&combined);
+        return combined;
+    }
+
+    // Returns the stored hash in checksum as a String.
+    inline StringBuilder hashToString(const slang::Digest& hash)
+    {
+        StringBuilder filename;
+
+        for (Index i = 0; i < 4; ++i)
+        {
+            auto hashSegmentString = String(hash.values[i], 16);
+
+            auto leadingZeroCount = 8 - hashSegmentString.getLength();
+            for (Index j = 0; j < leadingZeroCount; ++j)
+            {
+                filename.append("0");
+            }
+            filename.append(hashSegmentString.getBuffer());
+        }
+
+        return filename;
+    }
+
+    // TODO: fromString implementation?
+}