slang.git/tools/gfx/persistent-shader-cache.cpp, branch master

Refactor shader cache (#2558)

2022-12-12T18:25:48+00:00

* Fix a bug in Path::find * Fix code formatting * Fix LockFile and add LockFileGuard * Add PersistentCache and unit test * Replace file path dependency list with source file dependency list * Add note on ordering in Module/FileDependencyList * Remove old shader cache code * Refactor shader cache implementation * Temporarily skip unit tests reading/writing files * Fix warning * Reenable lock file test * Rename shader cache tests and disable crashing test * Testing * Stop using Path::getCanonical * Fix persistent cache lock and test * Fix threading issues * Move adding file dependency hashes to getEntryPointHash() * Fix handling of #include files * Allow specifying additional search paths for gfx testing device * Work on shader cache tests * Update project files * Revive shader cache graphics tests * Split graphics pipeline test * Fix compilation

Cleanup crypto utilities (#2549)

2022-12-02T15:34:53+00:00

* Consolidate crypto functions into single module * Migrate rest of code to new crypto module * Fix name conflict

FileStream-based implementation for updating cache index file (#2485)

2022-11-29T20:35:54+00:00

* Draft FileStream-based implementation for updating cache file * File streams fully integrated into shader cache code paths; Tests will not run unless file system is on disk as file streams do not play nicely with in-memory * Brought old code back as fallback path, but tests need to ensure previous is freed first * Testing structure updated, beginning cleanup work * All tests working * Cleanup changes * Removed an extra tab at the end of a line * Cleanup change * Undo externals change * Removed redundant logic for OS vs memory file system handling of the shader cache; Removed extra helper function left over from old cache implementation * Reverted performance change to generate contents hashes when modules are being loaded as this code path is not always followed; Contents hashing now uses a combination of hashing and checking the last modified time for all file dependencies, only reading in and hashing the contents of all files if the last modified hash does not match * Added handling to Module::updateContentsBasedHash for file dependencies which are not from a physical source file on disk; Added test for above Co-authored-by: Lucy Chen Co-authored-by: Yong He

Shader cache bugfixes and test additions (#2467)

2022-10-29T19:08:41+00:00

* Preliminary graphics shader test * Added test checking that a graphics shader is correctly split into two different entries * Removed testing only addition * Changed RequirementDictionary to an OrderedDictionary and added SerialTypeInfo for OrderedDictionary; Added entry point mangled name to the dependency hash * Added test covering failure case discovered as part of Falcor integration * Changed DifferentiableTypeSemanticContext::m_mapTypeToIDifferentiableWitness to an OrderedDictionary * Added serializedAST field to Module in order to save serialized ASTs to avoid reserialization as much as possible; Added classes field to Session in order to save the output of SerialClassesUtil::create to avoid recreating as much as possible * Changed AST hashing to hash the contents of a Module's file dependencies; Renamed all references to AST hashing to contents hashing * Further cleanup * Moved contents hash computation up to Linkage::loadModule and added field to Module to save the computed contents digest * Changed PreprocessorHandler::handleFileDependency to optionally take an ISlangBlob* containing file contents and changed FrontEndPreprocessorHandler::handleFileDependency to add the source code for an included file to the module's contents digest * Removed extraneous addToDigest call * Fixed accidental removal of source code hash for module being loaded

Cache eviction policy test refinement (#2456)

2022-10-20T16:33:48+00:00

* Refined cache eviction test; Added check to cache file load to ensure only the maximum number of allowed entries is loaded * Fixed logic error in entry count check * Changed getCacheFile to not append the newline at the end of cache files * Refined if check

PersistentShaderCache integration (#2453)

2022-10-19T16:36:41+00:00

* Shader cache index integrated into RendererBase; Added test for cache eviction policy (which currently does not pass) * Restructured main if block in getEntryPointCodeFromShaderCache; Post-rebase cleanup * undo local testing only change * Fixed issues causing shader cache tests to fail * Edited gfx.slang to reflect structural changes to IDevice::Desc and to include ShaderCacheDesc; Modified how the cache is reading in the file from disk; Added a check to the cache eviction policy test that checks for correct order of entries in the cache as well as eight total expected output files for D3D12 and Vulkan * Removed line in gfx-unit-test.cpp for local testing * Edited .gitignore to ignore all shaders automatically generated by the shader cache tests and removed the test shaders that were previously added; Review changes, most notably with an overhaul of how the cache eviction policy test handles checking order of entries * Ran premake; Removed local testing specific line (again) * Removed expected comparison files from earlier commit; Ran premake * Edited premake5.lua to also ignore the auto-generated shader files from specific shader cache tests * Fixed weird indent in premake5.lua

Shader cache index implementation (#2452)

2022-10-18T00:38:59+00:00