summaryrefslogtreecommitdiffstats
path: root/source/slang/slang.cpp
Commit message (Collapse)AuthorAge
...
* nsight Aftermath crash example (#2984)jsmall-nvidia2023-07-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Small fixes and improvements around reflection tool. * Make PrettyWriter printing a class. * Aftermath crash demo WIP. * Enable aftermath in test project. * Setting failCount. * Dumping out of source maps. * Improve comments. Simplify handling of compile products. * Other small fixes to aftermath example. * Added Emit SourceLocType. Track sourcemap association meaning. Improved documentation. * Small improvements. * Capture debug information for D3D11/D3D12/Vulkan. * Enable debug info. * Small improvements. * Improve aftermath example README.md.
* Simplify Lookup and improve compiler performance. (#2996)Yong He2023-07-18
| | | | | | | | | | | | | | | | | | | | | | | | | * Simplify lookup. * Various bug fixes. * Report type dictionary size in perf benchmark. * Remove type duplication. * increase initial dict size. * Bug fix. * Fix bugs. * Fixup. * Revert type legalization looping. * Fix specialization pass. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Allow setting of HLSLToVulkan options without having a target specified on ↵jsmall-nvidia2023-07-14
| | | | the command line options. (#2989)
* Create and cache flattened inheritance lists (#2740)Theresa Foley2023-07-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Create and cache flattened inheritance lists The basic change here is to have a cached lookup that can map a `Type`, or a `DeclRef` that might refer to a type or `extension`, to a list of the *facets* that comprise it. The notion of a *facet* here is similar to what the C++ standard calls "sub-objects". A declared type like a `struct` has: * a facet for its own direct members * one facet for each of its (transitive) base `struct` types * one facet for each `interface` it conforms to * one facet for each `extension` that applies to that type The set of facets for a type is de-duplicated (so that "diamond" inheritance patterns don't cause issues) and deterministically ordered, using a variation of the C3 linearization algorithm. The creation of a linearized list of facets should help the compiler implementation in two key places: * Testing if a type implements an interface (or inherits from a base type) should now only take time linear in the number of (transitive) bases of that type. We can simply scan the linearized facet list to see if it contains a facet corresponding to the given base. * Looking up the members of a type (or a value of a given type) should be greatly simplified, since all of the members can be found in a single linear scan of the facet list. In addition, those facets will be ordered so that facets for "more derived" types will precede those for "less derived" types, so that shadowing in the case of overrides should be easier to implement. This change only implements the first of these two improvements, since there is already a *lot* of churn involved. Notes and caveats: * The handling of conjunction types (e.g., `IFoo & IBar`) complicates the implementation, both because the simple approach to subtype testing alluded to above is no longer complete, and also because we need to be more careful about what forms of subtype witnesses we construct, so that we can maintain the currently-required invariant that two witnesses are only equal if they have matching structure. * We don't implement the full/"proper" C3 algorithm here because it has some failure cases that we'd still like to support. In particular if we have both `IX : IA, IB` and `IY : IB, IA`, the C3 algorithm says it is illegal to have `IZ : IX, IY` because the two bases it inherits from disagree on the relative ordering of `IA` and `IB` in their own linearizations. Handling such cases may make our implementation less efficient, and it will also require testing of those corner caes. * When it comes time to revamp the implementation of lookup, we will need to deal with the fact that a single linear list (seemingly) cannot give us sufficient information to decide which of two members of the same name should shadow the other, or if there is an ambiguity. Or rather, it *can* give us that information if we are willing to accept some very user-unfriendly behavior and simply say that declarations earlier in the linearization always shadow later declarations, even if the facets involved are not related by an inheritance relationship of any kind. * In order to remove one kind of vicious circularity from the approach, the linearization that we are computing for `extension` declarations will not be sufficient for lookups in the body of such an `extension`. A future change may need to have support for creating and caching two distinct linearizations for each `extension`: one that is to be used when that `extension` is pulled into the linearization for a type that it applies to, and another for when lookup will be performed in the context of the `extension` itself. * This change does *not* include the simple expedient of adding a direct cache for subtype tests to the `SharedSemanticsContext`, although adding such a cache would be a simple matter. * This change introduces more deduplication for subtype witnesses, which should enable more deduplication for other `Val`s (including `Type`s), but it does not introduce any assumptions that equal `Val`s or `Type`s must have identical pointer representations. * Eventually we may find that, similar to the situation with `Type`s, we will want to have a split between surface-level and canonicalized versions of other `Val`s, including subtype witnesses. * Fix clang error. * remove debugging code. --------- Co-authored-by: Yong He <yonghe@outlook.com>
* Use scratchData on `IRInst` to replace HashSets. (#2978)Yong He2023-07-12
| | | | | | | | | | | | | | | * Use scratchData on `IRInst` to replace HashSets. * Update test results. * Initialize scratchData. * Update autodiff documentation. * Use enum instead of bool. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Add perf benchmark utility. (#2977)Yong He2023-07-11
| | | | | | | | | | | | | * Add perf benchmark utility. * Update documentation. * Fix. * Fix doc. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Make DeclRefBase a Val, and DeclRef<T> a helper class. (#2967)Yong He2023-07-07
| | | | | | | | | | | | | | | | | | | | | * Make DeclRefBase a Val, and DeclRef<T> a helper class. * Fixes. * Workaround gcc parser issue. * Revert NodeOperand change. * Fix. * Fix clang incomplete class complains. * Fix code review. * Small cleanups and improvements. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Improvements around StringBlob (#2921)jsmall-nvidia2023-06-08
| | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Small fixes and improvements around reflection tool. * Make PrettyWriter printing a class. * Improvements around handling StringBlob and storing stdlib source in ISlangBlob. * Fix some issues with comments around StringBlob. * Default initialize StringBlob fields.
* Add API for querying total compile time. (#2898)Yong He2023-05-23
| | | | | | | | | | | | | * Add API for querying total compile time. * Optimize. * Remove redundant simplifyIR calls. * Fix. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Add slangc options for reporting downstream compile time. (#2878)Yong He2023-05-10
| | | | | | | | | * Add slangc options for reporting downstream compile time. * Update doc. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Various fixes for autodiff and slangpy. (#2876)Yong He2023-05-09
| | | | | | | | | | | | | * Various fixes for autodiff and slangpy. * Fix cuda code gen for `select`. * Fix getBuildTagString(). * Fix. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* HLSL->Vulkan binding support (#2865)jsmall-nvidia2023-05-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WIP around VK shift binding. * Refactor around options parsing. * Remove needless passing around of sink. * Some more tidying around OptionsParser. * Handle vulkan shift parsing. * Fix small issue around vk binding and "all". * Fixing some small issues. Missing break. * Split out VulkanLayoutOptions * WIP binding taking into account HLSL->Vulkan options. * First attempt at making binding work with HLSLVulkanOptions. * VulkanLayoutOptions -> HLSLToVulkanLayoutOptions * WIP with HLSL-Vulkan binding. * Some more testing around vk-shift. * Improvements around global binding. More tests. * Improve test coverage. Improve checking for requirements around default space. * Update command line options. * Small fixes. * Small fix in options reporting. * Fix warning issue. * Some fixes for isDefault for HLSLToVulkanLayoutOptions. * Update hlsl-to-vulkan-shift output. The difference was due to default handling if shift isn't specified, and not being specified was not correctly tracked.
* Various dxc/fxc compatibility fixes. (#2863)Yong He2023-05-02
| | | | | | | | | | | | | * Various dxc/fxc compatibility fixes. * Cleanup. * Fix test cases. * Fix comments. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* CommandOptions (#2856)jsmall-nvidia2023-04-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WIP CommandOptions * Fix some output issues. * Simplify word wrapping. * Add file extensions. * Change how lookup takes place. Add appendSplit functions to StringUtil. Make Categories hold the index range of their options. * Small improvement. * Lookup with partial option names. * Associate user values. * Encoding flags in the name. * Refactor setting up of command options. * Use CommandOptions in slang-options. * Remove old help text. * Cache the CommandOptions on the Session. * Range checking. Fix bug in the Options handling. * Extra checks for validity. * Get categories directly. * Slight improvements over output. * Added NameValue types. * Fix typo. Remove some now unused diagnostics. Fix diagnostic in testing, as output has changed. * Add minimal usage message. * Remove platform executable extension from diagnostics output. * Some improvements around getting names from NameValue types. * Improve some option descriptions. * Small fixes.
* Embed stdlib documentation to AST. (#2851)Yong He2023-04-27
| | | | | | | | | * Embed stdlib documentation to AST. * Extract documentation for attributes. --------- Co-authored-by: Yong He <yhe@nvidia.com>
* Fix most of the disabled warnings on gcc/clang (#2839)Ellie Hermaszewska2023-04-26
|
* StringBuilder to lowerCamel (#2840)jsmall-nvidia2023-04-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP lowerCamel Dictionary. * WIP more lowerCamel fixes for Dictionary. * Add/Remove/Clear * GetValue/Contains * Fix tabs in dictionary. Count -> getCount * Fix fields with caps. * Key -> key Value -> value Use m_ for members where appropriate. Use lowerCamel in linked list. * Some small fixes/improvements to Dictionary. * Kick CI. * Small tidy on String. * Append -> append * ToString -> toString ProduceString -> produceString * Small fixes. * StringToXXX -> stringToXXX * Fix typo introduced by Append -> append. * Made intToAscii do reversal at the end. --------- Co-authored-by: Yong He <yonghe@outlook.com>
* Dictionary using lowerCamel (#2835)jsmall-nvidia2023-04-25
| | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP lowerCamel Dictionary. * WIP more lowerCamel fixes for Dictionary. * Add/Remove/Clear * GetValue/Contains * Fix tabs in dictionary. Count -> getCount * Fix fields with caps. * Key -> key Value -> value Use m_ for members where appropriate. Use lowerCamel in linked list. * Some small fixes/improvements to Dictionary. * Kick CI.
* Improvements outputting containers (#2815)jsmall-nvidia2023-04-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Moved JSON source map writing logic to JSONSourceMapUtil. * Use ArtifactHandler to read/write SourceMaps. Use ObjectCastableAdapter to hold SourceMap Only serialize SourceMap <-> JSON on demand. * Make some types swappable. * BoxValue impl. * Added asBoxValue. * Remove const get funcs. * Fix typo in asBoxValue. * Fix another typo in asBoxValue. * Slightly simplify conversion to blob of SourceMap. * WIP Api improvements around sourcemap/artifact/line-directive. * Small fix for asBoxValue * WIP outputting container with multiple artifacts. * Added ArtifactContailerUtil::filter to produce an artifact hierarchy that only contains "signficant" and "blobable" artifacts. * Make emitting IR disjoint to using a container. Added -emit-ir option. Simplfiy output. * Fix typo in options parsing. * Add a test that ouputs with an emit source map. * Enable emitting our SlangIR module if no targets are specified. * Fix issues constructing container. * Extra checks getting obfuscated source map from a translation unit. * Fix typo.
* Make SourceMap a value type (#2812)jsmall-nvidia2023-04-19
| | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Moved JSON source map writing logic to JSONSourceMapUtil. * Use ArtifactHandler to read/write SourceMaps. Use ObjectCastableAdapter to hold SourceMap Only serialize SourceMap <-> JSON on demand. * Make some types swappable. * BoxValue impl. * Added asBoxValue. * Remove const get funcs. * Fix typo in asBoxValue. * Fix another typo in asBoxValue. * Slightly simplify conversion to blob of SourceMap. * Small fix for asBoxValue
* On demand SourceMap JSON serialization (#2811)jsmall-nvidia2023-04-18
| | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Moved JSON source map writing logic to JSONSourceMapUtil. * Use ArtifactHandler to read/write SourceMaps. Use ObjectCastableAdapter to hold SourceMap Only serialize SourceMap <-> JSON on demand.
* Round trip source map (#2810)jsmall-nvidia2023-04-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make output of obfuscation locs work in a slang-module. * Tidy up detection for writing serialized source locs. * Support for .zip references. Handling of obfuscated source maps read from containers. A test to check obfuscated source map working on a module. * When using obfuscation, always obfuscate locs instead of stripping them. We keep a source map, so we can still produce reasonable errors. * Write out source locs if debug information is enabled. * Check output without sourcemap. * Small fixes. * Small improvements around hash calculation for source map name. * Disable test that fails on x86 gcc linux for now. * Fix issues around obfuscated source map using lines rather than columns. Fix some issues around encoding/decoding. * Make column calculation of source locs take into account utf8/tabs. Don't special case obfuscated source map for lookup for source loc. * Support following multiple source maps. * Small fixes/improvements around SourceMap lookup.
* Reading artifact hierarchy from file system interface (#2787)jsmall-nvidia2023-04-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP simplifying artifact interface. * Use ContainedKind. * Remove LazyCastableList. Use ContainedKind for find. * Remove ICastableList. * Remove need for ICastableList. * Remove IArtifactContainer. * Small fixes. * Small improvements around Artifact. * Make explicit find is for *representations* that can cast. Fix bug in handling casting in lookup. * Made associated items artifacts too. * Small fixes. * Small improvements around writing a container. * WIP artifact container format. * Make the root a special case. * Special case if the artifact doesn't have children/associated. * First pass handling of interpretting a file system into artifact hierarchy. * Explain the final structure. Make the file system available. * Remove addArtifact from IArtifact interface - means will be compatible with previous version. * Rename function to get compile result as a filesytem.
* Artifact simplification (#2781)jsmall-nvidia2023-04-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP simplifying artifact interface. * Use ContainedKind. * Remove LazyCastableList. Use ContainedKind for find. * Remove ICastableList. * Remove need for ICastableList. * Remove IArtifactContainer. * Small fixes. * Small improvements around Artifact. * Make explicit find is for *representations* that can cast. Fix bug in handling casting in lookup. * Made associated items artifacts too. * Small fixes. * Small improvements around writing a container.
* Add SLANG_IID_PPV_ARGS (#2784)Ellie Hermaszewska2023-04-07
| | | | | | | | | | | * Add SLANG_IID_PPV_ARGS To mirror IID_PPV_ARGS from the COM Coding Practices: https://learn.microsoft.com/en-us/windows/win32/LearnWin32/com-coding-practices#the-iid_ppv_args-macro * Make getTypeGuid constexpr --------- Co-authored-by: Yong He <yonghe@outlook.com>
* Simplification around ArtifactRepresentation and unique identity (#2771)jsmall-nvidia2023-04-04
| | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP in handling artifacts/SourceFiles. * Add getUniqueIdentity to IPathArtifactRepresentation * Simplification because around using the representation to provide the uniqueIdentity.
* More builtin library support in torch backend. (#2760)Yong He2023-03-30
| | | Co-authored-by: Yong He <yhe@nvidia.com>
* Obfuscated source map writing (#2727)jsmall-nvidia2023-03-24
| | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP produce obfuscated source map and write when container is specified. * Make the sourcemap generated name stable.
* Source map obfuscation (#2717)jsmall-nvidia2023-03-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP source map. * Split out handling of RttiTypeFuncs to a map type. * Make RttiTypeFuncsMap hold default impls. * Slightly more sophisticated RttiTypeFuncsMap * Source map decoding. * Fix tabs. * Fix asserts due to negative values. * Use less obscure mechanisms in SourceMap. * Source map decoding. Simplifying SourceMap usage. * First attempt at ouputting a source map as part of emit. * Added support for -source-map option. SourceMap is added to the artifact. * Small improvements around column calculation in SourceWriter. * Source Loc obuscation WIP. * Fix some issues around SourceMap obfuscation. * Split out obfuscation into its own file. * Keep obfuscated SourceMap even through serialization bottleneck.
* Support for producing SourceMap on emit (#2707)jsmall-nvidia2023-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP source map. * Split out handling of RttiTypeFuncs to a map type. * Make RttiTypeFuncsMap hold default impls. * Slightly more sophisticated RttiTypeFuncsMap * Source map decoding. * Fix tabs. * Fix asserts due to negative values. * Use less obscure mechanisms in SourceMap. * Source map decoding. Simplifying SourceMap usage. * First attempt at ouputting a source map as part of emit. * Added support for -source-map option. SourceMap is added to the artifact.
* Support for PDB output from DXC (#2693)jsmall-nvidia2023-03-10
| | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Add versioning to CompileOptions for DownstreamCompiler so we can add new options without breaking binary interface. * Add support for debug info format to API/command line processing. * Small simplification. * Add support for adding PDB output from a compilation. * Use builtin offset of directly. * Fix typo in debug.
* Add `SourceBlobWithPathInfoArtifactRepresentation`. (#2682)Yong He2023-02-28
|
* Refactor shader cache (#2558)skallweitNV2022-12-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix a bug in Path::find * Fix code formatting * Fix LockFile and add LockFileGuard * Add PersistentCache and unit test * Replace file path dependency list with source file dependency list * Add note on ordering in Module/FileDependencyList * Remove old shader cache code * Refactor shader cache implementation * Temporarily skip unit tests reading/writing files * Fix warning * Reenable lock file test * Rename shader cache tests and disable crashing test * Testing * Stop using Path::getCanonical * Fix persistent cache lock and test * Fix threading issues * Move adding file dependency hashes to getEntryPointHash() * Fix handling of #include files * Allow specifying additional search paths for gfx testing device * Work on shader cache tests * Update project files * Revive shader cache graphics tests * Split graphics pipeline test * Fix compilation
* Cleanup crypto utilities (#2549)skallweitNV2022-12-02
| | | | | | | * Consolidate crypto functions into single module * Migrate rest of code to new crypto module * Fix name conflict
* Cleanup DigestBuilder and MD5HashGen (#2539)skallweitNV2022-11-30
| | | | | | | * Cleanup DigestBuilder and MD5HashGen * Fix templates Co-authored-by: Yong He <yonghe@outlook.com>
* FileStream-based implementation for updating cache index file (#2485)lucy96chen2022-11-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Draft FileStream-based implementation for updating cache file * File streams fully integrated into shader cache code paths; Tests will not run unless file system is on disk as file streams do not play nicely with in-memory * Brought old code back as fallback path, but tests need to ensure previous is freed first * Testing structure updated, beginning cleanup work * All tests working * Cleanup changes * Removed an extra tab at the end of a line * Cleanup change * Undo externals change * Removed redundant logic for OS vs memory file system handling of the shader cache; Removed extra helper function left over from old cache implementation * Reverted performance change to generate contents hashes when modules are being loaded as this code path is not always followed; Contents hashing now uses a combination of hashing and checking the last modified time for all file dependencies, only reading in and hashing the contents of all files if the last modified hash does not match * Added handling to Module::updateContentsBasedHash for file dependencies which are not from a physical source file on disk; Added test for above Co-authored-by: Lucy Chen <lucchen@nvidia.com> Co-authored-by: Yong He <yonghe@outlook.com>
* Shader cache bugfixes and test additions (#2467)lucy96chen2022-10-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Preliminary graphics shader test * Added test checking that a graphics shader is correctly split into two different entries * Removed testing only addition * Changed RequirementDictionary to an OrderedDictionary and added SerialTypeInfo for OrderedDictionary; Added entry point mangled name to the dependency hash * Added test covering failure case discovered as part of Falcor integration * Changed DifferentiableTypeSemanticContext::m_mapTypeToIDifferentiableWitness to an OrderedDictionary * Added serializedAST field to Module in order to save serialized ASTs to avoid reserialization as much as possible; Added classes field to Session in order to save the output of SerialClassesUtil::create to avoid recreating as much as possible * Changed AST hashing to hash the contents of a Module's file dependencies; Renamed all references to AST hashing to contents hashing * Further cleanup * Moved contents hash computation up to Linkage::loadModule and added field to Module to save the computed contents digest * Changed PreprocessorHandler::handleFileDependency to optionally take an ISlangBlob* containing file contents and changed FrontEndPreprocessorHandler::handleFileDependency to add the source code for an included file to the module's contents digest * Removed extraneous addToDigest call * Fixed accidental removal of source code hash for module being loaded
* Shader cache index implementation (#2452)lucy96chen2022-10-17
|
* Add getVersionString() to IDownstreamCompiler (#2446)lucy96chen2022-10-13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * checkpoint * Added shaderCachePath field to IDevice desc in gfx.slang, gfx-smoke.slang should be functional * ran premake * Added getVersionString() to IDownstreamCompiler and implemented it in DownstreamCompilerBase, GlslangDownstreamCompiler, and DXCDownstreamCompiler * Added hashInto to Val and implemented for all subtypes which contain _getHashCodeOverride implementations; nothing hooked up to caching yet * Revert erroneous commits from rebasing * Re-ran premake.bat; changed hashInto implementations to _hashIntoOverride * downstream compiler getVersionString hooked up into shader cache * Rebuild CI, Win32 Release builds successfully locally * Rebuild CI, Win32 and x64 Release build successfully locally * Reverted Val::hashInto changes, deferred for later; modified init() for downstream compilers to fetch, hash, and save the hashes of their corresponding dlls (currently implemented for glslang and dxc), changed getVersionString() to directly return the saved hash for key computation * call site changes post-merge; fixing CI build failures * Removed remaining hashInto; Updated hashToString to produce string one byte at a time * Fixed expected output to match new hashToString output order * Missed string edit in hashing related unit tests * Updated dxcapi.h; Replaced getVersionString implementation in glslang with the shared library timestamp, dxc implementation queries for IDxcVersionInfo and IDxcVersionInfo2 then fetches and returns the relevant values * Fixing CI build failures * Changed RawBlob to StringBlob for strings * Modified getVersionString for dxc to always return either the version plus commit hash or shared library timestamp
* Add specialization args test (#2444)lucy96chen2022-10-12
| | | | | * Added specialization args test; small cleanup changes to slang-digest.h * Moved slang::Digest inside namespace Slang
* Shader caching (#2432)lucy96chen2022-10-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Changed all getEntryPointCode calls to use RendererBase::getEntryPointCodeFromShaderCache * Hashing hooked up, tests pass but need to add more to fully test functionality * checkpoint * Checkpoint: File system creation seems functional, saving is broken * checkpoint: Fixed filename generation from MD5 hash, shader blob might be going missing ahead of pipeline state creation * Fixed a lot of bugs related to hash code generation, shader cache is likely working but needs further testing * Added workaround for module loading by re-creating the test device, shader cache test functional * Vulkan shader caching bug fixed, checkpoint commit before more refinement * pre-ToT merge checkpoint * checkpoint commit, improving cache keys * Significantly expanded items included in the dependency hash for Module; Added dependency hash functions to SpecializedComponentType and RenamedEntryPointComponentType * Temporarily disable shader cache test * Mid cleanup changes, solution successfully builds * Added several helper update functions to slang-md5 to help simplify usage; Added a function under ISession to compute a hash for all linkage-related items; Function renames and cleaned up some comments * Ran premake.bat; Renamed getASTBasedHashCode to computeASTBasedHash * Added slang unit tests for Checksum and MD5; Extended gfx shader cache test to test with multiple shader files and one shader file with multiple entry points * Solution builds and shader cache tests pass, but at least a couple other tests now failing * ran premake.bat * More cleanup changes * Added shaderCachePath field to IDevice desc in gfx.slang, gfx-smoke.slang should be functional * ran premake * cleanup changes; Adding test printf to getEntryPointCodeFromShaderCache to see if output can be seen in CI * Removed debugging printfs; Added handling for getEntryPointCode() failing * Cleanup changes; Jonathan's fixes to SerialWriter to zero initialize otherwise uninitialized memory; Change to SwizzleExpr creation to zero initialize elementCount * Changed enable_if_t to enable_if * Fixed enable_if * Added test for import vs include and changes to included and imported files; Fixed build errors in CUDA; Renamed shader cache statistics fields * cleanup changes * Readd removed file * Restructured computeDependencyBasedHash calls, added computeDependencyBasedHashImpl to all classes dervied from ComponentType * Applied same restructuring to the AST hash functions * Cleanup changes; Moved HashBuilder out to slang-digest.h and added some helper functions to streamline the process of adding items to a hash * Cleanup; Fixed incorrect expected results for shader import and include test
* IMutableFileSystem::saveFileBlob (#2427)jsmall-nvidia2022-10-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Remove ref count for Entry in RiffFileSystem. Free up backing Entry types (to work around Dictionary not doing this). * Some small improvements to RiffFileSystem. * Add testing for file systems. * Split out MemoryFileSystem. * Add some documentation around different FileSystems. * Small tiry up - removing unused headers, fixing some comments. Use StringBlob::moveCreate where appropriate. * Small improvement to MemoryFileSystem. Improve documentation comments a little. * Added PathKind * * Make MemoryFileSystem not have implicit directories * Make RelativeFileSystem only allow access to files in file system (kind of like chroot) * Added Path::simplifyAbsolute * Special handling for root of MemoryFileSystem * Improvements around paths for different impls * More improvements around RelativeFileSystem. Special case root handling. * Test archive serialization. Move testinf from compression. Remove the implicit directory test -> doesn't work on all file systems. * Small optimization that removes need for check for a parent unless an item is being *created*. * Add implicit path testing. * Add support for saveFileBlob Add testing for saveFileBlob * Removed TemporaryFileSet Added PlatformUtil::outputDebugMessage * Some small improvements around RelativeFileSystem. * Split out ImplicitDirectoryCollector so can use without requiring compression systems. * Split out StringSliceIndexMap into own files.
* Language feature: pointer sized int types. (#2401)Yong He2022-09-15
| | | | | | | | | | | | | | | | | | | | | * Language feature: pointer sized int types. * Fix. * small change to test. * Fix stdlib. * Fix. * Fix. * Add typedef for `size_t` in stdlib. * Fix test. * Add `intptr_t::size` constant. Co-authored-by: Yong He <yhe@nvidia.com>
* Deduplicate AST type nodes and cache lookup operations. (#2397)Yong He2022-09-13
| | | | | | | | | | | * wip: dedup AST type nodes and cache lookup. * Fix. * Remove profiling. * Fixes. Co-authored-by: Yong He <yhe@nvidia.com>
* Make ISlangFileSystem derive from ICastable (#2386)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make ISlangFileSystem derive from ICastable. * Make ArchiveFileSystem into an interface Make file systems atomically reference counted. * Small fix. * Some small fixes to work around issues of ICastable on ISlangFileSystem * Use ISlangFileSystem derived type instead of IArchiveFileSystem. Can always get other interface with castAs. * Some small fixes around change of interface returned from archive type functions. * Remove CacheFileSystem member from linkage. Can access easily from m_fileSystemExt if necessary with as cast. * Fix RiffFileSystem casting issue. * Add a check around CacheFileSystem.
* Remove artifact from SourceFile (#2384)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make DownstreamCompileOptions use POD types. * CharSliceAllocator -> SliceAllocator Added SliceConverter CharSliceCaster -> SliceCaster * First attempt at zero terminating around blobs. * Fix clang warning. * Add SlangTerminatedChars Make Blob implementations support it. Make most blobs 'terminated'. * Fix bug setting up sourceFiles for CommandLineDownstreamCompiler. * Traffic in TerminatedCharSlice for sourceFiles. Use ArtifactDesc to generate temporary file names for source. * Fix typo in testing for shared library/C++. * Working with source being passed as artifacts to DownstreamCompiler. * Use artifacts in SourceManager/SourceFile. * Support infering extension from the original file extension. * * Infer extension if can't determine from the artifact type * Split IOSFile/IExtFile representations * Move responsibility for creating OS file to the handler. * Disable the check memory path. * Remove artifact from SourceFile. Lazily generate SourceFile from artifacts as needed. * Fix some small bugs. * Remove maybeAddArtifact. * Load artifacts if repro capture is enabled. * Remove adding by string, because doing so means source will be allocated twice or there is a potential race around ref counting to the contained String. * Add built in source as a blob. * Fix warning. * Make StringBlob own the contents if moved. Fix some compilation issues. * Share StringBlob uniqueness code. * Do move unique on Ctor. * Change MoveUnique to not have any values. * MoveUnique can more sensibly be a struct. Co-authored-by: Yong He <yonghe@outlook.com>
* Make FileSystem files and OS files distinct (#2383)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make DownstreamCompileOptions use POD types. * CharSliceAllocator -> SliceAllocator Added SliceConverter CharSliceCaster -> SliceCaster * First attempt at zero terminating around blobs. * Fix clang warning. * Add SlangTerminatedChars Make Blob implementations support it. Make most blobs 'terminated'. * Fix bug setting up sourceFiles for CommandLineDownstreamCompiler. * Traffic in TerminatedCharSlice for sourceFiles. Use ArtifactDesc to generate temporary file names for source. * Fix typo in testing for shared library/C++. * Working with source being passed as artifacts to DownstreamCompiler. * Use artifacts in SourceManager/SourceFile. * Support infering extension from the original file extension. * * Infer extension if can't determine from the artifact type * Split IOSFile/IExtFile representations * Move responsibility for creating OS file to the handler. * Disable the check memory path.
* Passing source to Downstream compilation as artifacts (#2382)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make DownstreamCompileOptions use POD types. * CharSliceAllocator -> SliceAllocator Added SliceConverter CharSliceCaster -> SliceCaster * First attempt at zero terminating around blobs. * Fix clang warning. * Add SlangTerminatedChars Make Blob implementations support it. Make most blobs 'terminated'. * Fix bug setting up sourceFiles for CommandLineDownstreamCompiler. * Traffic in TerminatedCharSlice for sourceFiles. Use ArtifactDesc to generate temporary file names for source. * Fix typo in testing for shared library/C++. * Working with source being passed as artifacts to DownstreamCompiler. * Use artifacts in SourceManager/SourceFile. * Support infering extension from the original file extension.
* Disable module name translation when it ends with ".slang" (#2377)Yong He2022-08-24
| | | Co-authored-by: Yong He <yhe@nvidia.com>
* Use enums with backing types in Slang API (#2375)jsmall-nvidia2022-08-24
| | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Use enum types and specify backing rather than use typedefs so as to get enum type safety. * Add version of TextureFlavor that uses internal types. Co-authored-by: Yong He <yonghe@outlook.com>