summaryrefslogtreecommitdiffstats
path: root/source/core/slang-zip-file-system.cpp
Commit message (Collapse)AuthorAge
* Disable Link-Time-Optimization by default (#7345)Jay Kwak2025-06-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Disable Link-Time-Optimization by default LTO was requested for the release package a while ago. When we added it, LTO was enabled by default although it was needed only for the release packages. Later we found that the Release build cannot be incrementally recompiled when LTO is enabled. It sometimes works fine, but it required full recompilation when it doesn't work. We added a new CMake option, `SLANG_ENABLE_RELEASE_LTO`, to disable it for developers. But many Slang developers don't know the option exists. I was going to update the document, CONTRIBUTING.md, but I thought it will be better to change the default behavior. * Fix a compiler warning treated as an error on linux A padding variable was uninitialized, which is fine, but the compiler was complaining about it. * Fix other gcc error for uninitialized variable * Fix more compile warning treated as error * Fix compiler warning from gcc 11 It appears that this is a valid warning that the `delete this` is done on an offset 8 when the class uses multiple inheritance. The compiler warning is following: ``` In file included from /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:5, from /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:3: In destructor ‘virtual Slang::ComBaseObject::~ComBaseObject()’, inlined from ‘uint32_t Slang::ComBaseObject::_releaseImpl()’ at /home/runner/work/slang/slang/source/core/slang-com-object.h:49:16, inlined from ‘virtual uint32_t Slang::MemoryFileSystem::release()’ at /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:34:5: /home/runner/work/slang/slang/source/core/slang-com-object.h:33:31: error: ‘void operator delete(void*, std::size_t)’ called on pointer ‘<unknown>’ with nonzero offset 8 [-Werror=free-nonheap-object] 33 | virtual ~ComBaseObject() {} | ^ In destructor ‘virtual Slang::ComBaseObject::~ComBaseObject()’, inlined from ‘uint32_t Slang::ComBaseObject::_releaseImpl()’ at /home/runner/work/slang/slang/source/core/slang-com-object.h:49:16, inlined from ‘virtual uint32_t Slang::MemoryFileSystem::release()’ at /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:34:5, inlined from ‘Slang::ComPtr<T>::~ComPtr() [with T = ISlangMutableFileSystem]’ at /home/runner/work/slang/slang/include/slang-com-ptr.h:113:34, inlined from ‘void _modulePtr_impl(UnitTestContext*)’ at /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:92:1: /home/runner/work/slang/slang/source/core/slang-com-object.h:33:31: error: ‘void operator delete(void*, std::size_t)’ called on pointer ‘<unknown>’ with nonzero offset 8 [-Werror=free-nonheap-object] 33 | virtual ~ComBaseObject() {} | ^ /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp: In function ‘void _modulePtr_impl(UnitTestContext*)’: /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:36:69: note: returned from ‘void* operator new(std::size_t)’ 36 | ComPtr<ISlangMutableFileSystem>(new Slang::MemoryFileSystem()); | ^ ``` The problem is on the fact that `ComBaseObject` is not the first in the multiple inheritance: ``` class MemoryFileSystem : public ISlangMutableFileSystem, public ComBaseObject { public: // ISlangUnknown SLANG_COM_BASE_IUNKNOWN_ALL ``` It should be: ``` class MemoryFileSystem : public ComBaseObject, public ISlangMutableFileSystem ``` The chain of ComObject release is little complicated and it is easy to make a mistake. Here is summary with details, 1. `release()` is declared as a pure-virtual in ISlangUnknown, which is one of the base classes of `ISlangMutableFileSystem`. ``` struct ISlangUnknown { virtual SLANG_NO_THROW uint32_t SLANG_MCALL release() = 0; ``` 2. `release()` is implemented with the macro `SLANG_COM_BASE_IUNKNOWN_RELEASE`. ``` SLANG_NO_THROW uint32_t SLANG_MCALL release() SLANG_OVERRIDE \ { \ return _releaseImpl(); \ } inline uint32_t ComBaseObject::_releaseImpl() { // Check there is a ref count to avoid underflow SLANG_ASSERT(m_refCount != 0); const uint32_t count = --m_refCount; if (count == 0) { delete this; } return count; } ``` 3. The instance of `MemoryFileSystem` is handled by ComPtr. And `ComPtr::~ComPtr()` calls the `release()`. ``` ComPtr<ISlangMutableFileSystem> memoryFileSystem = ComPtr<ISlangMutableFileSystem>(new Slang::MemoryFileSystem()); SLANG_FORCE_INLINE ~ComPtr() { if (m_ptr) ((Ptr)m_ptr)->release(); } ``` 4. When `delete this` is called, because ComBaseObject is not the first in the multiple inheritance, `this` is 8 byte off from the actual instance address. A fix for this is to change the order of the inheritance and make ComBaseObject to be the first in the order.
* Move switch statement bodies to their own lines (#5493)Ellie Hermaszewska2024-11-05
| | | | | | | | | * Move switch statement bodies to their own lines * format --------- Co-authored-by: Yong He <yonghe@outlook.com>
* formatEllie Hermaszewska2024-10-29
| | | | | | | * format * Minor test fixes * enable checking cpp format in ci
* Move the file public header files to `include` dir (#4636)kaizhangNV2024-07-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Move the file public header files to `include` dir Close the issue (#4635). Move the following headers files to a `include` dir located at root dir of slang repo: slang-com-helper.h -> include/slang-com-helper.h slang-com-ptr.h -> include/slang-com-ptr.h slang-gfx.h -> include/slang-gfx.h slang.h -> include/slang.h Change cmake/SlangTarget.cmake to add include path to every target, and change the source file to use "#include <slang.h>" to include the public headers. The source code update is by the script like follow: ``` fileNames_slang=$(grep -r "\".*slang\.h\"" source/ -l) for fileName in "${fileNames_slang[@]}" do echo "$fileName" sed -i "s/\".*slang\.h\"/\"slang\.h\"/" $fileName done ``` * Fix the test issues * Fix cpu test issues by adding include seach path * Update cmake to not add include path for every target Also change "#include <slang.h>" to "include "slang.h" " to make the coding style consistent with other slang code. * Change public include to private include for unit-test and slang-glslang
* WIP: CMake (#3326)Ellie Hermaszewska2023-12-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * More robust input and output selection in generator tools * Add cmake build system * Get slang-test running with cmake * Bump lz4 and miniz dependencies * Make cmake build more declarative * Correct preprocessor logic in slang.h * Add cuda test to compute/simple * Remove empty cmake files * output placement for cmake, and commenting * Correct include paths in spirv-embed-generator * Format cmake with gersemi * Make cmake build clerer * Neaten header generation Also work around https://gitlab.kitware.com/cmake/cmake/-/issues/18399 by introducing correct_generated_properties to set the GENERATED flag in the correct scope * remove unused files * use 3.20 to set GENERATOR property properly * spelling * more flexible linker arg setting * replace slang-static with obj collection * Set rpath and linker path correctly * neaten generated file generation * tests working with cmake build * fix premake5 build * comment and neaten cmake * remove unnecessary dependency * Build aftermath example only when aftermath is enabled * Add slang-llvm and other dependencies * Put modules alongside binaries * Find slang-glslang correctly * Better option handling * comments * add llvm build test * Better option handling * cmake wobble * use UNICODE and _UNICODE * remove other workflows * use ccache * neaten * limit parallel for llvm build * use ninja for build * Windows and Darwin slang-llvm builds * cache key * verbose llvm build * cl on windows * sccache and cl.exe * use cl.exe * Correct package detection * less verbosity * Simplify miniz inclusion * fix build with sccache * Neaten llvm building * neaten * Neaten slang-llvm fetching * more surgical workarounds * Add ci action * Get version from git * better variable naming * add missing include * clean up after premake in cmake * more docs on cmake build * ci wobble * add imgui target * more selective source * do not download swiftshader * Some missing dependencies * only build llvm on dispatch * Disable /Zi in CI where sccache is present * simplify * set PIC for miniz * set policies before project * reengage workaround * more runs on ci * Add cmake presets * Add cpack * move iterator debug level to preset * Correct lib flag * simplify action * Neaten cmake init * Add todo * Add simple test wrapper * Add tests to workflow presets * rename packing preset * Correctly set definitions * docs * correct preset names * Make slang-test depend on test-server/test-process * neaten * use workflow in actions * install docs * Correct module install dir * debug dist workflow * Install headers * neaten header globbing * Neaten dependency handling * make lib and bin variables * Do not set compiler for vs builds, unnecessary * docs * allow setting explicit source for target * maintain archive subdir * cmake docs * install headers * place targets into folders * cmake docs * nest external projects in folder * remove name clash * Neater external packages * meta targets in folder structure * cleaner slang-glslang dll * Add missing static directive to slang-no-embedded-stdlib * more robust module copying * make slang-test the startup project * folder tweak * Make FETCH_BINARY the default on all platforms * Set DEBUG_DIR * add natvis files to source * skip spirv tests * remove test step from debug dist * Add build to .gitignore * redo warnings to be more like premake * Update imgui * clean more premake files * Disable PCH for glslang, gcc throws a warning * Add /MP for msvc builds * warning wobble * Add script to build llvm * Add slang-llvm and generators components * Build slang-llvm in ci * comments * fetch llvm with git * better abi approximation for cache * better sccache key * formatting * Correct logic around disabling problematic debug info for ccache * exclude gcc and clang from windows ci * Make dist workflows use system llvm * naming * restore normal dist builds * formatting * run tests in ci * Correct slang-llvm url setting * Rely on the system to find the test tool library * actions matrix wiggle * cope with OSX ancient bash * Correct compilers on windows * more ci debugging * Correct rpath handling on OSX * neaten * correct path to slang-llvm * Correct rpath separator on osx * Find slang-llvm correctly * smoke tests only on osx * ci wobble * Give MacOS module a dylib suffix * get swiftshader correctly * cope with bsd cp * remove debug output * full tests on osx * ci wobble * Add some vk tests to expected failures * simplify ci * ci wobble * exclude dx12 tests from github ci * remove cmake code for building llvm * warnings * warnings as errors for cl * spirv-tools in path * add aarch64 ci build * Add SLANG_GENERATORS_PATH option for prebuilt generators * neaten * Correct generator target name * remove yaml anchors because github actions does not support them * Demote CMake in docs Also add info on cross compiling * Restore premake CI * use minimal ci for cmake * Write miniz_export for premake build and .gitignore it * Mention build config tool options in docs * Remove redefined macro for miniz * regenerate vs project
* Improvements around absolute paths and file systems (#2433)jsmall-nvidia2022-10-06
| | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Add handling for root paths. * Fixes around absolute paths. * Add SimplifyStyle * Remove unrequire include. * Fix some details around RelativeFileSystem canonical paths. * For MemoryFileSystem make sure "/a" and "a" maps to same canonical path. * Add test for canonicalPath. * Improve comment. * More testing around canonical paths.
* IMutableFileSystem::saveFileBlob (#2427)jsmall-nvidia2022-10-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Remove ref count for Entry in RiffFileSystem. Free up backing Entry types (to work around Dictionary not doing this). * Some small improvements to RiffFileSystem. * Add testing for file systems. * Split out MemoryFileSystem. * Add some documentation around different FileSystems. * Small tiry up - removing unused headers, fixing some comments. Use StringBlob::moveCreate where appropriate. * Small improvement to MemoryFileSystem. Improve documentation comments a little. * Added PathKind * * Make MemoryFileSystem not have implicit directories * Make RelativeFileSystem only allow access to files in file system (kind of like chroot) * Added Path::simplifyAbsolute * Special handling for root of MemoryFileSystem * Improvements around paths for different impls * More improvements around RelativeFileSystem. Special case root handling. * Test archive serialization. Move testinf from compression. Remove the implicit directory test -> doesn't work on all file systems. * Small optimization that removes need for check for a parent unless an item is being *created*. * Add implicit path testing. * Add support for saveFileBlob Add testing for saveFileBlob * Removed TemporaryFileSet Added PlatformUtil::outputDebugMessage * Some small improvements around RelativeFileSystem. * Split out ImplicitDirectoryCollector so can use without requiring compression systems. * Split out StringSliceIndexMap into own files.
* Make ISlangFileSystem derive from ICastable (#2386)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make ISlangFileSystem derive from ICastable. * Make ArchiveFileSystem into an interface Make file systems atomically reference counted. * Small fix. * Some small fixes to work around issues of ICastable on ISlangFileSystem * Use ISlangFileSystem derived type instead of IArchiveFileSystem. Can always get other interface with castAs. * Some small fixes around change of interface returned from archive type functions. * Remove CacheFileSystem member from linkage. Can access easily from m_fileSystemExt if necessary with as cast. * Fix RiffFileSystem casting issue. * Add a check around CacheFileSystem.
* Passing source to Downstream compilation as artifacts (#2382)jsmall-nvidia2022-09-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make DownstreamCompileOptions use POD types. * CharSliceAllocator -> SliceAllocator Added SliceConverter CharSliceCaster -> SliceCaster * First attempt at zero terminating around blobs. * Fix clang warning. * Add SlangTerminatedChars Make Blob implementations support it. Make most blobs 'terminated'. * Fix bug setting up sourceFiles for CommandLineDownstreamCompiler. * Traffic in TerminatedCharSlice for sourceFiles. Use ArtifactDesc to generate temporary file names for source. * Fix typo in testing for shared library/C++. * Working with source being passed as artifacts to DownstreamCompiler. * Use artifacts in SourceManager/SourceFile. * Support infering extension from the original file extension.
* DownstreamCompileOptions using POD types (#2381)jsmall-nvidia2022-08-26
| | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Make DownstreamCompileOptions use POD types. * CharSliceAllocator -> SliceAllocator Added SliceConverter CharSliceCaster -> SliceCaster * First attempt at zero terminating around blobs. * Fix clang warning. * Add SlangTerminatedChars Make Blob implementations support it. Make most blobs 'terminated'. * Fix bug setting up sourceFiles for CommandLineDownstreamCompiler. * Traffic in TerminatedCharSlice for sourceFiles. Use ArtifactDesc to generate temporary file names for source. * Fix typo in testing for shared library/C++.
* Artifact and ICastable (#2351)jsmall-nvidia2022-08-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP with hierarchical enums. * Some small fixes and improvements around artifact desc related types. * Improvements around hierarchical enum. * Fixes to get Artifact types refactor to be able to execute tests. * Attempt to better categorize PTX. * Work around for potentially unused function warning. * Typo fix. * Simplify Artifact header. * Small improvements around Artifact kind/payload/style. * Added IDestroyable/ICastable * Add IArtifactList. * First impl of IArtifactUtil. * Use the ICastable interface for IArtifactRepresentation. * Added IArtifactRepresentation & IArtifactAssociated. * Add SLANG_OVERRIDE to avoid gcc/clang warning. * Fix calling convention issue on win32. * Fix missing SLANG_OVERRIDE. * First attempt at file abstraction around Artifact. * Added creation of lock file. * Move functionality for determining file paths to the IArtifactUtil. Add casting to ICastable. * Added some casting/finding mechanisms. * Simplify IArtifact interface, and use Items for file reps. * Fix problem with libraries on DXIL. * Split out ArtifactRepresentation. * Move ArtifactDesc functionality to ArtifactDescUtil. ArtifactInfoUtil becomes ArtifactDescUtil. * Split implementations from the interfaces for Artifact. * Use TypeTextUtil for target name outputting. * Add artifact impls. * Add ICastableList * Added UnknownCastableAdapter * Make ISlangSharedLibrary derive from ICastable, and remain backwards compatible with slang-llvm. * Refactor Representation on Artifact. * Make our ISlangBlobs also derive from ICastable. Make ISlangBlob atomic ref counted. * Fix typo.
* Associating GUID (or UUID) with types (#1776)jsmall-nvidia2021-04-01
| | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Add mechanism to embed guid inside of type.
* LZ4 compression support (#1654)jsmall-nvidia2021-01-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Testing out use of lz4. * Added ICompressionSystem, and LZ4 implementation. * Add support for deflate compression. Simplify compression interface - to make more easily work across apis. * WIP on CompressedFileSystem. * ImplicitDirectoryCollector * SubStringIndexMap - > StringSliceIndexMap. * WIP save stdlib in different containers. * Support for different archive types for stdlib. * Fix project. * CompressedFileSystem -> ArchiveFileSystem. Added CompressionSystemType::None * Added ArchiveFileSystem * Fix problem RiffFileSystem load withoug compression system. * Test archive types. Improve diagnostic message. * Fix typo in testing file system archives. * Split out archive detection. * Fix gcc warning issue. * Fix warning. * RiffArchiveFileSystem -> RiffFileSystem Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
* Building with embedded stdlib (#1634)jsmall-nvidia2020-12-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Move reflection to reflection-api. * Slight reorg to pull out potentially Slang internal functions from the reflection API impls. * Remove visual studio projects * Fix for slang-binaries copy. * Add the visual studio projects in build/visual-studio * Remove miniz project. * Differentiate the linePath from the filePath. * Improve comment in premake5.lua + to kick of CI. * Kick CI. * Use COM compile request for calls to functions inside api-less-slang. Add static-slang project. * Fix const typo issue. * Don't include 'core' link in 'api-less-slang' * Removed static-slang lib causes problems on linux with linking. Embed Slang stdlib Added StaticBlob Added dumpSourceBytes Use ConstArrayView for the archive. At startup allow loading of zip with stdlib. Made -save-stdlib -load-stdlib take a name Added '-save-stdlib-bin-source' to save out serialized stdlib as source. * Ability enable/disable stdlib embedding. * Fix problem with moduleDecl not having module pointer set when serialized in. * Set of debugdir for slang-test and examples. * Add slang-stdlib-api.cpp * Update slang filters for VS. * Try to use pic, and -mcmodel=medium * Some more efforts ot make premake work. * WIP premake5.lua from previously working version. * Remove api-less-slang project. * Disable dllexport on gcc/clang. * Embed via slangc-bootstrap. * Fix slang-profile. Always compiles without stdlib. * Use pic "On" * Remove slangc-bootstrap and embed-stdlib-generator if embedding not required. Make bootstrap run the generators. * Improve comments in premake5.lua. Kick off another CI build. * Remove generation of stdlib source from std-lib-serialize.slang
* Zip FileSystem support (#1617)jsmall-nvidia2020-12-02
* #include an absolute path didn't work - because paths were taken to always be relative. * Add miniz * Fix for separator in CacheFileSystem. Add compression unit test for zip. * Put zip compression into core. * Remove delimiter stripping if simplifying a path - as stripping will fix delimiters. * ZipFileSystem WIP. * More ZipFileSystem working. * Added isEmpty. Fixed small bug is contains. * First pass support for mutability on zip. * Improvements to File::read/writeAllBytes * Can access and save archive - but has memory leaks. * Fix memory leak. * Some ZIP compression tests. * Fix memory leak on ScopedAllocation. Fix off by one bug on UIntSet * Bug fix in UIntSet * Fix remaining ZipFileSystem issues. Adde stand alone unit-test. * Turn tabs to spaces in slang-io.h * Renamed mode ReadWrite (instead of just Write) * Make miniz it's own project. * Fix windows warning on win32. * Remove warnings needed when miniz was included as a header library. * Set the C++ standard via 'flags' in premake. * Add support for 'implicit' paths. * Add testing for implicit directories. Better handling of implicit directories. * Improve comments in ZipFileSystem. * Update comment around reader/writer transformation.