summaryrefslogtreecommitdiffstats
path: root/source/slang/slang-file-system.cpp
Commit message (Collapse)AuthorAge
* Added compiler-core project (#1775)jsmall-nvidia2021-04-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Split out compiler-core initially with just slang-source-loc.cpp * More lexer, name, token to compiler-core. * Split Lexer and Core diagnostics. * Move slang-file-system to core. * Add slang-file-system to core. * More DownstreamCompiler into compiler-core * Fix typo. * Add compiler-core to bootstrap proj. * Small fixes to premake * For linux try with compiler-core * Remove compiler-core from examples. * Added NameConventionUtil to compiler-core * Add global function to CharUtil to *hopefully* avoid linking issue. * Hack to make linkage of CharUtil work on linux.
* Zip FileSystem support (#1617)jsmall-nvidia2020-12-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * Add miniz * Fix for separator in CacheFileSystem. Add compression unit test for zip. * Put zip compression into core. * Remove delimiter stripping if simplifying a path - as stripping will fix delimiters. * ZipFileSystem WIP. * More ZipFileSystem working. * Added isEmpty. Fixed small bug is contains. * First pass support for mutability on zip. * Improvements to File::read/writeAllBytes * Can access and save archive - but has memory leaks. * Fix memory leak. * Some ZIP compression tests. * Fix memory leak on ScopedAllocation. Fix off by one bug on UIntSet * Bug fix in UIntSet * Fix remaining ZipFileSystem issues. Adde stand alone unit-test. * Turn tabs to spaces in slang-io.h * Renamed mode ReadWrite (instead of just Write) * Make miniz it's own project. * Fix windows warning on win32. * Remove warnings needed when miniz was included as a header library. * Set the C++ standard via 'flags' in premake. * Add support for 'implicit' paths. * Add testing for implicit directories. Better handling of implicit directories. * Improve comments in ZipFileSystem. * Update comment around reader/writer transformation.
* File system refactor (#1611)jsmall-nvidia2020-11-19
| | | | | | | | | | * #include an absolute path didn't work - because paths were taken to always be relative. * WIP FileSystem refactor. * Made loadFile load the file in binary mode. * Fixed some comments. Fixed typo in RelativePath - not used 'fixedPath'.
* Emit spir-v using MemoryArena to stop memory leak (#1479)jsmall-nvidia2020-08-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Use m_style for OSFindFilesResult * Refactor of FindFilesResult. * Fixes on linux for FindFiles. * Simplify FindFilesState, and linux support for pattern matching. * Small fixes to linux FindFilesState * Fix typo on linux FindFiles * Fix typo in linux FindFiles. * Renamed some variables, and improved comments on FindFiles. * Improve comments on FildFiles * Small improvements around FindFiles. * Refactor FindFiles again.. into a visitor and function in Path. * Fix some problems on linux. * Fix linux typo. * Renamed os -> find-file-util * find-file-utl -> directory-util * Make delete of PathInfo explicit. * Initialize alwaysCreateCollectedParam . * WIP spir-v emit using MemoryArena * Fix bug in spirv emit. * Fix bug with handling null termination on strings in spirv emit. * Small improvements in comments around emit spirv * Remove the 'dst' from emitOperand - we can only emit to the current inst. * Improve SpirV emit comments. * Don't store the created instruction in the InstConstructScope - as it's always the m_currentInst. Don't return the instruction after _beginInst. Slight comment improvements.
* Improvements around hashing (#1355)jsmall-nvidia2020-05-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fields from upper to lower case in slang-ast-decl.h * Lower camel field names in slang-ast-stmt.h * Fix fields in slang-ast-expr.h * slang-ast-type.h make fields lowerCamel. * slang-ast-base.h members functions lowerCamel. * Method names in slang-ast-type.h to lowerCamel. * GetCanonicalType -> getCanonicalType * Substitute -> substitute * Equals -> equals ToString -> toString * ParentDecl -> parentDecl Members -> members * * Make hash code types explicit * Use HashCode as return type of GetHashCode * Added conversion from double to int64_t * Split Stable from other hash functions * toHash32/64 to convert a HashCode to the other styles. GetHashCode32/64 -> getHashCode32/64 GetStableHashCode32/64 -> getStableHashCode32/64 * Other Get/Stable/HashCode32/64 fixes * GetHashCode -> getHashCode * Equals -> equals * CreateCanonicalType -> createCanonicalType * Catches of polymorphic types should be through references otherwise slicing can occur. * Fixes for newer verison of gcc. Fix hashing problem on gcc for Dictionary. * Another fix for GetHashPos * Fix signed issue around GetHashPos
* Enhanced C++ extractor (#1340)jsmall-nvidia2020-05-07
| | | | | | | | | | | | | | | | | | | | | * Extractor builds without any reference to syntax (as it will be helping to produce this!). * Change macros to include the super class. * Added indexOf(const UnownedSubString& in) to UnownedSubString. Refactored extractor * Output a macro for each type with the extracted info - can be used during injection in class * Simplify the header file - as can get super type and last from macro now * Store the 'origin' of a definition * Some small tidy ups to the extractor. * Improve comments on the extractor options. * Made CPPExtractor own SourceOrigins * Small fixes around SourceOrigin. * Small tidy up around macroOrign
* Using repro as a file system (#1104)jsmall-nvidia2019-11-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added RiffReadHelper * Move type to fourCC in Chunk simplifies some code. * Make MemoryArena able to track external blocks. Allow ownership of Data to vary. Changed IR serialization to use moved allocations to avoid copies. As it turns out all of the array writes could use unowned data, but doing so requires the IRData to stay in scope longer than IRSerialData, which it does at the moment - but perhaps needs better naming or a control for the feature. * Write out slang-module container. * WIP on -r option. Loading modules - with -r. * Making the serialized-module run (without using imported module). * Split compiling module from the test. * Separate module compilation with a function working. * Remove serialization test as not used. * Fix warning on gcc. * Updated test to have types across module boundary. * Allow entry point declaration. A test that tries to build with just an entry point declaration and a module. * Try to make link work with multiple modules. * Multi module linking first pass working. * Multi module test working with -module-name option * Added feature to repro manifest of approximation of command line that was used. * Use isDefinition - for determining to add decorations to entry point lowering. * Added support for repo-file-system.h More precise control of CacheFileSystem. Allow RelativeFileSystem to strip paths optionally. Use canonical paths in PathInfo cache. Fix bug in -D options for command line output of StateSerailizeUtil * Add missing slang-options.h * Fix bug in bit slang-state-serialize.cpp with bit removal. * Added documentation around -repro-file-system Added spLoadReproAsFileSystem function. * Fix warning.
* Expose more repro in API, support output params. (#1087)jsmall-nvidia2019-10-23
| | | | | | | | | | | | | * Added spEnableReproCapture to the API. * Added MemoryStreamBase - which can be used to read from without copyin the data. Added the missing Repro API functions - spEnableReproCapture and spExtractRepro. Added support for serializing output filenames. * Improved naming around Stream. Brought Stream and sub types closer to code conventions. * Renamed content -> contents in Stream.
* `Repro` functionality (#1085)jsmall-nvidia2019-10-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WIP on serialize/save state. * Relative string encoding. * Added RelativeContainer unit test. Split out RelativeContainer into core. * Fix bug in RelativeString encoding. * More work around relative container. * Fix checks. * Use RelativeBase for safe access. Use malloc/free/realloc instead of List. * Add natvis support for relative types. * Setting up of state (not includes) writing of repro state. * Capture after spCompile. * Writing SourceFile and file system files. Added -dump-repo * First pass at loading state. * First pass at reading repro. * Small optimization around Safe32Ptr * Refactor how repro data is stored - to make saving off the files more simple, by having all all backed by 'files'. Make file loading always set up PathInfo so we get uniqueIdentifier info. * Generate unique file names. * Added RelativeFileSystem Added saveFile to ISlangFileSystemExt and implemented for interfaces Added mechanism to save of files (and manifest) * Added ability to replace files in repo with directory holding their contents. * Add support for entry points. * Fix problem compiling on linux. * Added SIMPLE_EX option, where everything on command line must be specified. * Fix typo in unit test for relative container. * Fix another typo in unit test for RelativeContainer. * Fix small bugs. * Fix release unused variable issue in slang-state-serialize.cpp * Fix checking for SIMPLE_EX in testing, else broke COMMAND_LINE_SIMPLE. * Fix warnings on 32 bit debug build. * Added import-subdir-search-path-repro.slang test. Although disabled for now as writes to root of slang project. * Remove wrong version of import-subdir-search-path-repro.slang * Added import-subdir-search-path-repro.slang
* A more convoluted #pragma once file identity test, using relative paths. (#1021)jsmall-nvidia2019-08-15
| | | | | | | | | | | | | | * A more convoluted #pragma once file identity test, using relative paths. * Fix bug with passing - to slang as a command line option causes a crash. Ability to set file-system to use on command line. #pragma once tests try with 'normal' and 'read-file' only versions * OSFileSystem -> OSFileSystemExt LoadFileOSFileSystem -> OSFileSystem Implemented OSFileSystem like OSFileSystemExt as as singleton. Fixes to comments.
* Use slang- prefix on slang compiler and core source (#973)jsmall-nvidia2019-05-31
| | | | | | | | | | | | * Prefixing source files in source/slang with slang- * Prefix source in source/slang with slang- prefix. * Rename core source files with slang- prefix. * Update project files. * Fix problems from automatic merge.
* String/List closer to conventions, and use Index type (#959)jsmall-nvidia2019-04-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * List made members m_ Tweaked types to closer match conventions. * Use asserts for checking conditions on List. Other small improvements. * List<T>.Count() -> getSize() * List<T> Add -> add First -> getFirst Last -> getLast RemoveLast -> removeLast ReleaseBuffer -> detachBuffer GetArrayView -> getArrayView * List<T>:: AddRange -> addRange Capacity -> getCapacity Insert -> insert InsertRange -> insertRange AddRange -> addRange RemoveRange -> removeRange RemoveAt -> removeAt Remove -> remove Reverse -> reverse FastRemove -> fastRemove FastRemoveAt -> fastRemoveAt Clear -> clear * List<T> FreeBuffer -> _deallocateBuffer Free -> clearAndDeallocate SwapWith -> swapWith * List<T> SetSize -> setSize Reserve -> reserve GrowToSize growToSize * UnsafeShrinkToSize -> unsafeShrinkToSize Compress -> compress FindLast -> findLastIndex FindLast -> findLastIndex Simplify Contains * List<T> Removed m_allocator (wasn't used) Swap -> swapElements Sort -> sort Contains -> contains ForEach -> forEach QuickSort -> quickSort InsertionSort -> insertionSort BinarySearch -> binarySearch Max -> calcMax Min -> calcMin * Initializer::Initialize -> initialize List<T>:: Allocate -> _allocate Init -> _init IndexOf -> indexOf * * Put #include <assert.h> in common.h, and remove unneeded inclusions * Small refactor of ArrayView - remove stride as not used * getSize -> getCount setSize -> setCount unsafeShrinkToSize->unsafeShrinkToCount growToSize -> growToCount m_size -> m_count * Some tidy up around Allocator. * Use Index type on List. * Refactor of IntSet. First tentative look at using Index. * Made Index an Int Did preliminary fixes. Made String use Index. * Partial refactor of String. * String::Buffer -> getBuffer ToWString -> toWString * Small improvements to String. String:: Buffer() -> getBuffer() Equals() -> equals * Try to use Index where appropriate. * Fix warnings on windows x86 builds.
* Fixed building on CygWin with clang gcc (#953)jsmall-nvidia2019-04-25
| | | | | | | | | | | | | | * * Make Path:: use lowerCamel method names as per coding standard * Small improvements to make closer to standard * GetDirectoryName -> getParentDirectory - previous method name's action was somewhat unclear, hopefully this is better * * Can build on clang and gcc on CygWin * Fix problem on cygwin loading shared libraries * Renamed Path::isRelative to ::hasRelativeElement because isRelative implies the path is 'relative to the current path' and which isn't quite what it does * Documented how to build for CygWin * * Fix small bug creating platform shared library name. * Small typo fixes in building.md
* * Make Path:: use lowerCamel method names as per coding standard (#952)jsmall-nvidia2019-04-24
| | | | * Small improvements to make closer to standard * GetDirectoryName -> getParentDirectory - previous method name's action was somewhat unclear, hopefully this is better
* Feature/view path (#824)jsmall-nvidia2019-02-04
| | | | | | | | | | | | | | | | | | | | | * Use 'is' over 'as' where appropriate. * dynamic_cast -> dynamicCast * Replace 'dynamicCast' with 'as' where has no change in behavior/ambiguity. * Replace dynamicCast with as where doesn't change behavior/non ambiguous. * Keep a per view path to the file loaded - such that diagnostic messages always display the path to the requested file. * Add simplifyPath to ISlangFileSystemExt Simplify (if possible) paths that are set on SourceFile and SourcView - doing so makes reading paths simpler. * Fix small typo. * Improve documentation in source for getFileUniqueIdentity * Fix override warning.
* Feature/file unique identity (#789)jsmall-nvidia2019-01-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * * Fix memory bug around expanding va_args - needed buffer to have space for terminating 0 * Fix problem with FileWriter defaults being globals, as memory they allocate, will only be freed after return from main - work around by making StdWriters RefObject derived, and kept in scope such the writers are destroyed before checks for leaks is found * Added SimplifyPathAndHash mode for CacheFileSystem - will simplify the path and see if simplified path is in cache before reading file (limiting amout of underlying file requests) * * Added calcReplaceChar * Renamed DefaultFileSystem to OSFileSystem * Made OSFileSystem convert windows \ to / on linux * Simplified logic for caching in CacheFileSystem. * Added pragma-once-c to add extra test, but also so there is an 'include' directory in preprocessor tests. * Small fixes in pragma once test. * Simplified cache handling path, so that paths/simplified paths area always added. * Improve naming of methods for different caches. * Removed references to 'canonicalPath' and made 'uniqueIdentity' * * Re-add support for canonicalPath to ISlangFileSystem -> not for uniqueIdentifier but as a way to display 'canonicalPath' * Added peliminary support for being able to display verbose paths in a diagnostic * Added 'clearCache' support * Added verbose path support to SourceManager (now needs a ISlangFileSystemExt to do this) * Added support for '-verbose-path' option to slangc and slang-test.
* Path simplification/hash mode, plus bug fixes (#788)jsmall-nvidia2019-01-21
| | | | | | | | | | | | | | | | | | | | * * Fix memory bug around expanding va_args - needed buffer to have space for terminating 0 * Fix problem with FileWriter defaults being globals, as memory they allocate, will only be freed after return from main - work around by making StdWriters RefObject derived, and kept in scope such the writers are destroyed before checks for leaks is found * Added SimplifyPathAndHash mode for CacheFileSystem - will simplify the path and see if simplified path is in cache before reading file (limiting amout of underlying file requests) * * Added calcReplaceChar * Renamed DefaultFileSystem to OSFileSystem * Made OSFileSystem convert windows \ to / on linux * Simplified logic for caching in CacheFileSystem. * Added pragma-once-c to add extra test, but also so there is an 'include' directory in preprocessor tests. * Small fixes in pragma once test. * Simplified cache handling path, so that paths/simplified paths area always added. * Improve naming of methods for different caches.
* Feature/hash for source identity (#786)jsmall-nvidia2019-01-17
| | | | | | | | | | | | * * Added COMMAND_LINE_SIMPLE test type * Made how spawning works controllable by paramter/type SpawnType * Made break-outside-loop and global-uniform run command line slangc * calcRelativePath -> calcCombinedPath * Add 64 bit version of GetHash. * Add support for Hash based mode for CacheFileSystem.
* Feature/file system cache (#692)jsmall-nvidia2018-10-26
* First pass at caching file system. * default-file-system -> slang-file-system fix problem with location("build.linux") confusing windows build for now. * Added CompressedResult Fix problem in Result construction with it being unsigned * Add support for Path simplification. * Testing for Path::Simplify. * Refactored CacheFileSystem - automatically handles ISlangFileSystem or ISlangFileSystemExt appropriately. Removed WrapFileSystem - because wasn't possible to emulate some of the behavior if just loadFile is implemented. Split out StringBlob - so that no need to convert between ISlangBlob and String repeatidly. * Remove unwanted code in ~CompileRequest