| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
Closes https://github.com/shader-slang/slang/issues/8477
About a 50% reduction in deser performance for capability sets
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Improve performance of AST deserialization
The primary goal of these changes is to reduce the total time spent in the global session's `loadBuiltinModule()`, which gets called as part of global session creation to load the core module, and thus impacts every invocation of `slangc` and every user of the Slang compiler API.
The majority of the time is spent simply deserializing the core module's AST and IR and, of those two, the AST takes significantly longer to load than the IR (in the ballpark of 5x the time).
This change is focused on the serialization infrastructure but, given the performance situation described above, the focus is first and foremost on *deserialization* performance for the Slang *AST*, when using the *fossil* format.
That focus shows through in the changes that have been implemented.
Change serialization framework to use `template` instead of `virtual`
=====================================================================
The recently-introduced serialization framework in `slang-serialize.h` was centered around a dynamically-dispatched `ISerializerImpl` interface.
As a result, every single invocation of a `serialize(...)` call ultimately went through `virtual` function dispatch.
While the overhead of the `virtual` calls themselves does not have a major impact on the total deserialization performance, those calls end up serving as a barrier to further optimization.
This change changes operations that used to take a `Serializer const&` (which wraps an `ISerializerImpl*`), to instead declare a template parameter `<typename S>` and take an `S const&`.
The main consequence of the change is that `serialize()` functions for user-defined types will need to be template functions, and thus either be defined in headers (alongside the type that they serialize) or else in the specific source file that handles serialization (as is currently being done for the AST-related types in `slang-serialize-ast.cpp`).
Note that if we later decide that we want the ability to perform serialization through a dynamically-dispatched interface (e.g., to easily toggle between different serialization back-ends), it will be easier to layer a dynamically-dispatched implementation on top of the statically-dispatched `template` version than the other way around.
Generous use of `SLANG_FORCE_INLINE`
====================================
In order to unlock further optimizations, a bunch of operations were marked with `SLANG_FORCE_INLINE`.
It is important to note that forcing inlining like this is a big hammer, and needs to be approached with at least a little caution.
The simplest cases are:
* trivial wrapper function that just delegate to another function
* functions that only have a single call site (but exist to keep abstractions clean)
Externalize Scope for `begin`/`end` Operations
==============================================
The old `ISerializerImpl` interface had a bunch of paired begin/end operations that define the hierarchical structure of data being read.
Most serializer implementations (whether for reading or writing) use these operations to help maintain some kind of internal stack for tracking state in the hierarchy.
The overhead of maintaining such a stack with something like a `List<T>` amortizes out over many operations, but even that overhead is unnecessary when the begin/end pairs are *already* mirroring the call stack of the code invoking serialization.
This change modifies the `ScopedSerializerFoo` types so that they each provide a piece of stack-allocated storage to the serializer back-end's `beginFoo()` and `endFoo()` operations.
Currently only the `Fossil::SerialReader` is making use of that facility, but the other implementations of readers and writers in the codebase could be adapted if we ever wanted to.
Streamline `Fossil::SerialReader`
=================================
The most significant performance gains came from changes to the `Fossil::SerialReader` type, aimed at minimizing the cycles spent in the core `_readValPtr()` routine.
That function used to have a large-ish `switch` statement that implemented superficially very different reading logic depending on the outer container/object being read from.
The new logic pushes more work back on the `begin` and `end` operations (which get invoked far less frequently than simple scalar/pointer values get read), so that they always set up the state of the reader with direct pointers to the data and layout for the next fossilized value to be read.
The remaining work in `_readValPtr()` has been factored into a differnt subroutine - `_advanceCursor()` - that takes responsibility for advancing the data pointer, and updating the various other fields.
The `_advanceCursor()` routine is still messier than is ideal, because it has to deal with the various different kinds of logic required for navigating to the next value.
Various other conditionals inside the `SerialReader` implementation were streamlined, mostly by collapsing the `State::Type` enumeration down to only represent the cases that are truly semantically distinct.
Evaluated: Streamline Layout Rules for Fossil
=============================================
One potential approach that I implemented but then reverted (after finding it had little to no performance impact) was changing the fossil format to always write things with 4-byte alignment/granularity.
That would mean values smaller than 4 bytes would get inflated to a full 4 bytes, and scalar values larger than 4 bytes get written with only 4-byte alignment (requiring unaligned loads to read them).
I found that the only way to take advantage of the simplified layout rules to improve read performance would be to more-or-less eliminate the use of the layout information embedded in the fossil data, which would make it very difficult to validate that the data is correctly structured.
Possible Future Work: Further Type Specialization
=================================================
As it stands, the biggest overhead remaining on the critical path of `_readValPtr()` is the way the `_advanceCursor()` logic needs to take different approaches depending on the type of the surrounding context (advancing through elements of a container is very different than advancing through fields of a `struct`, for example).
The interesting thing to note is that at the use site within a `serialize()` function, it is usually manifestly obvious which case something is in.
If the code uses `SLANG_SCOPED_SERIALIZER_ARRAY` it is in a container, while if it uses `SLANG_SCOPED_SERIALIZER_STRUCT` it is in a struct.
This means that the contextual information is staticaly available, but just isn't exposed in a way that lets the core reading logic take advantage of it.
A logical extension of the work here would be to expand on the `Scope` idea added in this change such that most of the serialization operations (`handleInt32`, `handleString`, etc.) are actually dispatched through the scope, and then have each of the `SLANG_SCOPED_SERIALIZER_...` macros instantiate a *different* scope type (still dependent on the serializer).
* fixup
* format code
* typo
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Overview
--------
This change basically just flips a `#define` switch to enable the changes that were already checked in with PR #7482.
That earlier change added the infrastructure required to do on-demand deserialization, but it couldn't be enabled at the time due to problematic interactions with the approach to AST node deduplication that was in place.
PR #8072 introduced a new approach to AST node deduplication that eliminates the problematic interaction, and thus unblocks this feature.
Impact
------
Let's look at some anecdotal performance numbers, collected on my dev box using a `hello-world.exe` from a Release x64 Windows build.
The key performance stats from a build before this change are:
```
[*] loadBuiltinModule 1 254.29ms
[*] checkAllTranslationUnits 1 6.14ms
```
After this change, we see:
```
[*] loadBuiltinModule 1 91.75ms
[*] checkAllTranslationUnits 1 11.40ms
```
This change reduces the time spent in `loadBuiltinModule()` by just over 162ms, and increases the time spent in `checkAllTranslationUnits()` by about 5.25ms (the time spent in other compilation steps seems to be unaffected).
Because `loadBuiltinModule()` is the most expensive step for trivial one-and-done compiles like this, reducing its execution time by over 60% is a big gain.
For this example, the time spent in `checkAllTranslationUnits()` has almost doubled, due to operations that force AST declarations from the core module to be deserialized.
Note, however, that in cases where multiple modules are compiled using the same global session, that extra work should eventually amortize out, because each declaration from the core module can only be demand-loaded once (after which the in-memory version will be used).
Because of some unrelated design choices in the compiler, loading of the core module causes approximately 17% of its top-level declarations to be demand-loaded.
After compiling the code for the `hello-world` example, approximately 20% of the top-level declarations have been demand-loaded.
Further work could be done to reduce the number of core-module declarations that must always be deserialized, potentially reducing the time spent in `loadBuiltinModule()` further.
The data above also implies that `loadBuiltinModule()` may include large fixed overheads, which should also be scrutinized further.
Relationship to PR #7935
------------------------
PR #7935, which at this time hasn't yet been merged, implements several optimizations to overall deserialization performance.
On a branch with those optimizations in place (but not this change), the corresponding timings are:
```
[*] loadBuiltinModule 1 176.62ms
[*] checkAllTranslationUnits 1 6.04ms
```
It remains to be seen how performance fares when this change and the optimizations in PR #7935 are combined.
In principle, the two approaches are orthogonal, each attacking a different aspect of the performance problem.
We thus expect the combination of the two to be better than either alone but, of course, testing will be required.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Organize code better by splitting some big files
The basic change here is that the majority of the declarations in `slang-compiler.h` have been split out into a set of smaller and more focused files.
As a result, the implement of those declarations have been moved from `slang-compiler.cpp` and `slang.cpp` over to those new files when the proper home for code is obvious.
I have tried as much as possible to *not* make any edits to the code along the way, and just copy-paste declarations from one place to another as-is.
The exceptions I am aware of are:
* In some cases a function that used to be file-scope `static` was used by code that landed in two or more different `.cpp` files. In these cases, I changed the function to be non-`static` (removing the `_` prefix from its name, if it had one, per our naming conventions), and put a declaration for the function into the most appropriate header I could identify.
* I added a few comments in places where I saw ugly or unfortunate things in the code I was moving, and wanted to tag them with `TODO`s so we can hopefully get to them in the fullness of time.
* I added top-level comments to each of the new `.h` files that was introduced to try to explain the logic for what goes into that file.
* In cases where one of the new header files mostly existed to declare a single type, I sometimes added more detail to the doc comment on that type, to better explain the type and its role in the compiler (this is text that otherwise might have gone into the comment at the top leve lof the file, but I figured that the doc comment would have higher discoverability).
I expect that the most contentious choice here is that the `Session` class lands in `slang-global-session.h` while `slang-session.h` holds the `Linkage` class.
The names used in this change are consistent with how the relevant concepts in the public Slang API are named, and are consistent with how we *intend* to rename the classes themselves in time.
* format code
* fixup
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* bottleneck ir module reading and writing
* compute/simple working
* more complex tests working
* neaten
* factor out SourceLoc serialization
* document changes
* Appease clang
* Correct name serialization
* remove unnecessary code
* neaten
* neaten
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this change does not actually *enable* on-demand deserialization of ASTs, because doing so is incompatible with the current compiler architecture where we have both an `ASTBuilder` and a `SharedASTBuilder`, and there are important invariants about how all AST nodes related to the core module must be created before those of any module using the core module.
Instead, this change simply adds the *infrastructure* for on-demand deserialization, and ensures that those code paths get used at runtime, but actually "demands" all of the nodes in a given serialized AST immediately as part of the deserialization process.
Important notes about the implementation approach:
* PR #7242 ensured that all of the code accessing the direct member declarations of a `ContainerDecl` went through a small(-ish) set of accessor methods. This change takes advantage of that work by further abstracting the storage of the direct member declarations out in a type, `ContainerDeclDirectMemberDecls`, which makes it easy to add custom serialization logic for just that type.
* The `ContainerDeclDirectMemberDecls` type also stores two pointers (one a `RefPtr` and the other a plain pointer) that are only used in the case where the members of a given `ContainerDecl` are being accessed through on-demand deserialization. This can be queried using the `isUsingOnDemandDeserialization()` method but any code accessing a `ContainerDecl` through the intended public API should never need to care about that detail.
* Many of the accessor methods that were added in PR #7242 now branch on whether `isUsingOnDemandDeserialization()` is set. The normal code path is unchanged, and the implementation logic for the on-demand-deserialization case is largely held in `slang-serialize-ast.cpp`, to keep it close to the definitions of the serialized data structures themselves.
* A few types in the `slang-ast-*.h` headers have had `FIDDLE()` annotations added to them, so that they can be used to synthesize some of the serialization logic that was previously hand-written.
* The `_registerBuiltinDeclsRec()` function (which is used to scan the built-in module ASTs for the various "magic" declarations that the `SharedASTBuilder` needs to know about) was factored a bit to support the way that registration needs to behave differently in the case of loading a serialized module (if we kept using the existing recursive search, then it would force every declaration in the core module to be loaded right away). The new `_collectBuiltinDeclsThatNeedRegistrationRec()` function mirrors the overall traversal pattern to produce a flat list that gets included in the serialized AST module. Note in particular that we no longer call `registerBuiltinDecls()` from within `_readBuiltinModule()`.
* The interface of the `Module` type was slightly expanded so that there is a more complete API for accessing the declarations exported from the module. Previously they could only be queried by their mangled name, but the new API also allows the entire list to be iterated over. The `ensureLookupAcceleratorBuilt()` method factors out the logic for building those data structures for a module. Note that in the case where on-demand deserialization is being used for a module, the `findExportedDeclByMandledName()` query will use serialized data directly, rather than build the lookup accelerators as C++ data structures (this is required if we are to avoid immediately deserializing all of the (exported) declarations in the core module as soon as it is loaded).
* A few methods related to loading serialized modules (e.g., `loadSerializedModule()`) have been updated so that along with a pointer to the serialized `ModuleChunk` (which, for those who aren't aware, is a pointer directly into the serialized bytes of the module file), they receive an `ISlangBlob` that refers to the entire blob holding the serialized data (which the `ModuleChunk` is part of). Passing this pointer down allows code running under these methods to retain a reference-counted pointer to the blob to stop the memory of the serialized module from being released until deserialization has been completed.
* The data types defined in `slang-fossil.h` have been overhauled significantly:
* The most important change that is relevant to this work is the introduction of the `Fossilized<T>` template, which is used to statically map a "live" C++ type `T` to its binary fossilized representation. The `slang-fossil.h` file provides infrastructure allowing `Fossilized<T>` to be specialized for user-defined types, and also provides the necessary mappings for the core types like strings, arrays, and dictionaries.
* A key point is that in C++ code, one can take a value of some type `Foo`, serialize it using a `Fossil::SerialWriter`, get a pointer to that serialized data, and then directly cast it to a `Fossilized<Foo>*` and navigate the serialized data directly (without deserializing it back into a `Foo`). For that process to work, any specialization of `Fossilized<T>` must be sure to match the layout that will be produced by the `serialize()` implementation for `T`, when writing to a `Fossil::SerialWriter`.
* Another key change in the public interface of `slang-fossil.h` is that dynamically-typed traversal of the data used to be handled just with `FossilizedValRef`, but now uses a few different types. The `Fossil::ValRef<T>` and `Fossil::AnyValRef` types are used to capture the use cases that want reference-like behavior (basically a `Fossil::ValRef<T>` can be thought of as sort of like a `T&`), while `Fossil::ValPtr<T>` and `Fossil::AnyValPtr` are used for cases that want pointer like behavior (akin to `T*`).
* Then there are related changes in `slang-serialize-fossil.*`:
* The implementation of `Fossil::SerialReader` has been changed to use `Fossil::AnyValPtr` in most places where it formerly used `FossilizedValRef`. Using pointers (that can be null) instead of a weird kind of pseudo-reference (that could still be null) to traverse things was making the code harder to follow than it ought to be, in terms of understanding the levels of indirection in various places.
* Some of the state that was previously in `Fossil::SerialReader` has been split into `Fossil::ReadContext`. This type allows multiple `Fossil::SerialReader`s to be created to read from the same serialized blob(s), while maintaining a persistent mapping from fossilized data pointers to live object pointers. The `ReadContext` also maintains the work list of deferred deserialization actions waiting to be performed, and only flushes that list when the last currently-open `SerialReader` is about to go out of scope.
* In order to support the split of `Fossil::SerialReader` described above (and also to clean up something that didn't quite feel right in the original serialization design) the base serialization framework in `slang-serialize.h` has been tweaked so that a `Serializer` now wraps *two* pointers instead of just one. The first pointer continues to be an implementation of `ISerializerImpl`, which handles the actual reading/writing of data, while the other pointer is an explicit "context" pointer for operations that need additional user-defined context.
* Similar to the changes made to the accessors for direct member declarations in a `ContainerDecl`, the `Module::findExportedDeclByMangledName()` method was updated to conditionally execute a different code path in the case of a module that has been loaded from serialized data.
* Some improvements have been made to the fiddle tool:
* Most importantly, the error-handling logic around Lua script execution has been cleaned up to better match correct Lua idiom. Native functions exposed to the Lua scripts have been changed to just use `lua_call` instead of `lua_pcall`, so rather than attempt to intercept Lua errors they will just automatically propagate them.
* All Lua-related errors are caught at the top level, and reported in a way that uses the source location of the fiddle template that was being evaluated when the error was raised. In most cases, a Lua error should be accompanied by a stack trace of the Lua evluation state. The file paths and line numbers given should be accurate, but aren't directly double-clickable in the Visual Studio output panel, because they use a different format (a good future change might be to process the Lua stack trace and rewrite it into a format that is better for our needs).
* Fixed a subtle bug where having "raw" content (parts of the template that should neither be evaluated nor emitted into the output) that consisted of only whitespace could result in a template being translated to invalid Lua code.
* The bulk of the change is, unsurprisingly, in `slang-serialize-ast.cpp`.
* This file has been refactored enough to look like a complete rewrite. A lot of work has been put into comments that describe the overall approach being taken, so hopefully it can be understood even by somebody who wasn't familiar with the previous code. Some of these are just plain cleanups, rather than being directly related to on-demand serialization.
* Where possible, the code for reading and writing types that needed custom serialization has been moved so that the read/write functions are next to one another, making it easier to visually confirm that the serialized representations match on the read and write sides.
* Where possible, the serialization logic for all types (not just the AST nodes, as was the case before) is being generated via fiddle.
* Rather than just defining `serialize()` overloads for each of the relevant types, the code now defines `Fossilized<...>` specializations for these types as well, to enable statically-typed in-memory traversal of the serialized data. Note, however, that for the most part the `Fossilized<...>` representation types are *not* being used by the code (really only the `ASTModuleInfo` and `ContainerDeclDirectMemberDeclsInfo` types are traversed directly). This can be considered more as work to prove out the design of the `Fossil<...>` template approach, and it may or may not end up being relevant in the future.
* The trivial bit of work to enable on-demand deserialization is in `ASTSerialReadContext::handleContainerDeclDirectMemberDecls()` where, rather than recursively reading the contained declarations, the method effectively just grabs the current cursor of the `Fossil::SerialReader` (which is pointed into the fossilized data) and stashes it into the `ContainerDeclDirectMemberDecls`, along with a `RefPtr` to the `ASTSerialReadContext` itself. Those stashed pointers are what enables the accessors on `ContaienrDeclDirectMemberDecls` to look up information on-demand.
* The more interesting bits of the approach mostly come at the end of the file, where the accessor operations for on-demand deserialization are implemented. Once all the relevant work has been done to write the data structures, and produce `Fossilized<...>` types with the right layout, the work itself may seem almost trivial: a little bit of array iteration, and a little bit of binary-search lookup.
* As a reminder, all of this infrastructure for on-demand deserialization is now in place and able to be invoked by the rest of the compiler, but declarations are currently all being loaded eagerly. The `SLANG_DISABLE_ON_DEMAND_AST_DESERIALIZATION` macro is being used to enable a small bit of extra logic in `ASTSerialReadContext::_cleanUpASTNode` so that the "cleanup" on a just-deserialized `ContainerDecl` includes eagerly querying its list of direct member declarations, which will cause them to be recursively deserialized.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Most of what this change does is straightforward: take all the places in the code that used to operate directly on `ContainerDecl::members` and related fields, and instead have them call into a smaller set of accessor methods defined on `ContainerDecl`.
The primary motivation for making this change is that in order to implement on-demand loading of members from serialized AST modules, we need a way to identify and intercept the "demand" for those members.
On-demand loading benefits from having all accesses to the members of a `ContainerDecl` be as narrow as possible.
If a part of the code only need a member at a specific index, it should say so.
If it only needs access to members with a specific name, or a given subclass of `Decl`, then it should say so.
A secondary motivation for this change is that there have recently been several changes that added complexity and special cases by introducing code that operated on (and *mutated*) the member list of a container decl in ways that the existing code had never done before.
Any code that mutates the member list of a `ContainerDecl` needs to be sure to not disrupt the invariants that the lookup acceleration structures currently rely on.
One of the recent changes added a declaration-to-index map to the set of acceleration structures (with different validation/invalidation behavior than the others...) while other recent changes would remove or insert declarations in ways that could change the indices of other declarations in the same container.
It is not clear if any of these pieces of code were aware of the others, and the invariants that might be expected or broken along the way.
This change bottlenecks the vast majority of accesses to the members of a `ContainerDecl` through the following operations:
* Getting a `List` of all of the direct member declarations of a container
* Get the number of direct member declarations, and accessing them by index.
* Looking up the list of direct member declarations with a given name.
* Adding a new direct member declaration to the end of the list.
Some other operations are layered on top of those (e.g., getting a list of all the direct member declarations of a given C++ class).
These layered operations are still centralized on the `ContainerDecl`, with the intention that we *can* change them to be non-layered implementations if we ever need to for performance (e.g., by building a lookup structure for finding member declarations by their type).
The exceptional cases of access/mutation on the direct members of a `ContainerDecl` have also been encapsulated, but rather than expose what would risk appearing like general-purpose accessors (e.g., `removeDecl(d)`, `setDecl(index)`, etc.), these operations have been explicitly named after the specific use case that they serve in the codebase today, to discourage others from using them for more kinds of operations we'd rather not support.
These operations have also been given parameter signatures that match their use cases, to make it so that even somebody determined to abuse them would have to invent suitable arguments out of thin air.
In the case of the declaration-to-index mapping, this change eliminates that acceleration structure, in favor or slightly more complicated (and possibly inefficient, yes) code at the use site.
Over time, it would be good to closely scrutinize each of the use cases that requires more complicated interaction with the members of a `ContainerDecl`, to see whether any of them can be reframed in terms of the more basic operations, or if there is some clean abstraction we can introduce to make operations that mutate the member list feel like... hacky.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Disable Link-Time-Optimization by default
LTO was requested for the release package a while ago.
When we added it, LTO was enabled by default although it was needed only for
the release packages.
Later we found that the Release build cannot be incrementally recompiled
when LTO is enabled. It sometimes works fine, but it required full recompilation
when it doesn't work. We added a new CMake option, `SLANG_ENABLE_RELEASE_LTO`,
to disable it for developers. But many Slang developers don't know the
option exists.
I was going to update the document, CONTRIBUTING.md, but I thought it
will be better to change the default behavior.
* Fix a compiler warning treated as an error on linux
A padding variable was uninitialized, which is fine, but the compiler
was complaining about it.
* Fix other gcc error for uninitialized variable
* Fix more compile warning treated as error
* Fix compiler warning from gcc 11
It appears that this is a valid warning that the `delete this` is done
on an offset 8 when the class uses multiple inheritance.
The compiler warning is following:
```
In file included from /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:5,
from /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:3:
In destructor ‘virtual Slang::ComBaseObject::~ComBaseObject()’,
inlined from ‘uint32_t Slang::ComBaseObject::_releaseImpl()’ at /home/runner/work/slang/slang/source/core/slang-com-object.h:49:16,
inlined from ‘virtual uint32_t Slang::MemoryFileSystem::release()’ at /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:34:5:
/home/runner/work/slang/slang/source/core/slang-com-object.h:33:31: error: ‘void operator delete(void*, std::size_t)’ called on pointer ‘<unknown>’ with nonzero offset 8 [-Werror=free-nonheap-object]
33 | virtual ~ComBaseObject() {}
| ^
In destructor ‘virtual Slang::ComBaseObject::~ComBaseObject()’,
inlined from ‘uint32_t Slang::ComBaseObject::_releaseImpl()’ at /home/runner/work/slang/slang/source/core/slang-com-object.h:49:16,
inlined from ‘virtual uint32_t Slang::MemoryFileSystem::release()’ at /home/runner/work/slang/slang/source/core/slang-memory-file-system.h:34:5,
inlined from ‘Slang::ComPtr<T>::~ComPtr() [with T = ISlangMutableFileSystem]’ at /home/runner/work/slang/slang/include/slang-com-ptr.h:113:34,
inlined from ‘void _modulePtr_impl(UnitTestContext*)’ at /home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:92:1:
/home/runner/work/slang/slang/source/core/slang-com-object.h:33:31: error: ‘void operator delete(void*, std::size_t)’ called on pointer ‘<unknown>’ with nonzero offset 8 [-Werror=free-nonheap-object]
33 | virtual ~ComBaseObject() {}
| ^
/home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp: In function ‘void _modulePtr_impl(UnitTestContext*)’:
/home/runner/work/slang/slang/tools/slang-unit-test/unit-test-module-ptr.cpp:36:69: note: returned from ‘void* operator new(std::size_t)’
36 | ComPtr<ISlangMutableFileSystem>(new Slang::MemoryFileSystem());
| ^
```
The problem is on the fact that `ComBaseObject` is not the first in the
multiple inheritance:
```
class MemoryFileSystem : public ISlangMutableFileSystem, public ComBaseObject
{
public:
// ISlangUnknown
SLANG_COM_BASE_IUNKNOWN_ALL
```
It should be:
```
class MemoryFileSystem : public ComBaseObject, public ISlangMutableFileSystem
```
The chain of ComObject release is little complicated and it is easy to
make a mistake. Here is summary with details,
1. `release()` is declared as a pure-virtual in ISlangUnknown, which is
one of the base classes of `ISlangMutableFileSystem`.
```
struct ISlangUnknown
{
virtual SLANG_NO_THROW uint32_t SLANG_MCALL release() = 0;
```
2. `release()` is implemented with the macro
`SLANG_COM_BASE_IUNKNOWN_RELEASE`.
```
SLANG_NO_THROW uint32_t SLANG_MCALL release() SLANG_OVERRIDE \
{ \
return _releaseImpl(); \
}
inline uint32_t ComBaseObject::_releaseImpl()
{
// Check there is a ref count to avoid underflow
SLANG_ASSERT(m_refCount != 0);
const uint32_t count = --m_refCount;
if (count == 0)
{
delete this;
}
return count;
}
```
3. The instance of `MemoryFileSystem` is handled by ComPtr. And
`ComPtr::~ComPtr()` calls the `release()`.
```
ComPtr<ISlangMutableFileSystem> memoryFileSystem =
ComPtr<ISlangMutableFileSystem>(new Slang::MemoryFileSystem());
SLANG_FORCE_INLINE ~ComPtr()
{
if (m_ptr)
((Ptr)m_ptr)->release();
}
```
4. When `delete this` is called, because ComBaseObject is not the first
in the multiple inheritance, `this` is 8 byte off from the actual
instance address.
A fix for this is to change the order of the inheritance and make
ComBaseObject to be the first in the order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The files `slang-fossil.{h,cpp}` define a new serialization format that is designed to support data being memory-mapped in and then traversed as-is.
The `docs/design/serialization.md` document was updated with details on this new format.
The `slang-serialize-fossil.{h,cpp}` files define implementations of the recently introduced `ISerializerImpl` interface for reading/writing this new binary format.
The overall structure of these implementations is heavily based on the existing RIFF implementation from `slang-serialize-riff.{h,cpp}`.
Switching the AST serialization over to use this format required almost no changes to `slang-serialize-ast.cpp`.
The new format is more space-efficient than the RIFF-based format in memory (by factor of over 2x), but is actually *worse* than the RIFF-based format in terms of how it affects the size of `slang.dll`, because the new format is seemingly less amenable to LZ4 compression.
A few pieces of utility code were added or moved as part of this work:
* The `core/slang-internally-linked-list.*` implementation is just a type that was used as part of `core/slang-riff.*`, but that wasn't really RIFF-specific.
* The `core/slang-blob-builder.*` files implement a low-level utility for building a binary format in memory out of "chunks". The overall structure of this type is based on the RIFF-specific builder implementation, but has been generalized so that it should apply to other kinds of binary serialization.
* The `core/slang-relative-ptr.h` file implements a simple relative pointer type, which is currently only used by the `slang-fossil.h` format.
If there are concerns about adopting the new format immediately for the AST, this change could be modified to introduce all the new code, but leave the AST serialization using the previous RIFF-based format.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Language version + tuple syntax.
* Fix compile error.
* regenerate documentation Table of Contents
* Fix.
* regenerate command line reference
* Fix.
* Fix.
* Fix more test failures.
* revert empty line change,
* Retrigger CI
* #version->#lang
* Update source/core/slang-type-text-util.cpp
Co-authored-by: ArielG-NV <159081215+ArielG-NV@users.noreply.github.com>
* Remove comments.
* Fix parsing logic.
* Fix parser.
* Fix parser.
* update test comment
* Update options.
* regenerate documentation Table of Contents
* regenerate command line reference
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: ArielG-NV <159081215+ArielG-NV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change takes the new approach to serialization that was used for the AST and generalizes it in a few ways:
* The new approach is no longer tangled up with the RIFF format.
The serialization system supports multiple different implementations of the underlying format.
The existing RIFF format is now supported as one back-end, but support for others will follow in subsequent changes.
* The new approach is no longer deeply specialized to AST serialization.
The old code had things like serialization for `List`s and `Dictionary`s, but it was embedded inside the `AST{Encoding|Decoding}Context`, and thus couldn't be leveraged for other serialization tasks.
This change factors out a completely AST-independent `Serializer` implementation, with an `ASTSerializer` layered on top of it to provide the additional context needed.
* There is less duplication of code between reading and writing of serialized data.
The old code had both the `ASTEncodingContext` and `ASTDecodingContext`, with serialization logic for most types being implemented in both, but with the constraint that those implementations needed to be kept in sync to avoid serialization-related runtime failures.
A key property of the revamped approach is that a single `serialize()` method for a type implements both the reading and writing directions of serialization.
|
| | |
|
| |
|
|
|
| |
(#7059)
Co-authored-by: Julius Ikkala <julius.ikkala@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fixes related to AST serialization change
There are two fixes included here.
The smaller fix is in `slang-ast-decl.h`, where the `CallableDecl::primaryDecl` and `::nextDecl` fields need to be serialized to make sure that we can properly deserialize a module that contains any function redeclarations.
The larger fix is that the `Encoder` and `Decoder` types used to serialize out the AST nodes in a JSON-like hierarchy were being very strict about matching of integer types, which causes problems in any case where serialization code might use a type that is 32-bit on some targets and 64-bit on others, if serialized modules are ever created on one of those targets and consumed on the other (which happens for the core module for some of our targets).
The fix that this change implements is to make the serialization logic there more forgiving, and thus more robust:
* In the writing/encoding direction, the logic now looks at the actual value being encoded to decide whether to write it as a 32- or 64-bit value.
* In the reading/decoding direction, the logic handles the presence of *any* of the FOURCCs that are used to encode integers, for whatever type is being read. As a small bit of safety, there is a dynamic check made for cases where a value would be read with a different sign than it was written with.
The actual logic in `slang-serialize-ast.cpp` is largely unchanged (it continues to use pointer-sized integers in certain cases), but it should not cause problems because it bottlenecks through the `Encoder`/`Decoder` methods that were changed. The only fix made in the AST serialization itself is to account for all of the FOURCCs that can represent integers when peeking at the input to decide whether a `DeclBase` is represented as an indirection to a `Decl`, or is serialized inline (as a `DeclGroup`).
* format code
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* A new approach to AST serialization
This change completely overhauls the way that AST nodes are being serialized, and the offline source-code generation steps that enable that serialization.
In practice, this ends up being a complete overhaul of the way that *modules* are being serialized (not just the AST part), although things like the serialization format for the Slang IR and for source locations are not affected.
The rest of this commit message is broken down in to sections, in an attempt to help guide anybody looking at the code in how to make sense of all the changes.
The Old C++ Extractor
---------------------
AST serialization used to be driven by information scraped using the `slang-cpp-extractor` tool, which did an ad hoc parse of the C++ declarations of the AST node types and then generated a set of "X macros" that could be for macro-based code generation within the rest of the compiler.
While the existing approach was functional, it wasn't easy to understand or maintain, and it has been getting in the way of forward progress on other features we'd like to work on in the language and compiler.
This change removes the `slang-cpp-extractor` tool entirely.
Marking Up the AST Declarations
-------------------------------
The most notable change that contributors to the compiler may notice is the large number of invocations of a macro `FIDDLE()` on the declarations of the AST node types.
The basic idea is that only declarations (namespaces, types, fields) that are preceded by `FIDDLE()` are visible to the code generator tool.
So if somebody is working with the AST and wondering why a new node type isn't working, or why a field they added isn't being serialized correctly, it is probably because they need to add `FIDDLE()` in front of it.
Generating the Boilerplate Code
-------------------------------
The file `slang-ast-boilerplate.cpp` provides a good example of how the information extracted from the marked-up AST declarations gets used.
In that file, the `FIDDLE TEMPLATE` construct is used to generate type information for each of the AST node types.
Similar logic is used in `slang-ast-forward-declarations.h` to generate the declaration of the `ASTNodeType` enumeration, and forward-declare all the AST node classes.
For many parts of the code, simply including that file replaces the need for the old `slang-generated-*.h` files.
Replacing Visitors and Related Logic
------------------------------------
The old visitor types for the AST used the macros that were generated by `slang-cpp-extractor`, so something new was needed to replace them.
The same goes for the `SLANG_AST_NODE_VIRTUAL_CALL` macros.
The core of the solution implemented here is in `slang-ast-dispatch.h`.
Given a "dispatchable" AST node type (say, `Expr`), a call like:
```
ASTNodeDispatcher<Expr,R>(expr, [&](auto e) { return doSomething(e); })
```
is an expression of type `R`, which does the equivalent of something like:
```
switch(expr->getTag())
{
case ASTNodeType::VarExpr: return doSomething(static_cast<VarExpr*>(expr));
// ...
}
```
The `SLANG_AST_NODE_VIRTUAL_CALL` macro is now implemented in terms of `ASTNodeDispatcher`.
The implementation of the visitor types is more involved.
The code in this change retains some of the macro names from the original version, just to try and make the parallels more clear.
The visitor types are all implemented on top of the `ASTNodeDispatcher` approach, and use `FIDDLE TEMPLATE` to generate all the boilerplate `visit*()` method declarations.
Refactoring of `Linkage` Module Loading
---------------------------------------
Needing to revisit all the places where modules get deserialized made it clear that there is a lot of complexity and apparent duplication in the core routines on the `Linkage` that get used for loading modules.
This change tries to clean up some of that logic, but it is worth noting that there are two legacy features that get in the way of making things as clean as they should be:
* The `LoadedModuleDictionary` type that gets passed around a lot exists entirely to handle the corner case where somebody uses the Slang API to perform a compilation with multiple `TranslationUnitRequest`s in the same `FrontEndCompileRequest`, and one of the translation units `import`s the module defined by another of the translation units.
* There are a lot of special-case behaviors and routines entirely there to support the `ModuleLibrary` feature, although that feature should be considered deprecated (or at least subject to getting entirely re-designed down the line).
The basic idea of the cleanup is that all of the (non-deprecated) ways load a module from a serialized binary, or compile one from source should now bottleneck through `loadModuleImpl`, which then bifurcates into `loadSourceModuleImpl` for the compilation case and `loadBinaryModuleImpl` for the deserialization case.
High-Level Serialization Approach
---------------------------------
The old serialization logic used the [RIFF](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) format to encode the high-level structure of things, and this change retains that usage (and actually doubles down on the RIFF usage).
The old serialization system relied on the idea that for any given type `Foo` that wants to support serialization, there should be something like a `SerialFooData` type in C++, that can represent the state of a `Foo`, and then the actual serialization applied to that `SerialFooData`. This means that in most cases there are four pieces of code written:
* During serialization:
* Copying the data of a `Foo` in memory over to a `SerialFooData` in memory
* Writing the state of a `SerialFooData` into the serialized data stream
* During deserialization:
* Reading the state of a `SerialFooData` from a serialized data stream
* Copying the data of the `SerialFooData` in memory over to a `Foo`
The new logic gets rid of the intermediate `SerialFooData`.
In the serialization direction, we take a `Foo` and write it to the `RIFFContainer` directly, or using some other utilities layered on top of it.
In the deserialization direction, we have additional flexibility. Given a `RIFFContainer::Chunk*` that represents a serialized `Foo`, we often navigate through the in-memory representation of the RIFF data to get to the parts of the serialized value that we actually want/need, without needing to deserialize the entire `Foo`.
To support this kind of operation, this change introduces a few helper types like `ContainerChunkRef` an `ModuleChunkRef`, that are little more than typed wrappers around a `RIFFContainer::Chunk*`.
The Module "Container" Part
---------------------------
A serialized `Module` is encoded as a RIFF chunk, using logic in `slang-serialize-container.cpp` - both before and after this change.
This change reorganizes a lot of the code in that file, to account for the way that eliminating the intermediate `SerialContainerData` type streamlines the overall task of writing out the parts of the module.
In the deserialization logic... there isn't really much to do in `slang-serialize-container.cpp`. Most of the logic in `slang.cpp` and `slang-module-library.cpp` that pertains to deserializing modules uses the `ModuleChunkRef`-based approach, and simply extracts the pieces of the serialized module that it needs.
The Actual Serialization of the AST
-----------------------------------
The actual AST serialization logic is in `slang-serialize-ast.cpp`.
The basic approach in both the writing and reading directions is:
* Use the `FIDDLE TEMPLATE` system to generate a set of functions, one for each AST node type, that recursively invoke the read/write logic on each field of that node (after recursively invoking the case for its direct superclass)
* Use the `ASTNodeDispatcher` system to dispatch out to those functions whene reading or writing anything derived from `NodeBase`
* For now, handle all types *not* derived from `NodeBase` by hand.
There's a lot of room for improvement around that last item: it should be just as easy to generate the serialization and deserialization logic for other types that don't inherit from `NodeBase`, but the current change tries to err on the side of making the logic as explicit and simplistic as possible, rather than trying to get too clever too soon.
The actual serialization *format* used for the AST is almost comically simplistic: the code uses hierarchical RIFF chunks to emulate a JSON-like structure. This is a very wasteful representation (e.g., a `bool` or a null pointer each take up *8 bytes*), but the goal for now is to start with the simplest thing that could possibly work, and only add more cleverness once we are sure it won't get in the way of important future improvements (like lazy/on-demand deserialization or IR and AST, to improve compiler startup times).
The files `slang-serialize.{h,cpp}` have been co-opted to define a new pair of types `Encoder` and `Decoder` that are used for a more-or-less stream-oriented way or reading or writing RIFF chunks for the JSON-like structure.
Almost everything related to the actual AST serialization could do with a cleanup pass, and some time spent on picking good/better names for everything.
Smaller Stuff
-------------
* Cleaned up a lot of code that was using bare `ASTNodeType` or the extractor's `ReflectClassInfo` type to consistently use `SyntaxClass`.
* Fixed an apparent bug in how the destination-driven code genarator was handling `TryExpr`s
* Fixed an apparent bug in how the GLSL legalization pass was handling translation of certain `SV_*` semantics.
* format code
* fixup: template errors caught by non-VS compilers
* format code
* fixup: more template errors
* fixup: more stuff VS didn't catch
* fixup: it's amazing VS doesn't catch these...
* fixup: yet more template stuff VS ignores
* fixup: more VS template nonsense
* fixup: unreachable return macro usage
* fixup: more unreacable returns
* fixup: unused parameter
* fixup: strict aliasing
* fixup: allow missing entry point list chunk
* fixup: wasm build script
* fixup: AST changes since this PR was created
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>
|
| |
|
|
|
|
|
| |
* format
* Minor test fixes
* enable checking cpp format in ci
|
| |
|
|
|
|
|
|
|
| |
* Fix issue with failing tests
tests/serialization/serialized-module-test.slang
tests/serialization/extern/extern-test.slang
* Fix issue with session destruction order on Session.
* Improve comment.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP source map.
* Split out handling of RttiTypeFuncs to a map type.
* Make RttiTypeFuncsMap hold default impls.
* Slightly more sophisticated RttiTypeFuncsMap
* Source map decoding.
* Fix tabs.
* Fix asserts due to negative values.
* Use less obscure mechanisms in SourceMap.
* Source map decoding.
Simplifying SourceMap usage.
* First attempt at ouputting a source map as part of emit.
* Added support for -source-map option. SourceMap is added to the artifact.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Changed all getEntryPointCode calls to use RendererBase::getEntryPointCodeFromShaderCache
* Hashing hooked up, tests pass but need to add more to fully test functionality
* checkpoint
* Checkpoint: File system creation seems functional, saving is broken
* checkpoint: Fixed filename generation from MD5 hash, shader blob might be going missing ahead of pipeline state creation
* Fixed a lot of bugs related to hash code generation, shader cache is likely working but needs further testing
* Added workaround for module loading by re-creating the test device, shader cache test functional
* Vulkan shader caching bug fixed, checkpoint commit before more refinement
* pre-ToT merge checkpoint
* checkpoint commit, improving cache keys
* Significantly expanded items included in the dependency hash for Module; Added dependency hash functions to SpecializedComponentType and RenamedEntryPointComponentType
* Temporarily disable shader cache test
* Mid cleanup changes, solution successfully builds
* Added several helper update functions to slang-md5 to help simplify usage; Added a function under ISession to compute a hash for all linkage-related items; Function renames and cleaned up some comments
* Ran premake.bat; Renamed getASTBasedHashCode to computeASTBasedHash
* Added slang unit tests for Checksum and MD5; Extended gfx shader cache test to test with multiple shader files and one shader file with multiple entry points
* Solution builds and shader cache tests pass, but at least a couple other tests now failing
* ran premake.bat
* More cleanup changes
* Added shaderCachePath field to IDevice desc in gfx.slang, gfx-smoke.slang should be functional
* ran premake
* cleanup changes; Adding test printf to getEntryPointCodeFromShaderCache to see if output can be seen in CI
* Removed debugging printfs; Added handling for getEntryPointCode() failing
* Cleanup changes; Jonathan's fixes to SerialWriter to zero initialize otherwise uninitialized memory; Change to SwizzleExpr creation to zero initialize elementCount
* Changed enable_if_t to enable_if
* Fixed enable_if
* Added test for import vs include and changes to included and imported files; Fixed build errors in CUDA; Renamed shader cache statistics fields
* cleanup changes
* Readd removed file
* Restructured computeDependencyBasedHash calls, added computeDependencyBasedHashImpl to all classes dervied from ComponentType
* Applied same restructuring to the AST hash functions
* Cleanup changes; Moved HashBuilder out to slang-digest.h and added some helper functions to streamline the process of adding items to a hash
* Cleanup; Fixed incorrect expected results for shader import and include test
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Compile to a dxil library.
* Added CompileProduct.
* Support handling of ModuleLibrary.
* CacheBehavior -> Cache
* Use CompileProduct for -r references.
* CompileProduct -> Artifact.
* Determining an artifact type on binding.
* Determine binary linkability.
* Added Artifact::exists.
* Added ArtifactKeep.
* Small fixes.
* Small improvements to Artifact.
* Add zip extension.
* Fix some comments.
* Fix multiple adding of PublicDecoration.
Make public output export for DXIL/lib.
Add checking for simpleDecorations such that only added once.
* Use 'whole program' to identify library build.
* Move slang-artifact into compiler-core.
* Split out Keep free functions.
* Artifact::Keep -> ArtifactKeep.
* Handle libraries as artifacts.
* Add -target dxil so test infrastructure knows it needs DXC.
* Linking working in DXC.
* Improve handling around emit for 'export'.
* Add comment around Artifact name.
* Render test working with linking.
* Improvements around Artifact handling.
* Add ArtifactPayloadInfo.
* Small tidy up around artifact.
* Split out code to get info about Artifacts into artifact-info.cpp/.h
* IArtifact interface and IArtifactInstance interface.
* Fix small issues.
* Fix compilation warning issue.
* Fix missing SLANG_OVERRIDE.
* Small fixes to make compilation work on Visual Studio 2022.
* Small improvements to Artifact interface/naming.
* Added Desc with each element in IArchive to allow more flexibility in usage.
* Fix clang warning issue.
* Add ArtifactPayload::Diagnostics
* More discussion around IArtifact usage.
* Re-add slang-artifact.h which was removed during merge.
* Fix typo identified in review.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Fix handling of access modifiers inside type definition.
* Fix access problem for AST node.
Make dumping produce a single function with switch, to potentially make available without Dump specific access.
* WIP on serialization design doc.
* Remove project references to previously generated files.
* More docs on serialization design.
* Improve serialization documentation.
Remove unused function from IRSerialReader.
* Small fixes around naming. Remove long comment from slang-serialize.h - as covered in serialization.md
* Remove long comment in slang-serialize.h as covered in serialization.md
* More information about doing replacements on read for AST and problems surrounding.
* Typo fix.
* Spelling fixes.
* Value serialize.
* Value types with inheritence.
* Use value reflection serial conversion for more AST types
* Use automatic serialization on more of AST.
* Get the types via decltype, simplifies what the extractor has to do.
* Update the serialization.md for the value serialization.
* Small doc improvements.
* Update project.
* Remove ImportExternalDecl type
Added addImportSymbol and ImportSymbol type
Fixed bug in container which meant it wouldn't read back AST module
* Because of change of how imports and handled, store objects as SerialPointers.
* First pass symbol lookup from mangled names.
* Cache current module looked up from mangled name.
* Fix SourceLoc bug.
Improve comments.
* Added diagnostic on mangled symbol not being found
* Fix typo.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Added CharUtil.
Added TypeSet to extractor.
First pass at being able to specify all headers for multiple output headers.
* Fix includes for new C++ extractor convension.
Update premake5 to use new extractor mechanisms.
* Small improvements around StringUtil.
* Split out NameConventionUtil.
* Use a 'convert' to convert between convention types.
* Fix output of build message for C++ extractor.
Improve NameConventionUtil interface.
* Improve comments.
* Fix warning on gcc.
* Fix clang warning.
* Fix some typos in NameConventionUtil.
* Small fix to premake5.lua
* Fix generated includes.
* Remove m_reflectType as no longer applicable with TypeSet.
* Fix .gitignore for slang-generated-* files.
Added getConvention to determine convention from slice.
Add versions of split and convert that infer the from convention
* Fix typo in spliting camel.
* LineWhitespace -> HorizontalWhitespace
* Improve CharUtil comments.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* First pass at generalizing serializer.
* Split out ReflectClassInfo
* Use the general ReflectClassInfo
* Fix some typos in debug generalized serialization.
* Add calculation of classIds.
Make distinct addCopy/add on SerialClasses.
* Write up of more generalized serialization
* WIP to transition from ASTSerialReader/Writer etc to generalized SerialReader/Writer and associated types.
* Improvements to SerialExtraObjects.
Keep RefObjects in scope in factory
* Compiles with Serial refactor - doesn't quite work yet.
* First pass serialization appears to work with refector.
* Split out type info for general slang types.
* Split out slang-serialize-misc-type-info.h
* DebugSerialData -> SerialSourecLocData
DebugSerialReader -> SerialSourceLocReader
DebugSerialWriter -> SerialSourceLocWriter
* Remove unused template that only compiles on VS.
* Fix warning around unused function on non-VS.
* Improve output of type names that are in scopes in C++ extractor.
Update premake5.lua to run generation for RefObject derived types.
* C++ extractor working on RefObject type.
* Split out serialization functionality that spans different types into slang-serialization-factory.cpp/.h
Put AST type info into header.
Removed RefObjectSerialSubType - use RefObjectType
Add filtering for RefObject derived types
Remove construction and filteringhacks.
* Set up field serialization for SerialRefObject derived types.
* Fix template problem compiling on Clang/Gcc
* Work in progress to make Value types work.
* Added slang-value-reflect.cpp
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* First pass at generalizing serializer.
* Split out ReflectClassInfo
* Use the general ReflectClassInfo
* Fix some typos in debug generalized serialization.
* Add calculation of classIds.
Make distinct addCopy/add on SerialClasses.
* Write up of more generalized serialization
* WIP to transition from ASTSerialReader/Writer etc to generalized SerialReader/Writer and associated types.
* Improvements to SerialExtraObjects.
Keep RefObjects in scope in factory
* Compiles with Serial refactor - doesn't quite work yet.
* First pass serialization appears to work with refector.
* Split out type info for general slang types.
* Split out slang-serialize-misc-type-info.h
* DebugSerialData -> SerialSourecLocData
DebugSerialReader -> SerialSourceLocReader
DebugSerialWriter -> SerialSourceLocWriter
* Remove unused template that only compiles on VS.
* Fix warning around unused function on non-VS.
|
|
|
* Test if blob is returned.
* Rename serialize files so can be grouped.
* StringRepresentationCache -> SerialStringTable
* Split out SerialStringTable from slang-serialize-ir
* First pass at reorganizing serialization/containers. Remain some issues about debug info.
* Fix bug in calculating sourceloc.
* Improve calcFixSourceLoc
* Make allocations for payload RiffContainer align to at least 8 bytes. This is important for read, if the payload can contain 8 byte aligned data. Note this has no effect on Riff file format alignment rules.
* Improve comments around RiffContainer and alignment.
* Remove SerialStringTable, can just use StringSlicePool instead.
* Typo fix for Clang/Linux.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
