| Age | Commit message (Collapse) | Author |
|---|
|
The documentation added by #6844 included instructions to make sure that
the Fiddle `#include` in a file comes after all the other `#include`s,
but it's easy to accidentally violate this via `clang-format`, as
happened for `source/slang/slang-ast-modifier.h` in #7559. This PR
guards against this sort of violation by separating all Fiddle
`#include`s from other `#include`s via a blank line followed by a `//`
line (as we already do in most cases), and also adds a sentence about
this in `tools/slang-fiddle/README.md`.
As a bonus, I also enabled Markdown syntax highlighting for all the code
blocks in that doc file.
Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
|
|
Closes #8061.
Along with the fix, also enhanced coercion/overload resolution to filter
candidates based on the target type, allowing
`tests\language-feature\higher-order-functions\overloaded.slang` to
pass.
|
|
The `SLANG_UNREFLECTED` macro has been completely meaningless since we switched away from the old AST serialization/reflection approach, so the lingering uses of it in the code as pointless at best and misleading/confusing at worst.
|
|
* Implement throw statement
It already existed in the IR, so only parsing, checking and lowering was
missing.
* Initial catch implementation
Likely very broken.
* Error out when catch() isn't last in scope
* Prevent accessing variables from scope preceding catch
As those may actually not be available at that point.
* Add IError and use it in Result type lowering
* Add diagnostic tests
* Allow caught throws in non-throw functions
* Fix catch propagating between functions & SPIR-V merge issue
* Add test for non-trivial error types
* Fix MSVC build
* Fix invalid value type from Result lowering
* Also lower error handling in templates
* Lower result types only after specialization
* Attempt to disambiguate error enums by witness table
* Revert matching by witness, types should be distinct too
* Don't assert valueField when getting Result's error value
It may not exist if the function returns void, but getting the error
value is still legitimate.
* Update tests for new error numbers & get rid of expected.txt
* Change catch lowering to resemble breaking a loop
... To make SPIR-V happy.
* Fix dead catch blocks and invalid cached dominator tree
* More SPIR-V adjustment
* Lower catch as two nested loops
* Add defer interaction test and revert broken defer changes
* Fix enum type when throwing literals
* Cleanup and bikeshedding
* Document error handling mechanism
* Fix table of contents
* Use boolean tag in Result<T, E>
* Use anyValue storage for Result<T,E>
* Remove IError
* Fix formatting
* Eradicate success values from docs and tests
* Use parseModernParamDecl for catch parameter
* Implement do-catch syntax
* Implement catch-all
* Fix formatting
* Fix marshalling native calls that throw
---------
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* A new approach to AST serialization
This change completely overhauls the way that AST nodes are being serialized, and the offline source-code generation steps that enable that serialization.
In practice, this ends up being a complete overhaul of the way that *modules* are being serialized (not just the AST part), although things like the serialization format for the Slang IR and for source locations are not affected.
The rest of this commit message is broken down in to sections, in an attempt to help guide anybody looking at the code in how to make sense of all the changes.
The Old C++ Extractor
---------------------
AST serialization used to be driven by information scraped using the `slang-cpp-extractor` tool, which did an ad hoc parse of the C++ declarations of the AST node types and then generated a set of "X macros" that could be for macro-based code generation within the rest of the compiler.
While the existing approach was functional, it wasn't easy to understand or maintain, and it has been getting in the way of forward progress on other features we'd like to work on in the language and compiler.
This change removes the `slang-cpp-extractor` tool entirely.
Marking Up the AST Declarations
-------------------------------
The most notable change that contributors to the compiler may notice is the large number of invocations of a macro `FIDDLE()` on the declarations of the AST node types.
The basic idea is that only declarations (namespaces, types, fields) that are preceded by `FIDDLE()` are visible to the code generator tool.
So if somebody is working with the AST and wondering why a new node type isn't working, or why a field they added isn't being serialized correctly, it is probably because they need to add `FIDDLE()` in front of it.
Generating the Boilerplate Code
-------------------------------
The file `slang-ast-boilerplate.cpp` provides a good example of how the information extracted from the marked-up AST declarations gets used.
In that file, the `FIDDLE TEMPLATE` construct is used to generate type information for each of the AST node types.
Similar logic is used in `slang-ast-forward-declarations.h` to generate the declaration of the `ASTNodeType` enumeration, and forward-declare all the AST node classes.
For many parts of the code, simply including that file replaces the need for the old `slang-generated-*.h` files.
Replacing Visitors and Related Logic
------------------------------------
The old visitor types for the AST used the macros that were generated by `slang-cpp-extractor`, so something new was needed to replace them.
The same goes for the `SLANG_AST_NODE_VIRTUAL_CALL` macros.
The core of the solution implemented here is in `slang-ast-dispatch.h`.
Given a "dispatchable" AST node type (say, `Expr`), a call like:
```
ASTNodeDispatcher<Expr,R>(expr, [&](auto e) { return doSomething(e); })
```
is an expression of type `R`, which does the equivalent of something like:
```
switch(expr->getTag())
{
case ASTNodeType::VarExpr: return doSomething(static_cast<VarExpr*>(expr));
// ...
}
```
The `SLANG_AST_NODE_VIRTUAL_CALL` macro is now implemented in terms of `ASTNodeDispatcher`.
The implementation of the visitor types is more involved.
The code in this change retains some of the macro names from the original version, just to try and make the parallels more clear.
The visitor types are all implemented on top of the `ASTNodeDispatcher` approach, and use `FIDDLE TEMPLATE` to generate all the boilerplate `visit*()` method declarations.
Refactoring of `Linkage` Module Loading
---------------------------------------
Needing to revisit all the places where modules get deserialized made it clear that there is a lot of complexity and apparent duplication in the core routines on the `Linkage` that get used for loading modules.
This change tries to clean up some of that logic, but it is worth noting that there are two legacy features that get in the way of making things as clean as they should be:
* The `LoadedModuleDictionary` type that gets passed around a lot exists entirely to handle the corner case where somebody uses the Slang API to perform a compilation with multiple `TranslationUnitRequest`s in the same `FrontEndCompileRequest`, and one of the translation units `import`s the module defined by another of the translation units.
* There are a lot of special-case behaviors and routines entirely there to support the `ModuleLibrary` feature, although that feature should be considered deprecated (or at least subject to getting entirely re-designed down the line).
The basic idea of the cleanup is that all of the (non-deprecated) ways load a module from a serialized binary, or compile one from source should now bottleneck through `loadModuleImpl`, which then bifurcates into `loadSourceModuleImpl` for the compilation case and `loadBinaryModuleImpl` for the deserialization case.
High-Level Serialization Approach
---------------------------------
The old serialization logic used the [RIFF](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) format to encode the high-level structure of things, and this change retains that usage (and actually doubles down on the RIFF usage).
The old serialization system relied on the idea that for any given type `Foo` that wants to support serialization, there should be something like a `SerialFooData` type in C++, that can represent the state of a `Foo`, and then the actual serialization applied to that `SerialFooData`. This means that in most cases there are four pieces of code written:
* During serialization:
* Copying the data of a `Foo` in memory over to a `SerialFooData` in memory
* Writing the state of a `SerialFooData` into the serialized data stream
* During deserialization:
* Reading the state of a `SerialFooData` from a serialized data stream
* Copying the data of the `SerialFooData` in memory over to a `Foo`
The new logic gets rid of the intermediate `SerialFooData`.
In the serialization direction, we take a `Foo` and write it to the `RIFFContainer` directly, or using some other utilities layered on top of it.
In the deserialization direction, we have additional flexibility. Given a `RIFFContainer::Chunk*` that represents a serialized `Foo`, we often navigate through the in-memory representation of the RIFF data to get to the parts of the serialized value that we actually want/need, without needing to deserialize the entire `Foo`.
To support this kind of operation, this change introduces a few helper types like `ContainerChunkRef` an `ModuleChunkRef`, that are little more than typed wrappers around a `RIFFContainer::Chunk*`.
The Module "Container" Part
---------------------------
A serialized `Module` is encoded as a RIFF chunk, using logic in `slang-serialize-container.cpp` - both before and after this change.
This change reorganizes a lot of the code in that file, to account for the way that eliminating the intermediate `SerialContainerData` type streamlines the overall task of writing out the parts of the module.
In the deserialization logic... there isn't really much to do in `slang-serialize-container.cpp`. Most of the logic in `slang.cpp` and `slang-module-library.cpp` that pertains to deserializing modules uses the `ModuleChunkRef`-based approach, and simply extracts the pieces of the serialized module that it needs.
The Actual Serialization of the AST
-----------------------------------
The actual AST serialization logic is in `slang-serialize-ast.cpp`.
The basic approach in both the writing and reading directions is:
* Use the `FIDDLE TEMPLATE` system to generate a set of functions, one for each AST node type, that recursively invoke the read/write logic on each field of that node (after recursively invoking the case for its direct superclass)
* Use the `ASTNodeDispatcher` system to dispatch out to those functions whene reading or writing anything derived from `NodeBase`
* For now, handle all types *not* derived from `NodeBase` by hand.
There's a lot of room for improvement around that last item: it should be just as easy to generate the serialization and deserialization logic for other types that don't inherit from `NodeBase`, but the current change tries to err on the side of making the logic as explicit and simplistic as possible, rather than trying to get too clever too soon.
The actual serialization *format* used for the AST is almost comically simplistic: the code uses hierarchical RIFF chunks to emulate a JSON-like structure. This is a very wasteful representation (e.g., a `bool` or a null pointer each take up *8 bytes*), but the goal for now is to start with the simplest thing that could possibly work, and only add more cleverness once we are sure it won't get in the way of important future improvements (like lazy/on-demand deserialization or IR and AST, to improve compiler startup times).
The files `slang-serialize.{h,cpp}` have been co-opted to define a new pair of types `Encoder` and `Decoder` that are used for a more-or-less stream-oriented way or reading or writing RIFF chunks for the JSON-like structure.
Almost everything related to the actual AST serialization could do with a cleanup pass, and some time spent on picking good/better names for everything.
Smaller Stuff
-------------
* Cleaned up a lot of code that was using bare `ASTNodeType` or the extractor's `ReflectClassInfo` type to consistently use `SyntaxClass`.
* Fixed an apparent bug in how the destination-driven code genarator was handling `TryExpr`s
* Fixed an apparent bug in how the GLSL legalization pass was handling translation of certain `SV_*` semantics.
* format code
* fixup: template errors caught by non-VS compilers
* format code
* fixup: more template errors
* fixup: more stuff VS didn't catch
* fixup: it's amazing VS doesn't catch these...
* fixup: yet more template stuff VS ignores
* fixup: more VS template nonsense
* fixup: unreachable return macro usage
* fixup: more unreacable returns
* fixup: unused parameter
* fixup: strict aliasing
* fixup: allow missing entry point list chunk
* fixup: wasm build script
* fixup: AST changes since this PR was created
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* Eliminate back-reference in ChildStmt
This change is part of a larger effort to improve the code for AST
serialization in the Slang compiler.
Tree structures are understandably easier to serialize than DAGs,
and DAGs are easier than fully generaal graphs.
The Slang AST nodes form a tree structure... except when they don't.
Among the exceptions to nice tree-structured ASTs are:
1. References to `Decl`s are encoded as pointers to the AST `Decl`
nodes themselves. This can result in cycles in the graph, and
requires care in serialization.
2. Nodes that inherit from `Val` represent, well, *values* instead
of actual pieces of syntax, and as such they are deduplicated so
that identical values will (hopefully) be identical pointers.
This results in a DAG structure for `Val`s, but at least it's not
a general graph (except for cycles that go through a `Decl`).
3. There are some minor cases of DAG-structured sharing that the
parser can introduce to deal with cases when a traditional-style
declaration includes multiple declarators. E.g., given:
```
static int a, b;
```
The resulting `DeclGroup` will include distinct `Decl`s for `a`
and `b`, which will share the `static` modifier through a
`SharedModifiers` node, and the `int` type specifier through a
`SharedTypeExpr` node.
This duplication can be ignored, for the purposes of serialization,
since duplicating those parts of the AST has no major down-sides.
4. There is the case of `ChildStmt`, used for things like `break`
and `continue`, which stores a direct `Stmt*` to the enclosing
parent statement being targetted. Storing the target is useful so
that IR lowering doesn't need to repeat the work that the semantic
checking logic did to associate each child statement with its parent.
The parent link inside of `ChildStmt` creates a cycle in the AST
`Stmt` hierarchy, since the outer statement contains the inner,
and the inner statement stores a pointer to the outer.
This change eliminates the last of these sources of complication for
AST serialization, by changing the `ChildStmt` type to stored an
integer ID for the enclosing statement that it matches to, and having
each `BreakableStmt` (used to represent the outer `switch`, or loop,
or whatever) generate its own unique ID as part of semantic checking.
Note: if necessary, it is reasonable for the outer statement to have
its unique ID generated as part of parsing, rather than semantic
checking.
* format code
* Change unique ID to be a proper Decl
The fix here is to make the "unique ID" representation be a full
`Decl`-derived AST node, so that it is both allowed to break the
tree-structuring rules cleanly, and it is also trivially guaranteed
to be unique across all loaded ASTs.
* format code
---------
Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
|
|
|
|
* Support stage_switch.
* Update proposal status.
* Fix gl_InstanceID.
* Fix.
|
|
* Use two-stage parsing to disambiguate generic app and comparison.
* Typo fix.
* Update doc.
|
|
* format
* Minor test fixes
* enable checking cpp format in ci
|
|
* Remove use of `G0` and `__target_intrinsic` in stdlib.
* Fix.
* Fix calling intrinsic in global scope.
|
|
* Fix assertions due to malformed switch statements
Fixes the issue #2955
* Checks for multiple case statements with same values
* Checks for multiple default cases
* Constant-folds case exprs into an Integer value
* fix the comments, and updated error code
* one-line comment on diagnostic code
|
|
* Capability type checking.
* Fix.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add `target_switch` and `__intrinsic_asm` statement.
* Cleanup.
* WaveGetActiveMask, WaveGetActiveMask, WaveCountBits.
* WaveIsFirstLane.
* More wave intrinsics.
* wave intrinsics.
* merge fix.
* Fix.
* Fix.
* Update test.
* update test.
* Fix.
---------
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Add syntax for multi-level break.
* Fix.
* Fix.
Co-authored-by: Yong He <yhe@nvidia.com>
|
|
* Bring heterogeneous-hello-world back up to date.
* Reintroduced heterogeneous-hello-world into the premake
* No longer uses compiled bytecode for entry point, instead a loadModule
call is hardocoded with the slang file name.
* Entry point is, similarly, hardcoded for now.
* Added a bypass to slang-legalize-types for an unneeded GPUForeach check
* Run premake and change to relative path
* Removed experimental and added README
Co-authored-by: Yong He <yonghe@outlook.com>
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP: First pass in supporting output of line error information.
* Add support for lexing to better be able to indicate SourceLocation information.
* Fix lexer usage in DiagnosticSink in C++ extractor.
* Update diagnostics tests to have line location info.
* Fixed test expected output that now have source location information in them.
* Better handling of tab.
* Fix test expected results for tabbing change.
* DiagnosticLexer -> DiagnosticSink::SourceLocationLexer
Added line continuation tests.
* Fix typo.
* Added String::appendRepeatedChar
* Change to rerun tests.
* Added source locations to IR dumping.
* Output column for IR dump source loc.
* Add support for closing brace location to AST.
Use closing brace location in lowering when adding return void.
* Set the source location through SourceLoc - simplifies identifying if current loc is valid.
* Copy terminator sloc.
* Test for improved #line handling.
* Made writer the last parameter for dumpIR.
Small improvements to comments.
* Disable sloc output on dump IR by default.
* Fix issue with #line and inlining.
* Fix for output with improved #line output.
* Small comment change - mainly to kick off TC build.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
|
* #include an absolute path didn't work - because paths were taken to always be relative.
* Make AST serialization types, marker include _AST_. Ie SLANG_CLASS -> SLANG_AST_CLASS and SLANG_ABSTRACT_CLASS -> SLANG_ABSTRACT_AST_CLASS
|
|
This PR introduces parsing and semantic checking for a GPU foreach loop
for heterogeneouis programming. A GPU foreach loop takes the form:
```
__GPU_FOREACH(renderer, gridDims, LAMBDA(uint3 dispatchThreadID) {
kernelCall(args, ...); });
```
And will allow the host code to call into a kernel with the correct
renderer and grid dimensions. This commit also introduces a hack to
unify types in the heterogeneous hello world file, which will hopefully
be amended in the future.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
|
* Add a ASTBuilder to a Module
Only construct on valid ASTBuilder (was being called on nullptr on occassion)
* Add nodes to ASTBuilder.
* Compiles with RefPtr removed from AST node types.
* Initialize all AST node pointer variables in headers to nullptr;
* Initialize AST node variables as nullptr.
Make ASTBuilder keep a ref on node types.
Make SyntaxParseCallback returns a NodeBase
* Don't release canonicalType on dtor (managed by ASTBuilder).
* Give ASTBuilders a name and id, to help in debugging.
For now destroy the session TypeCache, to stop it holding things released when the compile request destroys ASTBuilders.
* Moved the TypeCheckingCache over to Linkage from Session.
* NodeBase no longer derived from RefObject.
* Only add/dtor nodes that need destruction.
First pass compile on linux.
|
|
* Compiles.
* Small tidy up around session/ASTBuilder.
* Tests are now passing.
* Fix Visual Studio project.
* Fix using new X to use builder when protectedness of Ctor is not enough.
Substitute->substitute
* Add some missing ast nodes created outside of ASTBuilder.
* Compile time check that ASTBuilder is making an AST type.
* Moced findClasInfo and findSyntaxClass (essentially the same thing) to SharedASTBuilder from Session.
|
|
* Fields from upper to lower case in slang-ast-decl.h
* Lower camel field names in slang-ast-stmt.h
* Fix fields in slang-ast-expr.h
* slang-ast-type.h make fields lowerCamel.
* slang-ast-base.h members functions lowerCamel.
* Method names in slang-ast-type.h to lowerCamel.
* GetCanonicalType -> getCanonicalType
* Substitute -> substitute
* Equals -> equals
ToString -> toString
* ParentDecl -> parentDecl
Members -> members
|
|
SLANG_CLASS, to simplify macro injection and not require macro redefinition in each file. (#1345)
|
|
* Extractor builds without any reference to syntax (as it will be helping to produce this!).
* Change macros to include the super class.
* WIP replacing defs files.
* Added indexOf(const UnownedSubString& in) to UnownedSubString.
Refactored extractor
* Output a macro for each type with the extracted info - can be used during injection in class
* Simplify the header file - as can get super type and last from macro now
* Store the 'origin' of a definition
* Some small tidy ups to the extractor.
* Improve comments on the extractor options.
* Made CPPExtractor own SourceOrigins
* Small fixes around SourceOrigin.
* Small tidy up around macroOrign
* WIP Visitor seems now to work correctly.
Split out types used by ast into slang-ast-support-types.h
* Fix remaining problems with C++ extractor being used with AST nodes.
Add CountOf to extractor type ids.
Added ReflectClassInfo::getInfo to turn an ASTNodeType into a ReflectClassInfo
* Fix compiling on linux.
Fix typo in memset.
* Small tidy up around comments/layout.
Moved NodeBase casting to NodeBase.
* Make premake generate project that builds with cpp-extractor for AST.
* Get the source directory from the filter in premake.
* Fix typo in source path
* Explicitly set the source path for premake generation for AST.
* Special case handling of override to apease Clang.
* Use a more general way to find the slang-ast-reflect.h file to run the extractor.
* Appveyor is not triggering slang-cpp-extractor - try putting dependson together.
* Put building slang-cpp-extractor first.
* Disable some project options to stop MSBuild producing internal compiler errors.
* Try reordering the projects in premake5.lua
* Hack to try and make slang-cpp-extractor built on appveyor.
* Disable flags - not required for MSBuild on appveyor.
* Disable flags not required for build on AppVeyor.
* Updated Visual Studio projects with slang-cpp-extractor.
* Added Visual Studio slang-cpp-extractor project.
|
