| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Cleanup refactoring work around the IR builder
We have some long-term goals for the IR that require a more centralized and disciplined set of rules for how IR instructions get created/emitted. I had been working on trying to set things up so that all IR instruction creation goes through a single bottleneck point, but the non-trivial work in that branch was getting drowned out by the sheer volume of cleanup and refactoring changes. This change tries to pull together several of the more important cleanups.
The big pieces are:
* `IRBuilder` and `SharedIRBuilder` now protect their data members and rely on users to initialize them more directly via constructor of an `init()` method. This change affects a *bunch* of sites where `IRBuilder`s were created. I changed use sites to use the constructors whenever possible, and to use `init()` in cases where we had longer-lived builders that needed to be initialized multiple times.
* The insertion location for the `IRBuilder` now uses an encapsulated type called `IRInsertLoc`. This new type can replace what used to be just two `IRInst*` fields in the builder, and also covers some new functionality (if we ever want to take advantage of it). Very little client code cares about this change, but it is still a nice cleanup in terms of making things more explicit.
* The creation of an `IRModule` has been moded *out* of `IRBuilder`, because in practice we `IRBuilder` always wants to be associated with a pre-existing `IRModule` at creation time (via its `SharedIRBuilder`). There is now an `IRModule::create()` operation instead. This required changing the sequencing at many `IRModule` creation sites, since most had been contriving to make an `IRBuilder` first. There were also several cleanups because code had been carelessly using non-reference-counted pointers for `IRModule`s in ways that broke now that `IRModule::create()` always returns a `RefPtr`.
* The core operations to actually allocate memory for IR instructions were moved into `IRModule` (since they interact with the memory pool that the module owns). These *were* called `createEmptyInst()` but have been renamed into `_allocateInst()`. In principle these seem like they should only be needed to be called by the `IRBuilder`, but in practice they are also needed by the IR deserialization logic.
* A few core operations for emitting IR instructions that were associted with `IRBuilder` were moved to actually be methods on `IRBuilder`. First is `_findOrEmitConstant` which is the primary bottleneck for creating simple scalar constant values. Another is `_createInst` (formerly part of the templated `createInstImpl` along with `createInstWithSizeImpl`) which is the main bottleneck for allocation and initialization of any instruction other than a constant (well, the `IRModuleInst` is the other exception...). Finally, there is also `_maybeSetSourceLoc()`, which is obvious to scope inside the `IRBuilder` once it is protecting the source-location info.
Notes:
* The `minSizeInBytes` parameter to `_createInst()` might not actually be needed at all. At this point any `IRInst` subtypes that need data allocated for things other than their operands already get created manually via `_allocateInst` or `_findOrEmitConstant`, so I *think* we could remove that part. I will handle that in a subsequent cleanup if it turns out to be the case.
* There is one IR pass (`slang-ir-string-hash.cpp`) that is using manual `_allocateInst()` instead of going through an `IRBuilder`. It could be easily cleaned up to not do so (and I will probably make that change down the line), but for now I wanted to avoid doing anything that wasn't close to pure refactoring if I could.
* At this point in our design an `IRBuilder` is a very lightweight thing - it basically just owns the insertion location plus a source location to write into instructions. A lot of our code currently treats `IRBuilder`s like they are expensive and/or need to be re-used (which leads to them being used in more mutable/stateful ways). It is quite likely that as we clean up other aspects of the implementation of IR creation/emission we can make `IRBuilder` use feel more lightweight in ways that can streamline and simplify code.
* The next step for this work is to identify the different paths that eventually lead to `_createInst()` being called, and unify them at a single bottleneck operation that can own the decisions around when to create an instruction vs. when to re-use an existing one (rather than those decisions being baked into the various `IRBuilder` subroutines that create instructions of the various subtypes).
* fixup: gcc/clang C++ spec details
|
| |
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Improvements to repro diagnostics.
* Fix typo.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Use 'Process' to communicate with an command line tool.
* Remove slang-win-stream
* Tidy up windows ProcessUtil.
* First version of BufferedReadStream.
* Windows working IPC for steams.
* Test proxy count option.
* Split Process/ProcessUtil. Process is platform dependant. ProcessUtil are functions that are platform independent.
* First implementation of Unix Process interface.
* Unix process compiles on cygwin.
* Fix typo in unix process.
* Separate unix pipe stream error of invalid access, from pipe availability.
* Fix in standard line extraction.
* Make fd non blocking.
* Fix issues with Windows Process streams.
* Added UnixPipe.
* Some fixes around UnixPipeStream.
* Make a unix stream closed explicit.
* Hack to debug linux process/stream.
* Revert to old linux pipe handling.
* Pass executable path for unit tests.
Split out CommandLine into own source.
* Small improvements in process/command line.
* Check process behavior with crash.
* Make stderr and stdout unbuffered for crash testing.
* Only turn disable buffering in crash test.
* Disable crash test on CI.
* Fix crash on clang/linux.
* Enable crash test.
Remove _appendBuffer as can use StreamUtil functionality.
* Added inital processing for http headers.
* Small improvements to HttpHeader.
* First pass HTTPPacketConnection working on windows.
* Enable other Process communication tests.
* Update comments.
* WIP JSON RPC.
* Add terminate to Process.
Made JSONRPC a Util.
* Small tidy up around HTTPPacketConnection.
* Improve process termination options.
* WIP for test-server.
* Add diagnostics error handling to test-server.
* Improved JSON support.
Parsing/creating JSON-RPC messages.
* WIP JSONRPC parsing.
* First pass RttiInfo support.
* WIP converting between JSON/native types.
* Project files.
* Split out RttiUtil.
Made RttiInfo constuction thread safe.
* WIP RTTI<->JSON.
* Add diagnostics to JSON<->native conversions.
* Make RttiInfo for structs globals. Avoids problem around derived types (like pointers), being able to cause an abort.
* Add pointer support to RTTI.
Fixed some compilation issues on linux.
* Add fixed array support.
* Added Rtti unit test.
* Add rtti unit test.
* Split out quoted/unquoted key handling.
Fix bugs in JSON value/container.
Added JSON native test.
* Make default array allocator use malloc/free.
Remove the new[] handler (doesn't work on visuals studio).
* Fix for linux warning.
* Remove some test code.
* Fix issues on x86 win.
* Fix warning on aarch64.
* Fix some bugs in JSON parsing/handling.
Make Rtti work copy/dtor/ctor struct types.
* Testing JSON<->native with fixed array.
Make makeArrayView explicit if it's just a single value.
Added array type.
* Fix getting arrayView.
* Improve JSON diagnostic name.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Add support for LLVM for host callable.
Added CodeGenTransitionMap.
* Remove hack to enable host callable for LLVM.
* Small improvements around transitions/downstream compiler.
* Fix typo in method name.
* Fix comment.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Refactor Stream. Working on all tests.
* Split out CharEncode.
* Make method names lower camel.
m_prefix in Writer/Reader
* Tidy up around CharEncode interface.
* Small improvements around encode/decode.
* Better use of types.
* Remove readLine from TextReader.
* Remove exceptions from Stream/Text handling.
* Fix some typos.
* Fix tabbing.
* Fix missing override.
* Remove remaining exception throw/catch via using signal mechanism.
* Remove exceptions that are not used anymore.
* Document the Stream interface.
* Remove index for decoding 'get byte' function.
* Fix CharReader -> ByteReader.
|
| |
|
| |
Co-authored-by: Yong He <yhe@nvidia.com>
|
| |
|
|
|
|
|
|
|
| |
* Fix AbortCompilationException leaking through loadModule API.
* Update.
* Fix.
Co-authored-by: Yong He <yhe@nvidia.com>
|
| | |
|
| |
|
| |
Co-authored-by: Yong He <yhe@nvidia.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* Add GLSL450 intrinsics to SPIRV direct emit.
* Fix.
* Fix compiler error.
* Fix.
* Fix compiler error.
* Make direct-spirv tests actually run.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Further implementation of SPIRV direct emit.
This change implements:
- Struct, Vector, Matrix and Unsized Array types.
- Basic arithmetic opcodes, vector construct, swizzle etc.
- getElementPtr, getElement, fieldAddress, extractField.
- SPIRV target intrinsics with SPIRV asm code in stdlib.
- RWStructuredBuffer and StructuredBuffer.
- Pointer storage class propagation.
- Control flow.
* Fix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add debug symbols for release build.
* Hack to try and capture failing compilation.
* Typo fix for capture hack.
* Specify return type on lambdas.
* Added const.
* Try breakpoint.
* Up count
* Let's capture everything so we can valgrind.
* Disable always writing repros.
* Make Scope non RefCounted.
* Fix issue with not serializing Scope.
* More comments around changes to Scope.
Remove Scope* from serialization.
* Remove code used for testing original issue.
|
| |
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Remove StructTag and associated systems.
* Fix typo and remove unit test for StructTag.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP Abi struct.
* Use AbiSystem on SessionDesc.
* Use mask/shift constants.
* Fix issue causing warning on linux.
* Abi -> Api.
* Fix typo.
* Refactor to use StructTag.
* Mechanism to be able to follow fields.
* Field adding is working.
* WIP with StructTagConverter.
* First pass of StructTag appears to work. Still needs diagnostics.
* Small tidy up around Field.
* Use bit field to record what fields are recorded to remove allocation around the m_stack.
Use ScopeStack for RAII.
* Return SlangResult instead of pointers.
* Use SlangResult with copy.
* Split StructTagConverter implementations.
* Fix some bugs around lazy converting.
* First pass at unit test for StructTag.
* Testing StructTag going backwards in time.
* First pass as StructTag diagnostics.
* Make Traits a namespace.
* Fix some issues with Traits not being a class.
* Fix 32 bit warning.
|
| |
|
|
|
|
|
|
|
| |
* Properly fill `declref` in `Linkage::getContainerType`.
* Fix timestamp query on cpu
* Fix typo.
Co-authored-by: Yong He <yhe@nvidia.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Include a "stack trace" with nested-import errors
When errors occur in nested `#include` files it is often helpful to have a "stack trace" / traceback of the `#include` chain that led from a root translation unit to the file with an error.
This change implements a similar feature for `import`s.
It is worth noting that `import`s don't really *require* this kind of compiler support the way `#include`s do because the intention is that the meaning of an `import`ed file does not depend on the order or nesting of `import`s. As such, when trying to *fix* an error in an `import`ed file, you usually don't care how it came to be `import`ed into your shaders.
The use case here is somebody adapting a large body of Slang code to use in a different codebase, such that they have certain `.slang` files they don't actually intend to have compile correctly, and they want to be able to diagnose how they came to include those files when/if they cause problems.
The actual feature implementation is pretty simple because we already track a stack of active `import`s so that we can detect and diagnose recursive `import`s. This change simply changes the disagnostics when there is an error in imported code so that instead of just noting the inner-most `import` site it lists all the `import` sites that were active at the time.
The change includes a test case to confirm that the behavior works (at least for the case of a parse error).
* fixup: test outputs
Co-authored-by: Yong He <yonghe@outlook.com>
Co-authored-by: jsmall-nvidia <jsmall@nvidia.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Various Fixes to gfx, reflection and emit.
- Fix GLSL emit to properly output `*bitsTo*` functions for `IRBitCast` insts.
- Add line directive mode setting for `ISession`.
- Extend `TypeLayout::getElementStride` to handle `VectorType` case.
- Fix `IDevice::readBufferResource` 's D3D12 implementation to copy only the requested bytes out.
- Fix `render-test` to use the `ISession` from `gfx` instead of creating its own `ISession` to make sure `gfx` and `render-test` agree on WitnessTable and RTTI IDs.
- Extend `render-test` to support filling vector and matrix values in the new `set x = ...` TEST_INPUT syntax.
- Add a `dynamic-dispatch-15` test case to make sure packing / unpacking works correctly across all targets, and to make sure render-test's RTTI/WitnessTable ID filling logic is correct for non-trivial cases.
* Remove default-major test
* Fix cyclic reference in `ExtendedTypeLayout`.
* Move `lineDirectiveMode` setting to `TargetDesc`.
Add `structureSize` to `TargetDesc` and `SessionDesc` for future binary compatibility.
* Cleanup.
Co-authored-by: Yong He <yhe@nvidia.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Added SourceLoc handling for command line parsing.
* Fix typo in debug.
* Fix issue around the DiagnosticSink used in options parsing not having a writer available - by having DiagnosticSink parenting.
* Small rename for clarity.
* WIP extracting command line args for downstream tools.
* Unit tests/bug fixes around extracting args.
* Use DownstreamArgs in the EndToEndCompileRequest
* Passing downstream compiler options downstream.
* Fix issue with endToEndReq being nullptr.
* Fix issue with diagnostics number change.
* Small improvements to how the source line is displayed if it's too long.
Default to 120, as suggested in previous review.
* Make render test use x-args parsing and CommandArgReader.
* Added missing diagnostics.
* More DownstreamArgs to linkage so can be seen by 'components'.
Added dxc-x-arg test.
* Used combination of name and args instead of two Lists, which whilst equivalent was perhaps a little confusing.
* Added documentation for -X support.
* Added test for x-args parsing diagnostic. Improved diagnostic with list of known names.
* Fix issues from merge.
* Fix lookup for -matrix-layout-column-major in render test.
* Remove commented out line.
|
| |
|
| |
Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Added SourceLoc handling for command line parsing.
* Fix typo in debug.
* Fix issue around the DiagnosticSink used in options parsing not having a writer available - by having DiagnosticSink parenting.
* Small rename for clarity.
* WIP extracting command line args for downstream tools.
* Unit tests/bug fixes around extracting args.
* Use DownstreamArgs in the EndToEndCompileRequest
* Passing downstream compiler options downstream.
* Fix issue with endToEndReq being nullptr.
* Fix issue with diagnostics number change.
* Small improvements to how the source line is displayed if it's too long.
Default to 120, as suggested in previous review.
Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Overhaul the preprocessor
The old Slang preprocessor was based on a simple mental model that tried to unify two parts of macro expansion:
* Scanning for macro invocations in a sequence of tokens
* Producing the expanded tokens for a macro expansion by substituting arguments into its body
The basic was that substitution of macro arguments into a macro definition is superficially similar to top-level macro expansion, just with an environment where the macro arguments act like `#define`s for the corresponding parameter names. That approach was "clever" and could conceivably have been extended to include a lot of advanced preprocessor features (e.g., a preprocessor-level `lambda` would be easy to support!), but it was basically impossible to make it correctly handle all the corner cases of the full C/C++ preprocessor.
The fundamental problem with the old approach was that it conflated the two parts of expansion listed above into one implementation, while the various special cases of the C/C++ preprocessor rely on treating the two cases very differently. The new approach here (which is somewhere between a refactor and a full rewrite of the preprocessor) changes things up in a few key ways:
* The abstraction still cares a lot about streams of tokens, but it now treats the top level streams (`InputFile`s) as fairly different from the lower-level streams (`InputStream`s)
* Macro expansion is handled as a dedicated type of stream that wraps another stream. This allows macro expansion to be applied to anything, and supports cases where multiple rounds of macro expansion are required by the spec.
* Macro *invocations* and the substitution of their arguments are now handled by a completely new system.
* Macro arguments are no longer treated as if they were `#define`s
* The macro body/definition is analyzed at definition time to detect various kinds of issues, and to derive a list of "ops" that make it easier to "play back" the definition at substitution time
* Token pasting and stringizing are now only handled in macro definitions (rather than being allowed anywhere), and their use cases are restricted to only those that make sense (e.g., you can't stringize anythign except a macro parameter, because anything else wouldn't make sense)
The key new types here are the `ExpansionInputStream` which handles scanning for macro invocations, and the `MacroInvocation` type, which handles playing back the macro body with substitutions.
The `ExpansionInputStream` is the easier of the two to understand. By refactoring it to use a single token of lookahead, the one major detail it had to deal with before (abandoning expansion of a function-like macro if the macro name was not followed by `(`) is significantly easier to manage.
The more subtle part is the `MacroInvocation` type, and most of the complexity there is around handling of token pasting, and the fact that either or both of the operands to a token paste might be empty.
Many of the test cases that exposed the problems in the preprocessor have been moved from `current-bugs` to `preprocessor` since they now work correctly.
* debugging: enable extractor command line dump
* fixup
* fixup
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Added SourceLoc handling for command line parsing.
* Fix typo in debug.
* Fix issue around the DiagnosticSink used in options parsing not having a writer available - by having DiagnosticSink parenting.
* Small rename for clarity.
Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP Fxc as downstream compiler.
* First pass FXC downstream compiler working.
* GCC compile fix.
* Fix FXC parsing issue.
* Special case filesystem access.
* Use StringUtil getSlice.
* Fix isses with not emitting source for FXC.
* WIP on DXC.
* Small fixes for DXBC handling.
* Removed DXC from ParseDiagnosticUtil (can use generic)
Try to improve output for notes from DXC.
* FIrst pass of Glslang as DownstreamCompiler
* Fix some problems with parsing for glslang replacement.
* Add slang-glslang-compiler.cpp/.h
* Fix downstream for spir-v output.
* dissassemble -> disassemble
* Fix typo and improve some naming/comments.
* Remove getSharedLibrary from DownstreamCompiler
* Removed some no longer used diagnostics.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change originated as an attempt to re-enable a test case, but it has ended up disabling more tests (for good reasons) than it re-enables.
The main change here is a significant overhaul of the way that the D3D12 render path extracts information from the Slang reflection API to produce a root signature.
There were also some supporting fixes in the reflection information to make sure it returns what the D3D12 back-end needed.
The big picture here is that the D3D12 path now uses the descriptor ranges stored in the reflection data more or less directly.
It still needs to use register/space offset information queried via the "old" reflection API, but it only does so at the top level now, for the program and entry points themselves.
All other layout information is derived directly from what Slang provides.
Smaller changes:
* The "flat" reflection API was expanded to include `getBindingRangeDescriptorRangeCount()` which was clearly missing.
* The "flat" reflection results for a constant buffer or parameter block that didn't contain any uniform data and was mapped to a plain constant buffer needed to be fixed up. That logic is still way to subtle to be trusted.
* Several additional tests were disabled that relied on static specialization, global/entry-point generi type parameters, structured buffers of interfaces or other features we don't officially support with shader objects right now. All of the affected tests were somehow passing by sheer luck and because they often passed in specialization arguments via explicit `TEST_INPUT` lines.
* The `inteface-shader-param` test is re-enabled now that we can properly describe its input with the new `set` mode on `TEST_INPUT`
* `ShaderCursor::getElement()` can now be used on structure types (in addition to arrays) to support by-index access to fields
* The `TEST_INPUT` system was expanded to support both by-name and by-index setting of structure fields for aggregates
* The `TEST_INPUT` system was expanded to allow an `out` prefix to mark parts of an expression as outputs on a `set` lines
* The `TEST_INPUT` system was expanded so that anything that would be allowed on a `TEST_INPUT` line by itself (like `ubuffer(...)`) can now be used as a sub-expression on a `set` line
Co-authored-by: Yong He <yonghe@outlook.com>
|
| |
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Add mechanism to embed guid inside of type.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Split out compiler-core initially with just slang-source-loc.cpp
* More lexer, name, token to compiler-core.
* Split Lexer and Core diagnostics.
* Move slang-file-system to core.
* Add slang-file-system to core.
* More DownstreamCompiler into compiler-core
* Fix typo.
* Add compiler-core to bootstrap proj.
* Small fixes to premake
* For linux try with compiler-core
* Remove compiler-core from examples.
* Added NameConventionUtil to compiler-core
* Add global function to CharUtil to *hopefully* avoid linking issue.
* Hack to make linkage of CharUtil work on linux.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* First pass support for __LINE__ and __FILE__.
* Test include handling with __FILE__
Fix diagnostic compare when input is empty.
* Fix some issues in preprocessor handling of special macros like __LINE__
Add a more complex test.
* Use CONCAT2 in tests, because preprocessor doesn't quite get parameter expansion correct.
* Make __FILE__ and __LINE__ behave more like Clang/Gcc.
* A test for preprocessor bug.
* Fix __LINE__ and __FILE__ in macro expansion, should be initiating location.
* Fix some comments.
* Small tidy up around builtin macros.
* Small improvements for macro type names.
Escape found paths.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Use capability system in docs.
Simplify how requirements/availability is produced.
* Small fixes in output of availablity.
* Updated stdlib doc.
* Small improvements.
* Added doc test type.
Improved readability of straight .md text
Made -doc option output to diagnostic stream.
* Add test for checking requirements info is correctly extracted.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add a CPU renderer implementation
This change adds a CPU back-end to `gfx` and ensures that most of our existing CPU tests pass when using it.
Detailed notes:
* Most of the CPU renderer implementation is copy-pasted from the CUDA case, so they share a lot of similar logic
* The main addition to the CPU renderer is a semi-complete implementation of host-memory textures. The logic here handles all the main shapes (Buffer, 1D, 2D, 3D, Cube) and all the currently-supported `Format`s that are sample-able as-is (no D24S8). The implementation is not intended to be fast, and it currently only does nearest-neighbor sampling, but otherwise it tries to avoid cutting too many corners and should be ar reasonable starting point for a more complete (but not performance-oriented) implementation.
* Refactored the CPU prelude `IRWTexture` interface to inherit from `ITexture`, since in most cases a single type will end up implementing both. It might be worth it to collapse it all down to a single interface later.
* Changed the CPU prelude `ITexture`/`IRWTexture` interface so that it takes both a pointer *and* a size for output arguments. This change seems necessary to allow a shader variable declared as a `Texture2D<float>` to fetch a single `float` when the underlying texture might be using RGBA32F.
* Added to the `IComponentType` public API so that we can query a "host callable" for an entry point and not just a binary.
* Turned off the `-shaderobj` flag on two tests that weren't yet compatible with shader objects but still had the flag left in on the path (since previously the CPU path always used the non-`gfx` non-shader-object logic anyway)
* Disabled one test (`dynamic-dispatch-11`) that relied on the `ConstantBuffer<IInterface>` idiom that we know we are planning to chagne soon anyway.
* Made a few changes to the CUDA path to bring it into line with what I added for the CPU path. These were mostly bug fixes around indexing logic for sub-objects and resources.
* fixup
|
| |
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* MarkDown -> Markdown
slang-doc-mark-down -> slang-doc-markdown-writer
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Split out AST 'printing'.
* Replace listener with List<Section>
* Section -> Part.
* Kind -> Type Flags -> Kind for ASTPrinter::Part
* Improve comments around ASTPrinter.
* toString -> toText on Val derived types. toText appends to a StringBuilder.
* Added toSlice free function.
Added operator<< for Val derived types.
Use << where appropriate in doing toText.
* More work at mark down output.
* Fill in sourceloc for enum case.
Add more sophisticated location determination for EnumCase.
Refactored documentation output into DocMarkdownWriter.
* Improvements for sig output.
* Split up slang-doc into extractor and writer.
* WIP generic support for doc support.
* Some refactoring to make DocExtractor have potential to be used without Decls.
* Made doc extraction work without Decls.
* Output generic parameters.
* Add generic parameter extraction.
* Added writing variables.
* Add an interface test.
* Fix toArray.
* Support for extensions, and inheritance.
* Disable the doc test.
* Added flags to compileStdLib.
* More work around handling generics in markdown output.
* More improvements around associated type handling.
* List method names only once.
Output in/out/inout/const
* Fix namespace printing.
* WIP summarizing doc output.
* Small fixes and improvements for doc output.
* Output all stdlib in single doc file.
* Remove compile flags from addBuiltinSource.
* Find only unique signatures.
First pass at trying to get requirements.
* First pass at requirements for stdlib docs.
* Remove __ function/methods
* Added Target Availability
* Add markup access.
Make sections of stdlib hidden.
* MarkdownAccess -> Visibility
Add isVisible methods
Use ASTPrinter to print decl name.
* Add current stdlib doc output.
* Disable doc test for now.
* Fix clang issue.
* Don't use bullets and numbering , just use numbering.
* Put methods in source order.
* Fix bad-operator-call.slang test that fails because it now outputs out parameters as such.
* Refactor MarkDownWriter to separate 'extraction' from output.
* Fix typo around @ lines.
* Fix issue with extracting 'before' when preceeded by complex attributes/modifiers.
* Fix handling of generics with the same name.
* Work around for having overloading with generics - we don't want to output generic params as part of name.
* Remove generic paramters from name.
* Simplify handling of outputting overridable names.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Split out AST 'printing'.
* Replace listener with List<Section>
* Section -> Part.
* Kind -> Type Flags -> Kind for ASTPrinter::Part
* Improve comments around ASTPrinter.
* toString -> toText on Val derived types. toText appends to a StringBuilder.
* Added toSlice free function.
Added operator<< for Val derived types.
Use << where appropriate in doing toText.
* More work at mark down output.
* Fill in sourceloc for enum case.
Add more sophisticated location determination for EnumCase.
Refactored documentation output into DocMarkdownWriter.
* Improvements for sig output.
* Split up slang-doc into extractor and writer.
* WIP generic support for doc support.
* Some refactoring to make DocExtractor have potential to be used without Decls.
* Made doc extraction work without Decls.
* Output generic parameters.
* Add generic parameter extraction.
* Added writing variables.
* Add an interface test.
* Fix toArray.
* Support for extensions, and inheritance.
* Disable the doc test.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Split out AST 'printing'.
* Replace listener with List<Section>
* Section -> Part.
* Kind -> Type Flags -> Kind for ASTPrinter::Part
* Improve comments around ASTPrinter.
* toString -> toText on Val derived types. toText appends to a StringBuilder.
* Added toSlice free function.
Added operator<< for Val derived types.
Use << where appropriate in doing toText.
* More work at mark down output.
* Fill in sourceloc for enum case.
Add more sophisticated location determination for EnumCase.
Refactored documentation output into DocMarkdownWriter.
* Improvements for sig output.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP extracting source documentation.
* WIP doc extraction.
* More stuff around doc markup extraction.
* More WIP around doc extraction.
* Fix some indexing issues.
* Initial doc extraction working.
* Renaming of types in markup extraction process.
* Extracting markup content.
Removing indenting.
Other fixes and improvements around document tools.
* WIP support for documentation system.
* Remove some commented out sections.
* Remove some comments that no longer apply.
* Improvements around SourceFile - such that more granularity around line ops.
Made some functionality explicitly work without source.
Improved Doc types nameing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP: First pass in supporting output of line error information.
* Add support for lexing to better be able to indicate SourceLocation information.
* Fix lexer usage in DiagnosticSink in C++ extractor.
* Update diagnostics tests to have line location info.
* Fixed test expected output that now have source location information in them.
* Better handling of tab.
* Fix test expected results for tabbing change.
* DiagnosticLexer -> DiagnosticSink::SourceLocationLexer
Added line continuation tests.
* Fix typo.
* Added String::appendRepeatedChar
* Change to rerun tests.
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Testing out use of lz4.
* Added ICompressionSystem, and LZ4 implementation.
* Add support for deflate compression.
Simplify compression interface - to make more easily work across apis.
* WIP on CompressedFileSystem.
* ImplicitDirectoryCollector
* SubStringIndexMap - > StringSliceIndexMap.
* WIP save stdlib in different containers.
* Support for different archive types for stdlib.
* Fix project.
* CompressedFileSystem -> ArchiveFileSystem.
Added CompressionSystemType::None
* Added ArchiveFileSystem
* Fix problem RiffFileSystem load withoug compression system.
* Test archive types.
Improve diagnostic message.
* Fix typo in testing file system archives.
* Split out archive detection.
* Fix gcc warning issue.
* Fix warning.
* RiffArchiveFileSystem -> RiffFileSystem
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use "capability" system to select VKRT extension
Slang currently supports translation of ray tracing shader code to Vulkan GLSL code that uses the `GL_NV_ray_tracing` extension. A multi-vendor equivalent of that extension has been released as `GL_EXT_ray_tracing` and we want Slang to support that extension as well.
At the simplest, making the change from one extension to the other is just a matter of changing a few strings, since it does not appear that anything of significance was changed at the GLSL level (or even in SPIR-V). Where this gets trickier is when we have users who want us to support *both* extensions, and to be able to switch between them.
The solution we've implemented here more or less amounts to:
* If you don't tell the compiler which extension to use, it will default to `GL_EXT_ray_tracing` (the newer multi-vendor one).
* If you explicitly want the older extension, you can opt into it using the `-profile` option or via a new API for explicitly adding capabilities to your target.
Making that work required a few different kinds of changes:
* The options parsing and public API needed ways to add optional capabilities to a target.
* During GLSL code emit, we can check the capabilities that were added to the target to see if the `GL_NV_ray_tracing` extension was explicitly enabled and, if not, default to using the `GL_EXT_ray_tracing` names for things. This step is needed because some of the modifiers/attributes involved in the extension have to be handled explicitly in the code generator rather than implicitly as part of mapping intrinsic functions.
* We add two different translations to the relevant operatiosn in the stdlib, one marked with each of the extensions. If profile/capability-based overload resolution can be relied on to pick the right one, this should Just Work.
* Next, a bunch of work had to go into making capability-based overloading Just Work for the purposes of this change. There's been a nearly complete reworking of the implementation of `CapabilitySet` here to make it more suitable for our needs.
* The tests that were using ray tracing translation for Vulkan needed to be updated. For some of them I updated their baselines to use `GL_EXT_ray_tracing` so that they can test the new path. For others, I updated the command line for the test case so that it explicitly opts into using `GL_NV_ray_tracing`. The result is that we have some coverage of each extension. I would have liked to have each test run in both modes, but our pass-through glslang support doesn't support `-D` options, so I couldn't take that step easily.
This change does *not* add support for `GL_EXT_ray_query`, the extension that supports "DXR 1.1" style queries under Vulkan. Adding support for that extension should hopefully be a smaller step because it doesn't have the same multiple-extensions issue.
This change does *not* address a lot of possible avenues for improvement or cleanup around the capability system. It focuses only on those changes that are necessary to make the ray tracing feature work and leaves the rest for future work.
* fixup: infinite loop
* Comment-only change to retrigger TC build
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add first steps toward a "capability" system
We already have cases in the stdlib where we mark declarations as being specific to certain targets, e.g.:
```
// My ordinary function to add two numbers.
// Works everywhere.
//
void myFunc(int a, int b) { return a + b; }
// On the "coolgpu" target, we can use a secret intrinsic
// that adds numbers even faster!
//
__specialized_for_target(coolgpu)
void myFunc(int a, int b) { return __secretIntrinsic(a, b); }
```
The existing logic for dealing with these modifiers (`__specialized_for_target` and `__target_intrinsic`) was almost entirely string-based. We would turn the chosen compilation target into a string, and then use that to try and search for the "best" definition of a function at a few steps:
* During IR linking, we always pick one definition of an `[import]`ed function, and that definition will be the one with the "best" target-specialization modifier (if any)
* During final code generation, we always look up the "best" target-intrinsic modifier, and use it as the template for the code we output.
This change preserves the basic flow there, but replaces the ad hoc string-based logic with something a bit more principled, in terms of a new `CapabilitySet` type.
A `CapabilitySet` represents a set of zero or more atomic features (here represented as `CapabilityAtom`s). What a `CapabilitySet` means depends on how and where it is used:
* A compilation target implies a `CapabilitySet` where the contents of the set are the features the target *supports*.
* A `CapabilitySet` attached to a declaration (or a modifier on that declaration) describes a set of feature that declaration *requires*.
The current implementation of `CapabilitySet` is wasteful and inefficient, but that is something we can iterate on over time.
In practice, most of the current code only ever uses capability sets that are either empty (because they represent a function with no specific requirements) or singleton (because they represent asingle atomic capability like "is a GLSL target," "is an HLSL target," etc.).
The main goal here was to put in the skeleton of a new system, including some of the features it might need down the line, and then to leave changes that eventually use the greater flexibility for later. Eventually, the capability system should encompass:
* Differences between shader model versions, GLSL versions, SPIR-V versions, etc. (currently tracked with other modifiers)
* Optional extensions, and functions that are made available only with certain extensions (currently tracked with other modifiers)
* Front-end checking that the call graph of a program doesn't violate any capability-requirements (e.g., having a GLSL+HLSL portable function call a GLSL-only subroutine)
* Hypothetically we can also try to fold stage-specific (vertex-only, fragment-only, etc.) functions into this system, but doing so would require more linker cleverness if we allow overloading on stages (since we might have to clone a caller if it calls through to a callee with multiple stage-specific versions)
One important complication that the system has to deal with just because of the "do what I mean" nature of the current compiler is that somethings a current Slang user might compile for target X and specify version N, but then use a function that actually requires version N+1 of that target. Currently the Slang compiler silently "upgrades" the version(s) used by user code in these cases, because it is often what users want in cross-compilation scenarios.
Dealing with the "silent upgrade" situation requires us to be a little careful and sometimes pick a "best" capability set that doesn't appear to be supported on our target. Refining that system and potentially getting rid of the "do what I mean" behavior over time could be a goal for future changes.
* fixup: handle case where value is incompatible during linking
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Move reflection to reflection-api.
* Slight reorg to pull out potentially Slang internal functions from the reflection API impls.
* Remove visual studio projects
* Fix for slang-binaries copy.
* Add the visual studio projects in build/visual-studio
* Remove miniz project.
* Differentiate the linePath from the filePath.
* Improve comment in premake5.lua + to kick of CI.
* Kick CI.
* Use COM compile request for calls to functions inside api-less-slang.
Add static-slang project.
* Fix const typo issue.
* Don't include 'core' link in 'api-less-slang'
* Removed static-slang lib causes problems on linux with linking.
Embed Slang stdlib
Added StaticBlob
Added dumpSourceBytes
Use ConstArrayView for the archive.
At startup allow loading of zip with stdlib.
Made -save-stdlib -load-stdlib take a name
Added '-save-stdlib-bin-source' to save out serialized stdlib as source.
* Ability enable/disable stdlib embedding.
* Fix problem with moduleDecl not having module pointer set when serialized in.
* Set of debugdir for slang-test and examples.
* Add slang-stdlib-api.cpp
* Update slang filters for VS.
* Try to use pic, and -mcmodel=medium
* Some more efforts ot make premake work.
* WIP premake5.lua from previously working version.
* Remove api-less-slang project.
* Disable dllexport on gcc/clang.
* Embed via slangc-bootstrap.
* Fix slang-profile. Always compiles without stdlib.
* Use pic "On"
* Remove slangc-bootstrap and embed-stdlib-generator if embedding not required. Make bootstrap run the generators.
* Improve comments in premake5.lua.
Kick off another CI build.
* Remove generation of stdlib source from std-lib-serialize.slang
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* "Shader Toy" example and related fixes
This change introduces a new `shader-toy` example program that is primarily designed to show how Slang's features for type-based encapsulation and modularity can be applied to modularity for effects along the lines of those from `shadertoy.com`.
The Example
-----------
The example is being checked in with an example "toy" effect that I hastily put together, so that it would not be encumbered with any IP concerns. I wrote the effect using the shadertoy.com editor, so I can be sure it is valid GLSL. During bringup of the application I used a pre-existing and larger effect for testing, so some of the support code that was added is not being used at present.
The big-picture idea here is to have an exmaple that shows how to modularize things using Slang interfaces and generics, and then to use the Slang compiler API to manage the compilation, composition, specialization, and linking steps. For better or worse this leads to the sequence of API calls involved being much longer than what was in something like the `hello-world` example.
Future Work (Example)
---------------------
There is a lot of room for improvement and expansion here, so this should be viewed as a checkpoint of work in progress rather than something I'm claiming as a finalized demonstration of all we'd like to achieve. Areas for future work include:
* We need to copy the integration of "Dear, IMGUI" that was already done for the `model-viewer` example so that this example can have a UI.
* Now that the compilation flow is broken into all these additional steps, it should be possible to have the application load multiple effects as distinct modules, and then provide a UI for switching between them. The chosen effect module would be used to specialize the top-level shader(s) before kernel generation.
* The checked-in logic includes a compute shader that can execute an effect, but that hasn't been tested nor has it been wired up to any kind of UI. We should have a way to switch between multiple execution methods, with a goal of eventually including CPU execution.
* The "GLSL compatibility" code needs a lot of improvements before it is likely to be usable for a nontrivial number of shaders. Some of that work is waiting on Slang compiler fixes, though.
* We should consider allowing the individual "toy" effects to define their own uniform parameters and expose those via a UI and reflection. The catch in this case is not that this would be difficult to do, but that it would be a semantic change to how shader toy effects currently work.
The Compiler Fixes
------------------
Doing this work exposed a few bugs in Slang, and this change includes fixes for the ones that were quick to address.
We already had logic in `slang-check-shader.cpp` that was validating the entry points in a compile request - either by checking the explicitly-listed entry points, or by scanning for `[shader("...")]` attributes. The problem is that the routine that did that checking was not being invoked on all compiles. The logic that handled entry points was only being run for manual compiles using `SlangCompileRequest`, while anything using `import` or `loadModule` would ignore entry points. I refactored the relevant code into a subroutine that will be invoked in all compilation scenarios.
There were already `TODO` comments in `SpecializedComponentType` which made the point about how a specialized entry point like `myShader<YourType>` would need to properly show that it has dependencies on both the module that defines `myShader` *and* the module that defines `YourType`, while only the former was being handled at present. I went ahead and implemented the logic to scan the generic arguments for a specialized compoment type in order to determine what module(s) the arguments depend on (both type arguments and witness tables). With that change, using `IComponentType::link` on a specialized component will properly pull in the module(s) that the generic arguments come from.
In `slang-ir-legalize-types.cpp` we could run into assertion failures in debug builds because of code trying to legalize layout `IRAttr`s for fields or parameters with types that need legalization. In practice it is safe to skip these layout attributes, because legalization of the fields/parameters they pertain to would result in creation of entirely new layout attributes, and the old ones would then be unreferenced.
Future Work (Fixes)
-------------------
There are other compiler bugs that this work exposed, but which this change does not address. These will need to be resolved as part of subsequent changes:
* Slang allows for default-initialization of variables of a generic type. That is, given `<T : ISomething>` a user is allowed to declare `T x = {};` and the Slang front-end does not complain. Instead, this leads to an internal compiler error during IR lowering.
* The Slang `__init()` feature probably needs to be upgraded to a properly supported feature, and we probably need a way to make implementing default-initialization an easy thing (e.g., any `struct` type that has initial-value expressions for all its fields should automatically and implicitly satsify an `init();` requirement declared in an interface)
* Iniside an `__init()` definition, code has mutable access to members of the enclosing type, but for some reason the front-end is incorrectly treating `this` as immutable in those contexts. As a result you can write to `someField` but not `this.someField`.
* User-defined operator overloads flat out don't work (which isn't surprising given that no clients have decided to use them yet, and we have no test coverage for them). This is actually due to the shadowing rules being used for lookup right now, so a fix for this issue is going to have far-reaching consequences around what overloads are visible where (and anything that impacts overload resolution is a big can of worms, including around performance).
* fixup: test case had missing main function
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Move reflection to reflection-api.
* Slight reorg to pull out potentially Slang internal functions from the reflection API impls.
* Remove visual studio projects
* Fix for slang-binaries copy.
* Add the visual studio projects in build/visual-studio
* Remove miniz project.
* Differentiate the linePath from the filePath.
* Improve comment in premake5.lua + to kick of CI.
* Kick CI.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP for COM CompileRequest.
* Add more methods to IGlobalSession.
* Fix createCompileRequest.
Made slangc tool use COM style methods.
* m_ prefix variables in EndToEndCompileRequest
|
| |
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* WIP FileSystem refactor.
* Made loadFile load the file in binary mode.
* Fixed some comments.
Fixed typo in RelativePath - not used 'fixedPath'.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Mangling/module name extraction for GenericDecl
* Add comment on SerialFilter to explain re-enabling Stmt.
* Support setting up SyntaxDecl when reconstructed after deserialization.
* Improvements to setup SyntaxDecl.
* Fix typo so can read compressed SourceLocs.
* Fix issue with SourceManger.
* Simple test for serializing out stdlib and reading back in.
* Fix calling convention.
* Add override to StdLib impls.
* Fix typo.
* Apply testing to an actual compute test when using load-stdlib
Make -load/compile-stdlib processable by Slang
Move out testing into util into TestToolUtil so can be shared.
* Slightly more concise setup of session.
* Fix some errors introduced with session handling.
* Made setup for compile same across slangc and slangc-tool.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* #include an absolute path didn't work - because paths were taken to always be relative.
* Mangling/module name extraction for GenericDecl
* Add comment on SerialFilter to explain re-enabling Stmt.
* Support setting up SyntaxDecl when reconstructed after deserialization.
* Improvements to setup SyntaxDecl.
* Fix typo so can read compressed SourceLocs.
* Fix issue with SourceManger.
|
