summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-03-16Fix the "acceleration structure in compute" bug for GL_NV_ray_tracing too ↵Tim Foley
(#1759) A recent change broke code that uses `RayTracingAccelerationStructure` in non-RT shader stages for Vulkan/GLSL when also *not* doing any ray tracing in the shader code. A recent fix patched that up for code using `GL_EXT_ray_tracing` and/or `GL_EXT_ray_query`, but that fix didn't apply on the path that uses `GL_NV_ray_tracing` via an opt-in. This change fixes that gap and checks in a test for it.
2021-03-16Update binaries (#1758)Tim Foley
2021-03-16Enable building glslang from source (#1757)Tim Foley
* Enable building glslang from source Somehow the slang-glslang binaries we are currently using aren't the most up-to-date ones, so I am enabling building glslang from source so that we can produce new binaries. * fixup: run generators
2021-03-15Enable `gfx::CUDADevice` on linux. (#1756)Yong He
2021-03-15Preliminary docs on 'Doc System'. (#1755)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * First docs on 'doc system'. * Small improvements to doc system documentation. Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-03-15Test Doc System (#1754)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Use capability system in docs. Simplify how requirements/availability is produced. * Small fixes in output of availablity. * Updated stdlib doc. * Small improvements. * Added doc test type. Improved readability of straight .md text Made -doc option output to diagnostic stream. * Add test for checking requirements info is correctly extracted. Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-03-15Fix handling of RT accelerations structures for non-RT stages (#1753)Tim Foley
* Fix handling of RT accelerations structures for non-RT stages The recent change that added support for the `GL_EXT_ray_query` extension made is so that a shader that declares a `RaytracingAccelerationStructure` as an input to a non-RT shader stage but then never *uses* it wouldn't enable any RT extension, resulting in a compilation failure in glslang. This change reverts that behavior so that such shaders enable `GL_EXT_ray_tracing`, since that is the older of the two RT extensions that introduce `accelerationStructureEXT`. It is possible that we will need to revisit this decision based on which of the two extensions ends up being more broadly supported, but I think that right now it is fair to say that there exist drivers that support `GL_EXT_ray_tracing` but not `GL_EXT_ray_query`, so the former is the better default. * fixup: failing test
2021-03-15Improvements in Docs requirements/availability (#1751)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Use capability system in docs. Simplify how requirements/availability is produced. * Small fixes in output of availablity. * Updated stdlib doc. * Small improvements. Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-03-12Cleanup CPU renderer. (#1752)Yong He
2021-03-12Add a CPU renderer implementation (#1750)Tim Foley
* Add a CPU renderer implementation This change adds a CPU back-end to `gfx` and ensures that most of our existing CPU tests pass when using it. Detailed notes: * Most of the CPU renderer implementation is copy-pasted from the CUDA case, so they share a lot of similar logic * The main addition to the CPU renderer is a semi-complete implementation of host-memory textures. The logic here handles all the main shapes (Buffer, 1D, 2D, 3D, Cube) and all the currently-supported `Format`s that are sample-able as-is (no D24S8). The implementation is not intended to be fast, and it currently only does nearest-neighbor sampling, but otherwise it tries to avoid cutting too many corners and should be ar reasonable starting point for a more complete (but not performance-oriented) implementation. * Refactored the CPU prelude `IRWTexture` interface to inherit from `ITexture`, since in most cases a single type will end up implementing both. It might be worth it to collapse it all down to a single interface later. * Changed the CPU prelude `ITexture`/`IRWTexture` interface so that it takes both a pointer *and* a size for output arguments. This change seems necessary to allow a shader variable declared as a `Texture2D<float>` to fetch a single `float` when the underlying texture might be using RGBA32F. * Added to the `IComponentType` public API so that we can query a "host callable" for an entry point and not just a binary. * Turned off the `-shaderobj` flag on two tests that weren't yet compatible with shader objects but still had the flag left in on the path (since previously the CPU path always used the non-`gfx` non-shader-object logic anyway) * Disabled one test (`dynamic-dispatch-11`) that relied on the `ConstantBuffer<IInterface>` idiom that we know we are planning to chagne soon anyway. * Made a few changes to the CUDA path to bring it into line with what I added for the CPU path. These were mostly bug fixes around indexing logic for sub-objects and resources. * fixup
2021-03-11MarkDown -> Markdown (#1748)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * MarkDown -> Markdown slang-doc-mark-down -> slang-doc-markdown-writer
2021-03-11stdlib documentation (#1745)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out AST 'printing'. * Replace listener with List<Section> * Section -> Part. * Kind -> Type Flags -> Kind for ASTPrinter::Part * Improve comments around ASTPrinter. * toString -> toText on Val derived types. toText appends to a StringBuilder. * Added toSlice free function. Added operator<< for Val derived types. Use << where appropriate in doing toText. * More work at mark down output. * Fill in sourceloc for enum case. Add more sophisticated location determination for EnumCase. Refactored documentation output into DocMarkdownWriter. * Improvements for sig output. * Split up slang-doc into extractor and writer. * WIP generic support for doc support. * Some refactoring to make DocExtractor have potential to be used without Decls. * Made doc extraction work without Decls. * Output generic parameters. * Add generic parameter extraction. * Added writing variables. * Add an interface test. * Fix toArray. * Support for extensions, and inheritance. * Disable the doc test. * Added flags to compileStdLib. * More work around handling generics in markdown output. * More improvements around associated type handling. * List method names only once. Output in/out/inout/const * Fix namespace printing. * WIP summarizing doc output. * Small fixes and improvements for doc output. * Output all stdlib in single doc file. * Remove compile flags from addBuiltinSource. * Find only unique signatures. First pass at trying to get requirements. * First pass at requirements for stdlib docs. * Remove __ function/methods * Added Target Availability * Add markup access. Make sections of stdlib hidden. * MarkdownAccess -> Visibility Add isVisible methods Use ASTPrinter to print decl name. * Add current stdlib doc output. * Disable doc test for now. * Fix clang issue. * Don't use bullets and numbering , just use numbering. * Put methods in source order. * Fix bad-operator-call.slang test that fails because it now outputs out parameters as such. * Refactor MarkDownWriter to separate 'extraction' from output. * Fix typo around @ lines. * Fix issue with extracting 'before' when preceeded by complex attributes/modifiers. * Fix handling of generics with the same name. * Work around for having overloading with generics - we don't want to output generic params as part of name. * Remove generic paramters from name. * Simplify handling of outputting overridable names.
2021-03-11Change representation of initial data for textures (#1747)Tim Foley
* Change representation of initial data for textures Before this change, initial data for a texture has been provided with the `ITextureResource::Data` type, where a call to `IDevice::createTexture()` would take zero or one `Data` and, if present, use it to initialize all the subresources of a texture. The organization of `Data` was not actually quite how its own documentation comment described it (the implementations didn't agree with the comment), and while it aggressively factored out redundancies (e.g., only storing the stride for each mip level once, instead of once per subresource for large arrays), the result was that setting up a `Data` correcty was a bit confusing. This change makes the initial data for a texture using a `SubresourceData` type that is almost identical to what D3D11 uses, so that developers are more likely to be comfortable filling it in. All of the existing implementations were easily adapted to use the new type, so it seems like a net win. Note: Both Vulkan and D3D11 do away with the idea of initializing a texture with data as part of allocating it, and we might eventually want to do the same given the complexity that this system entails. The main reason to preserve this detail is for better compatibility with D3D11, where immutable textures/buffers need to have their data specified at creation time. It seems good to preserve the ability to have immutable resources on target APIs where this distinction could affect performance (e.g., immutable resources do not need state/transition tracking on APIs like D3D11). * fixup: CUDA
2021-03-11Add Linux support to `platform` and `gfx`. (#1744)Yong He
2021-03-10A bunch of overlapping semantic-checking fixes (#1743)Tim Foley
This change originally started with the simple goal of allowing generic functions with default argument values on their parameters to work: ``` void someFunction<T>(T value, int optional = 0); ``` The core problem there was that the compiler code was (correctly) anticipate the case where the default argument value for a parameter depends on a generic parameter, such as: ``` interface IDefaultable { static This getDefault(); } void anotherFunction<T : IDefaultable>(T first, T second = T.getDefault()); ``` Supporting this latter case requires some kind of ability to apply subsitutions to an `Expr`, but our compiler logic simply errored out in that case. The first major fix that went into this change was to add a new `SubstExpr<T>` type that behaves a lot like `DeclRef<T>` in that it stores a `T*` plus a set of substititions that need to be applied to it. In addition, it was found that even if `anotherFunction<ConcreteType>(...)` might work, when generic argument inference was used for just `anotherFunction(...)` would fail because it includes a strict match on the number of arguments/parameters in the call expression. The next problem that arose was that the test I'd created used an interace with an `__init` requirement, and it appeared that our code generation didn't work for that case: ``` interface IStuff { __init(int val); } void f<T : IStuff>(T x = T(0)); ``` In this case, the `T(0)` initialization would get compiled to `(ConcreteType) 0` in the output rather than calling the function generated for the `__init` inside `ConcreteType`. The basic problem there was a bit of crufty old logic we have in place to work around the large number of `__init` declarations in the stdlib that don't have proper `__intrinsic_op` modifiers on them. We really need to fix the underlying problem there, but I worked around it by having the IR lowering pass only do its workaround magic on stdlib declarations. The next problem down this line was that my test had two different `__init` declarations in the concrete type and the logic for checking interface conformance was picking the wrong one to satisfying an interface requirement despite it being obviously wrong (not even the right number of parameter). This last problem led me down the rabbit-hole of trying to actually get our semantic checking for interface requirements right. There were a few pieces to that work: * Actually checking that the parameter and result types for two callables match is the simple part. If that was all that would be required we would have implement this logic a long time ago. * Next we have to deal with functions that make use of the `This` type, associated types, etc. We have to know that when the interface uses `This`, we want to treat that as equivalent to `ConcreteType`, and similarly for associated types. Getting that working is mostly a matter of setting up a this-type subsitution for the interface member being checked. * Finally, when comparing generic declarations like `IBase::doThing<T>` and `Derived::doThing<U>` we need to deal with the way that `T` and `U` represent the "same" logical type parameter, but are distinct `Decl`s. This is handled by specializing the base declaration to the parameters of the derived one (e.g., forming `IBase::doThing<U>` using the `U` from `Derived::doThing`). The result seems to be passing our tests, but there are still a few gotchas lurking, I'm sure.
2021-03-10Swapchain resize and rename to `IDevice` (#1741)Yong He
* Swapchain resize * Fix.
2021-03-08Add GLSL support for SV_InnerCoverage (#1740)Tim Foley
This was a fairly straightforward addition once I found the correct GLSL extension spec to use.
2021-03-08Refactor window library. (#1739)Yong He
* Refactor window library. * Fix project file * Fix warnings.
2021-03-07Bug fix in window creation. (#1738)Yong He
2021-03-05Add Vulkan/SPIR-V support for TraceRayInline() (#1737)Tim Foley
For the most part, this translation is straightforward because the `GL_EXT_ray_query` extension is well aligned with the DXR 1.1 `RayQuery` feature. Many function map one-to-one from one extension to the other. A few notable details: * The equivalent of the `RayQuery<Flags>` type is non-generic in GLSL, and the GLSL path previously didn't have support for trying to look up an intrinsic type name on an IR type declaration, so that required some tweaks to the emit logic. * All the GLSL functions are free functions instead of member functions, but our IR doesn't recognize that distinction anyway * The main `TraceRayInline()` call is the one that took the most tweaking, just because it takes a `RayDesc` structure for D3D/HLSL but takes individual vector sand scalars for VK/GLSL. The approach here is a standard one for how we manage this stuff in the stdlib (and I wanted to avoid adding even more `$` magic for intrinsics). * For several other calls, the HLSL API had distinct `Candidate***()` and `Committed***()` calls that return information about a candidate hit vs. the one committed into the query. In contrast, the GLSL API uses a single call that takes an additional "must be compile-time constant" `bool` parameter to select between the two behaviors. This is even the case for one call that basically returns a value of a different `enum` type depending on the state of that `bool`. The D3D API model here seems almost strictly better and I have no idea why the GLSL extension was defined this way. * Because both the `GL_EXT_ray_query` and `GL_EXT_ray_tracing` extensions declare the `accelerationStructureEXT` type, we can no longer infer what extension is supposed to be used based only on the presene of such a type. The logic right now is a bit slippery, because in theory a program that declares an acceleration structure but never traces into it could end up getting a compilation error now. We will have to see if that corner case comes up in practice. :( The one big detail that is looming after doing this work is that both the HLSL and GLSL exposures of ray queries are extremely "slippery" about the actual identity of queries (e.g., when is one query a copy of another, vs. just being a new variable that references the existing query). Somehow queries get their identity from the original declaration, and as such our "default constructor" approach to them seems semanticay correct, but the whole thing is kind of slippery at a foundational level and I don't know how to fix it with the API as defined. Oh well; just something to keep an eye on. Co-authored-by: Yong He <yonghe@outlook.com>
2021-03-05Doc tooling improvements (#1734)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out AST 'printing'. * Replace listener with List<Section> * Section -> Part. * Kind -> Type Flags -> Kind for ASTPrinter::Part * Improve comments around ASTPrinter. * toString -> toText on Val derived types. toText appends to a StringBuilder. * Added toSlice free function. Added operator<< for Val derived types. Use << where appropriate in doing toText. * More work at mark down output. * Fill in sourceloc for enum case. Add more sophisticated location determination for EnumCase. Refactored documentation output into DocMarkdownWriter. * Improvements for sig output. * Split up slang-doc into extractor and writer. * WIP generic support for doc support. * Some refactoring to make DocExtractor have potential to be used without Decls. * Made doc extraction work without Decls. * Output generic parameters. * Add generic parameter extraction. * Added writing variables. * Add an interface test. * Fix toArray. * Support for extensions, and inheritance. * Disable the doc test. Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-03-05Cache stdlib when creating global session. (#1736)Yong He
* Cache stdlib when creating global session. * Fix * Fix
2021-03-04Refactor `gfx` to surface `CommandBuffer` interface. (#1735)Yong He
* Refactor `gfx` to surface `CommandBuffer` interface. * Fixes. * Fix code review issues, and make vulkan runnable on devices without VK_EXT_extended_dynamic_states. * Update solution files * Move out-of-date examples to examples/experimental Co-authored-by: Yong He <yhe@nvidia.com>
2021-03-03Add GLSL/SPIR-V support got GetAttributeAtVertex (#1733)Tim Foley
This change allows varying fragment shader inputs to be declared in a way that allows the `GetAttributeAtVertex` operation to compile to valid code for both D3D and GLSL/SPIR-V/Vulkan. The key is that rather than just use ordinary `nointerpolation`-qualified inputs the code must declare these varying inputs with a new `pervertex` qualifier that marks them as *only* being usable with `GetAttributeAtVertex`. The `pervertex`-tagged inputs then translate to GLSL inputs using the `pervertexNV` qualifier Note that this change does *not* include any enforcement of the requirements around how these qualifiers are used (and the compiler doesn't have enforcement for the existing operations like `EvaluateAttributeAtCentroid`). The underlying problem is that the inerpolation-mode qualifiers and explicit interpolation functions in HLSL constitute a kind of rate-qualified type system, but without any systematic rules. It seems wasteful to encode a bunch of ad hoc rules for this stuff as special cases in the compiler when the clear right answer is to implement a systematic approach to rates.
2021-03-02Clean up declarator handling during source emit (#1732)Tim Foley
This change tidies up some code related to the handling of declarators for the purpose of "unparsing" types into C-like declarations. The big change is that the `EDeclarator` type is changed to `DeclaratorInfo` and now has a bit of a subtype hierarchy under it rather than just using a `union`. The declarations have been moved to the header for CLikeSourceEmitter` so that they can be used by subclasses. I also removed the `IRDeclaratorInfo` type that was being declared but never actually used, and moved the case for pointers from that type into the main `EDeclarator`/`DeclaratorInfo`.
2021-03-02Fix issue with long identifier names in GLSL output (#1731)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * First pass at handling 'names' that are too long in GLSL output. * Test to check functionality with very long func name. * Add access a long names buffer. * Fix typo in assert. Fix issue with coercion error for 1.0f / 0x7fffffff Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-03-02Add command-line control over SPIR-V version (#1730)Tim Foley
* Add command-line control over SPIR-V version By default the Slang compiler policy is usually to produce output with the fewest dependencies possible. If input code can be encoded as SPIR-V 1.0, that is what we will use by default. The catch here is that in some cases later SPIR-V versions introduced improvements to the encoding that can affect performance (e.g., around large global arrays of constants), so that a user might explicitly want to require a newer SPIR-V version (restricting the driver versions their code can work on) in the hopes of seeing better performance. This change uses the system of capabilities that was previously introduced so that an option like `-profile glsl_450+spirv_1_5` can be used to explicitly request a specific SPIR-V version. Consistent with the existing implementation, the requested version will be taken as a minimum, and the final version might be higher based on other requirements (e.g., use of intrinsic functions that require a higher version). The test case included here is a little iffy in terms of long-term maintanenace. It relies on having both a `.slang` file and a `.glsl` file that we compile with the same options and then compare the SPIR-V, but that means there is no direct testing that the output SPIR-V actually uses the necessary version. If we break the inference of SPIR-V versions for both the regular and pass-through paths at once, this test won't flag the problem. A better test is probably needed soon. This change *only* adds support for controlling the SPIR-V version via capabilities specified via the command line or API. It would be nice to a future change to allow something like `[require(spirv_1_5)]` to be added to an entry point function to allow the user to embed their expectation/requirement into the source code. * fixup: clang warning
2021-03-01Doc improvements (#1729)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out AST 'printing'. * Replace listener with List<Section> * Section -> Part. * Kind -> Type Flags -> Kind for ASTPrinter::Part * Improve comments around ASTPrinter. * toString -> toText on Val derived types. toText appends to a StringBuilder. * Added toSlice free function. Added operator<< for Val derived types. Use << where appropriate in doing toText. * More work at mark down output. * Fill in sourceloc for enum case. Add more sophisticated location determination for EnumCase. Refactored documentation output into DocMarkdownWriter. * Improvements for sig output.
2021-02-26Shader object specialization work-in-progress (#1728)Tim Foley
* Shader object specialization work-in-progress The big change here is in the `setObject()` implementations, where we now take write the witness table ID and data for the value being assigned in both the CUDA and graphics-API paths (it is possible the code could be shared...). The logic for deciding whether a value "fits" in the existential value payload should actually be correct here, since it uses the reflection data. The other relevant change is that the logic for writing out the ordinary/uniform data for a shader object on the graphics-API path has been updated so that it only allocates the GPU buffer *after* it knows the specialized layout, and can thus allocate space for any extra parameter data that wasn't in the original layout but got added by specialization. There is some inactive code in place that tries to sketch how the implementation should handle writing the data of sub-objects for interface-type fields into the appropriate areas of the allocated buffer for a parent object, but that is stubbed out for now pending implementation of the relevant reflection information. This change also introduces logic in the graphics-API path to create a specialized layout for a shader object on-demand (so that it will only be created after the specialization arguments are known or can be inferred). The implementation needs to treat ordinary shader objects and root shader objects differently because the Slang API handles specialization differently for ordinary types vs. `IComponentType`s. Some notes and caveats: * The CUDA path doesn't need to compute specialized layouts the way the graphics-API path does because layout doesn't change based on specialization for that path (just as it won't for the CPU path) * This code just skips over the RTTI field in existential values because it seems that we currently aren't using it in generated code. * We are completely missing the logic for recursively writing the resource ranges of sub-objects bound to interface-type fields into the descriptor set(s) of the parent object. The missing link there is reflection API support, just as it is for filling in the ordinary/uniform data. We need a way to get the binding range offset (and binding array stride) for the "pending" data of a specialized interface-type field. * The logic for computing specialization arguments based on the shader objects bound to interface-type fields has a lot of holes. Some of the indexing math is flat-out incorrect, and it also doesn't make any attempt to handle sub-object ranges with more than one element in them. I tweaked some of the code there to make it *more* correct, but that doesn't mean it is actually correct at this point. * The logic for computing a specialized `IComponentType` for a `ProgramVars` in the graphics-API path seems to have a lot of overlap with `maybeSpecializeProgram()`, so we should look into ways to avoid the duplication over time. * clang error fix
2021-02-24Partial fix for macro expasnion of token pastes (#1727)Tim Foley
The underlying problem here requires that we have an object-like macro with an expansion that starts with a non-identifier token: ``` ``` Then we need a function-like macro that uses a token paste in a way that can expand to that object-like macro: ``` ``` Finally, for the specific case a user ran into, we need to invoke that function-like macro in the context of a preprocessor conditional expression: ``` // ... #error "unimplemented" ``` The way a problem manifest is that the preprocessor logic that handles conditional expressions tries to "peek" one token ahead and see what is coming, and while the peeking logic handles macro expansion it does *not* handle token pasting right now. That means that the peek operation sees `MY_FEATURE` and assumes that it is seeing an identifier in a preprocessor conditional that doesn't have a macro expansion. The logic then goes on to read the token, but what it gets back is *not* an identifier, and is instead the numeric literal token `1`, because the reading logic handles token pasting. The quick fix I applied here is to make the logic that deals with preprocessor conditionals go ahead and automatically consume a token from the input, and then decide what to do based on that token, so that it always makes use of the reading logic that handles token pasting. The lingering problem is that we still have cases in the preprocessor that use the peeking logic which doesn't handle pasting, and we might find that those cases have reason to want the same kind of expansion behavior we needed here. A more systematic fix would be to have the peeking logic automatically handle token pasting as well as macro expansion, but doing so would be a more complicated change because detecting the `##` when peeking ahead requires two tokens of lookahead, and our current implementation only assumes we can support one. Co-authored-by: Yong He <yonghe@outlook.com>
2021-02-24Explicit swapchain interface in `gfx`. (#1726)Yong He
* Explicit swapchain interface in `gfx`. * Correctly return nullptr when `IRenderer` creation failed. * Fix crashes on CUDA tests. * Cleanups.
2021-02-24Add support for GetAttributeAtVertex for D3D (#1725)Tim Foley
This operation was added along with the `SV_Barycentrics` system-value input, and allows for a `nointerpolation` varying input to a fragment shader to be fetched at a specific vertex index within the primitive that is causing the fragment shader to be invoked. This change adds support for the new operations in the standard library, and also includes a test case to make sure that we emit it correctly when producing HLSL/DXIL. This change also includes a small bug fix to our emission logic for function parameters so that we properly emit layout-related attributes for varying parameters declared directly on an entry point. (Note that most attribute end up being declared in `struct` types in existing HLSL shaders, and our IR passes produce only global variables for attributes on GLSL; the only case this affects is inidividual scalar/vector attributes declared declared as entry-point parameters, when outputting HLSL) Note that this change only adds support for the new function on the HLSL/DXIL path, and doesn't yet add any cross-compilation support for GLSL/SPIR-V. The reason for this is that the equivalent GLSL feature(s) appear to use a different model to the HLSL version, and we need to invent a suitable approach to align them to make portable code possible.
2021-02-23Documentation markup extraction (#1724)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP extracting source documentation. * WIP doc extraction. * More stuff around doc markup extraction. * More WIP around doc extraction. * Fix some indexing issues. * Initial doc extraction working. * Renaming of types in markup extraction process. * Extracting markup content. Removing indenting. Other fixes and improvements around document tools. * WIP support for documentation system. * Remove some commented out sections. * Remove some comments that no longer apply. * Improvements around SourceFile - such that more granularity around line ops. Made some functionality explicitly work without source. Improved Doc types nameing.
2021-02-23Some ad hoc parser fixes (#1723)Tim Foley
The `AdvanceIfMatch()` method was introduced to the parser as a way to avoid infinite loops when parsing nested list structures (e.g., `()`-enclosed parameter lists). The basic idea is that it tries to detect if we have scanned "too far" looking for a closing token, and reports a match to whatever logic was doing the looping to break the statemate. Unfortunately, the `TryRecoverBefore` logic was changed at some point so that it doesn't necessarily advance any tokens at all, because we generally don't want to skip over a `}` while searching for a `)`. As a result, we could still end up in an infinite loop where we didn't consume any additional tokens as part of recovery, but wouldn't bail out of the search for a match. This change tries to introduce a slightly more systematic setup where `AdvanceIfMatch` is now parameterized on a type of matched token pair (not just the closing token), and each such matched token pair introduces a list of tokens where if we see them as our lookahead we should bail out (e.g., when looking for a `)` we should give up the search upon seeing a `}`). After installing that fix I found that my simple test case still gave a surprising error because when mistakenly parsing a function body the parser would look for a `{` and then a `}` to close the body. The search for a closing `}` could accidentally consume a `}` meant for an outer scope, and lead to a cascading failure. I madea quick fix to the parsing of block statements so that we don't look for a closing `}` if we never had an opening `{`, but that isn't really a systematic solution like we truly need. For now, these fixes will avoid the infinite-loop case, and should give a better diagnostic in the case a user ran into, but we need to take time to do some more top-down work on the parser sooner or later.
2021-02-22Add basic support for fragment shader interlock (FSI) (#1722)Tim Foley
Both D3D "rasterizer ordered views" (ROVs) and GLSL "fragment shader interlock" (FSI) are aimed at the same basic use case: they allow for fragment shaders to contain operations that require mutual exclusion and/or deterinistics ordering between fragment shader invocations that affect the same framebuffer coordinates. The language-level exposure of the features varies greatly between the two API families, though: * ROVs define an implicit ordering and mutual exclusion constraint: certain resoure parameters are marked as `RasterizerOrdered`, and reads/writes to these resources must be sequences *as if* fragment-shader invocations ran in sequential order for each pixel. * FSI defines paired begin/end functions that mark a critical section of code. All memory operations in the critical section must be sequences *as if* fragment-shader invocations ran in sequential order for each pixel. In order to make this model tractable, only a single critical section is allowed per fragment shader, and the begin/end must appear at the top level of the shader entry point function (not under control flow or after a possible conditional `return`. The simplest way for Slang to support portable programs that run across both API families is to insist that code that cares about these ordering guarantees must use *both* mechanisms, and then each of them will only affect the API that cares about it. Slang already supports ROV resource types, and already lowers them to plain textures for GLSL/SPIR-V. This change adds the missing feature of a begin/end function pair for FSI, which will map to empty functions on non-GLSL targets.
2021-02-19Add a chapter on target platforms (#1720)Tim Foley
* Add a chapter on target platforms The primary goals of this chapter are: * Make users aware of just how many different ways of handling things there are across targets. If a user leaves this chapter thinking "how in the world can you abstract over all these differences?", then we have done our job, because they are primed to understand why layout and parameter binding are **necessarily** complicated. * Help users to understand/recall the relevant capabilities and restrictions of the platforms they care about most. If somebody only cares about D3D12 and Vulkan, I want them to leave with a detailed understanding of how those two differ so they can understand the *specifics* of where the layout and parameter-binding algorithms have to treat those targets differently. All of this could conceptually be just a background section in the layout and parameter-binding chapter, but putting it off in its own chapter avoids that one taking forever to actually get where it is going. * Typos
2021-02-19Make gfx library visible to external user. (#1719)Yong He
* Make gfx library visible to external user. * Fixup
2021-02-17Fix typo in user guide.Yong He
2021-02-17Streamline shader object creation (#1717)Tim Foley
This change kind of rolls together two different simplifications: 1. The `createShaderObject()` shouldn't really need to take an `IShaderObjectLayout` because it could just take the `slang::TypeLayoutReflection` instead and create the shader-object layout behind the scenes. 2. For that matter, it needn't take a `slang::TypeLayoutReflection` either, becaues it could just take a `slang::TypeReflection` and query the layout of that type behind the scenes. The combination of these two changes means: * `IShaderObjectLayout` is gone from the public API, as is `createShaderObjectLayout()` * `createShaderObject()` directly takes a `slang::TypeReflection` and allocates a shader object of that type The result is simpler and more streamlined application code. Note that under the hood the implementation still has shader-object layouts, using the `ShaderObjectLayoutBase` class. A few locations had to change to use `RefPtr`s instead of `ComPtr`s now that the class is no longer a public COM-lite API type. The hope is that this change makes it easier to allocate/cache layouts for things like specialized types "under the hood," as is needed to implement parameter setting for static specialization.
2021-02-17Further documentation on Slang specific features (#1716)Yong He
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-02-17Use CPU memory for shader object ordinary data (#1714)Tim Foley
This change makes it so that the shared shader object implementation across graphics APIs (everything except CUDA and CPU) uses a host-memory buffer to store ordinary (aka "uniform") data while the shader object is being set up / modified, and then allocates and initializes a GPU-memory buffer for the data on-demand once setup is complete. This choice is a necessary step for supporting interface/existential-type fields in the presence of static specialization, because any fixed-size GPU buffer we would try to allocate at the time an object is first created might not turn out to be large enough if static specialization must handle a concrete type that doesn't "fit" into the fixed-size space reserved for an existential value (resulting in the value having to be placed in an overflow region outside the original object). This change does *not* include any of the work related to actually laying out existential-type fields in this fashion. It instead just focuses on changing when and where the GPU memory allocation is performed to one that is more appropriate for those subsequent changes.
2021-02-17More #line improvements (#1713)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP: First pass in supporting output of line error information. * Add support for lexing to better be able to indicate SourceLocation information. * Fix lexer usage in DiagnosticSink in C++ extractor. * Update diagnostics tests to have line location info. * Fixed test expected output that now have source location information in them. * Better handling of tab. * Fix test expected results for tabbing change. * DiagnosticLexer -> DiagnosticSink::SourceLocationLexer Added line continuation tests. * Fix typo. * Added String::appendRepeatedChar * Change to rerun tests. * Added source locations to IR dumping. * Output column for IR dump source loc. * Add support for closing brace location to AST. Use closing brace location in lowering when adding return void. * Set the source location through SourceLoc - simplifies identifying if current loc is valid. * Copy terminator sloc. * Test for improved #line handling. * Made writer the last parameter for dumpIR. Small improvements to comments. * Disable sloc output on dump IR by default. * Fix issue with #line and inlining. * Fix for output with improved #line output. * Small comment change - mainly to kick off TC build. Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-02-17Add `SampleGrad` overload for lod clamp. (#1711)Yong He
* Add `SampleGrad` overload for lod clamp. * Fix gfx to run the test on vulkan. * Whitespace change to trigger CI build * remove presentFrame call in render-test Co-authored-by: Yong He <yhe@nvidia.com> Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-02-16Fixes to get shader-object example working on CUDA (#1708)Tim Foley
The purpose of these changes is to make the `shader-object` example work correctly on CUDA. Originally I had tried to add changes to the "flat" reflection information so that it introduced descriptor ranges to match the binding ranges it added for interface/existential-type fields. This approach helped the CUDA code that was using that information to try and compute uniform offsets for those fields, but it broke most of the other renderer back-ends. Instead, I removed the relevant asserts from `CUDAShaderObject::setObject()`. Note taht there are leftover changes from my edits to the flat reflection information, around how it handles "leaf" fields that consume multiple resource kinds. I believe that those changes are, on balance, "more correct" now than they were before, so I decided to leave them in. The other major fix here is to specialize the `CUDAShaderObject::setObject()` logic to handle the case of setting a shader object for a parameter that has interface type instead of a constant-buffer or parameter block. Mostly I just copy bytes from the child object into the parent object. There are a few caveats, though: * I am not writing the RTTI or witness-table information, so dynamic dispatch won't work. * I am assuming a hard-coded offset of 16 bytes for the any-value, which will work for now but is a bit too "magical" and might also break once we support conjunctions of interfaces with dynamic dispatch * I am assuming that the child value to be writen into the field will "fit" into the any-value area. We need some way to determine whether or not things fit dynamically (ideally using the reflection data), and adapt accordingly. * I had to add another method on the base CUDA shader object type to handle setting data using a device-memory pointr instead of a host-memory pointer * There's not a lot we can do about it, but in the case of assigning an ordinary `CUDAShaderObject` into an interface-type field of a `CUDAEntryPointShaderObject` we end up needing to perform a device->host memory copy, because the bytes of the value will have already been written to GPU memory, but need to be in GPU memory for the dispatch call. * The implementation I'm using here basically assumes that the child shader object must have been finalized before it gets plugged into the parent shader object. We haven't yet made a policy decision about that bit.
2021-02-16Add an accessor for IRInst opcode (#1707)Tim Foley
* Add an accessor for IRInst opcode This main changing is renaming `IRInst::op` over to `IRInst::m_op` and then adds an accessor `IRInst::getOp()` to read it. The rest of the changes are just changing use sites to `getOp` (or to `m_op` in the limited cases where we write to it). This work is in anticipation of a future change that might need to store an extra bit in the same field as the opcode. It seemed better to do this massive refactoring as a separate PR. * fixup
2021-02-12Add associated type and generic value parameter doc section (#1706)Yong He
* Add associated type and generic value parameter doc section * Typos and corrections.
2021-02-12Initial support for DXR payload access qualifiers (#1705)Tim Foley
This change adds initial support for a feature being proposed for inclusion in dxc: https://github.com/microsoft/DirectXShaderCompiler/pull/3171. The main features are: * A `[payload]` attribute that indicates which `struct` types are intended to be used as payloads. Consistent use of this attribute should mean that an application no longer needs to manually specify a maximum payload size when creating a ray-tracing pipeline. * `read(...)` and `write(...)` qualifiers which can be attached to fields of `struct` types (usually `[payload]`-attributed types) to indicate which ray tracing pipeline stages are allowed read/write access to that part of the payload. Use of these qualifiers should allow an implementation to optimize storage of ray payload elements across RT pipeline stages. The work in this change just adds basic parsing for these features, translation to matching IR decorations, and then emission of HLSL text based on those decorations. Notable gaps in this first change include: * No work is currently being done to validate access to ray payloads in RT entry points based on these qualifiers. * The stage names in `read(...)` and `write(...)` are not being validated, and are being stored in the IR as text. These should probably use the `Stage` enumeration in some fashion, but we would need to have a way to encode the additional `caller` pseudo-stage that the feature uses. * No work is currently being done to adjust or react to the chosen shader model when emitting HLSL code. We should *either* have these attributes force a switch to a higher shader model, *or* skip emission of these attributes if the chosen shader model / profile does not imply support for them. * No tests are currently included for this work, because tests would rely on using a custom `dxcompiler.dll` build with the new feature supported.
2021-02-12First part of interfaces and generics doc. (#1704)Yong He
Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>
2021-02-12Further documentation work (#1703)Tim Foley
* Move around the conventional/convenience features chapters * Add a first draft of a section on compilation using `slangc` and the COM-lite API Co-authored-by: Yong He <yonghe@outlook.com>
2021-02-12Support `bit_cast` between complex types. (#1702)Yong He
* Support `bit_cast` between complex types. * Fix vs project file * Fix clang build error * fix * fix * Fix * FIx * Fix * Fix * Fix * Fix * Fix linux compile error Co-authored-by: Tim Foley <tfoleyNV@users.noreply.github.com>