summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2021-09-10First Slang LLVM integration (#1934)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * First integration with 'slang-llvm'. * Fix project. * Fix test output. * First pass assert support. * Add inline impls for min and max. * Add abs inline abs impl for llvm. * Make abs not use ternary op * Fix typo in slang-llvm.h * Sundary fixes to make remaining tests using llvm backend pass.
2021-08-26Add API to control interface specialization. (#1925)Yong He
2021-08-17Add GLSL450 intrinsics to SPIRV direct emit. (#1921)Yong He
* Add GLSL450 intrinsics to SPIRV direct emit. * Fix. * Fix compiler error. * Fix. * Fix compiler error. * Make direct-spirv tests actually run.
2021-08-12Further implementation of SPIRV direct emit. (#1920)Yong He
* Further implementation of SPIRV direct emit. This change implements: - Struct, Vector, Matrix and Unsized Array types. - Basic arithmetic opcodes, vector construct, swizzle etc. - getElementPtr, getElement, fieldAddress, extractField. - SPIRV target intrinsics with SPIRV asm code in stdlib. - RWStructuredBuffer and StructuredBuffer. - Pointer storage class propagation. - Control flow. * Fix.
2021-07-28Experimental DXR1.0 support in gfx. (#1915)Yong He
* Experimental DXR1.0 support in gfx. - Add `dispatchRays` command. - Add `createRayTracingPipelineState` method to construct a D3D ray tracing state object from a linked slang program and user specified shader table. Limitations/simplifications: no local root signature support, shader table entries contains only shader identifiers and is specified at pipeline creation time, owned by the pipeline state object. * Root object binding for raytracing pipelines. * `maybeSpecializePipeline` implementation for raytracing pipelines. * Add ray-tracing-pipeline example. * Fixes. * Update README.md * Update comments on the lifespan of specialized pipelines Co-authored-by: Yong He <yhe@nvidia.com> Co-authored-by: jsmall-nvidia <jsmall@nvidia.com>
2021-07-20Minor refactor to gfx D3D12 implementation. (#1913)Yong He
* Minor refactor to gfx D3D12 implementation. - Allow more flexible collection of shader stages in a shader program. - Add `createRayTracingPipelineState` public interface. (no implementation). * Fix Vulkan initialization. Co-authored-by: Yong He <yhe@nvidia.com>
2021-07-19Enable swiftshader in linux CI builds (#1909)Yong He
2021-07-09Enable testing with Swiftshader. (#1906)Yong He
2021-07-09[gfx] Inline raytracing fixes in response to code review. (#1905)Yong He
Co-authored-by: Yong He <yhe@nvidia.com>
2021-07-08Allow render-test to run inline ray tracing tests. (#1903)Yong He
* Update VS projects to 2019. * Empty commit to trigger build * Implement gfx inline ray tracing on D3D12. * Allow render-test to run inline ray tracing tests. Co-authored-by: Yong He <yhe@nvidia.com>
2021-07-08Implement gfx inline ray tracing on D3D12. (#1902)Yong He
* Update VS projects to 2019. * Empty commit to trigger build * Implement gfx inline ray tracing on D3D12.
2021-06-30[gfx] Add inline ray tracing support. (#1899)Yong He
2021-06-25Fixes related to combined texture/sampler types (#1894)Theresa Foley
* Fixes related to combined texture/sampler types Work on #1891 Our intention has always been to support combined texture/sampler types in Slang, both targets like OpenGL where that is the only option available and for targets like Vulkan where it can be beneficial to performance. Because Slang's current users mostly focus on D3D12+Vulkan codebases, they strongly prefer separate textures and samplers, and the relevant support code in Slang has "bit-rotted" over many releases until the functionality that was there isn't useful any more. This change significantly overhauls the implementation of combined texture/sampler types and adds a test that uses them in the hopes of avoiding future regressions. The new combined texture/sampler types use the prefix `Sampler`, so where there is an existing standalone `Texture2D` type the equivalent combined texture/sampler type will be `Sampler2D`. The intention is that this naming mirrors the GLSL conventions (where the type is `sampler2d`) while following HLSL naming precedent (to the extent it exists). The operations available on a `Sampler2D` are intended to be those that are available on a `Texture2D`, and it is just that in cases where the `Texture2D` operation would take a separate sampler argument: Texture2D t = ...; SamplerState s = ...; float4 result = t.Sample(s, uv); the equivalent `Sampler2D` operation just elides that argument: Sampler2D s = ...; float4 result = s.Sample(uv); In terms of implementation, there are a lot of subtle details here: * I've tried to use the same metaprogramming logic that generates all the stdlib declarations for `Texture*` to also generate `Sampler*` in the hopes that this helps keep them in sync. * The big catch to the above is that it means that for certain operations the indidces of parameters depend on whether or not an explicit sampler parameter is used. Rather than try to tweak the indices in the stdlib generation logic (which is already complicated) I went and added Yet Another Hack to the logic that handles intrinsic definition strings. Basically, the special-case handling of `$p` has been modified so that it *also* applies a negative offset to future parameter references in the same intrinsic string. * Trying to actually bring this up in our test framework revealed that the "flat" reflection API was seemingly not reflecting combined texture/sampler types correctly at all (it was reflecting them as just plain textures). Other than that issue, the Vulkan path seems to work fine with combined texture/samplers. * I also had to add logic to the `TEST_INPUT` parsing to re-introduce handling of the combined types (that was something I consciously left out to reduce the amount of code in the earlier refactor there). Luckily, the architecture is such that a combined texture/sampler can leverage most of the existing logic for the separate cases. * fixup: reveiw feedback
2021-06-24Remove StructTag and associated systems (#1895)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Remove StructTag and associated systems. * Fix typo and remove unit test for StructTag.
2021-06-23[gfx] Add `IBufferResource::getDeviceAddress()`. (#1892)Yong He
2021-06-18StructTag versioning (#1888)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP Abi struct. * Use AbiSystem on SessionDesc. * Use mask/shift constants. * Fix issue causing warning on linux. * Abi -> Api. * Fix typo. * Refactor to use StructTag. * Mechanism to be able to follow fields. * Field adding is working. * WIP with StructTagConverter. * First pass of StructTag appears to work. Still needs diagnostics. * Small tidy up around Field. * Use bit field to record what fields are recorded to remove allocation around the m_stack. Use ScopeStack for RAII. * Return SlangResult instead of pointers. * Use SlangResult with copy. * Split StructTagConverter implementations. * Fix some bugs around lazy converting. * First pass at unit test for StructTag. * Testing StructTag going backwards in time. * First pass as StructTag diagnostics. * Make Traits a namespace. * Fix some issues with Traits not being a class. * Fix 32 bit warning.
2021-06-11Properly fill `declref` in `Linkage::getContainerType`. (#1882)Yong He
* Properly fill `declref` in `Linkage::getContainerType`. * Fix timestamp query on cpu * Fix typo. Co-authored-by: Yong He <yhe@nvidia.com>
2021-06-10CUDA layout corner cases/testing (#1881)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Add support for sizeOf/alignOf/offsetOf to stdlib. Add $G intrinsic expansion that works of the generic parameters not the param type * Test cuda layout. * Fix CUDA layout issues. Fix reflection to handle other built in types. Fix __offsetOf * Tests of reflection and layout as reported directly from CUDA. * Comment about use of aligned size as size. * Fix warning from VS. * Check alignment is pow2. * Small improvements to alignment calcs. * Tab to spaces. * Fix alignment pointer sizes on 32 bit OS for CUDA. * Fix CUDA reflection on 32 bit.
2021-06-10Support timestamp queries in `gfx`. (#1880)Yong He
* Support timestamp queries in `gfx`. * Fix tab Co-authored-by: Yong He <yhe@nvidia.com>
2021-06-08Various fixes to CUDA backend. (#1877)Yong He
- Fix emitting `StructuredBuffer<ISomething>::Load`, which triggers emitting for `IROp_WrapExistential` that is previously unhandled. - Fix cuda layout around vectors, they should be aligned to 1,2,4,8,16 bytes instead of just using element type's alignment. That means `float4` has alignment of 16 instead of 4. - Fix `SLANG_CUDA_HANDLE_ERROR` macro definition. - Fix navis sometimes fail to find `Slang::kIROp_*` enum values when debugging external projects. Co-authored-by: Yong He <yhe@nvidia.com> Co-authored-by: jsmall-nvidia <jsmall@nvidia.com>
2021-06-08Fix RWTexture issues on CUDA (#1876)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Re-enable CUDA RWTexture tests. Re-enable RWTexture1D test Make sure tests have only single mip for RWTexture (required for CUDA) * Fix issue with reading CUDA surface. Re-enable working CUDA RWTextureTest. Enable 1D case.
2021-06-06Fixed issue around 4xFloat16 texture on CUDA (#1874)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Fixes around Float16. Incorrect calculation of 'elementSize'.
2021-06-04Fix D3D11 `uploadBufferResource`. (#1869)Yong He
2021-06-04Escape `\` in slang-embed. (#1870)Yong He
2021-06-02Various Fixes to gfx, reflection and emit. (#1867)Yong He
* Various Fixes to gfx, reflection and emit. - Fix GLSL emit to properly output `*bitsTo*` functions for `IRBitCast` insts. - Add line directive mode setting for `ISession`. - Extend `TypeLayout::getElementStride` to handle `VectorType` case. - Fix `IDevice::readBufferResource` 's D3D12 implementation to copy only the requested bytes out. - Fix `render-test` to use the `ISession` from `gfx` instead of creating its own `ISession` to make sure `gfx` and `render-test` agree on WitnessTable and RTTI IDs. - Extend `render-test` to support filling vector and matrix values in the new `set x = ...` TEST_INPUT syntax. - Add a `dynamic-dispatch-15` test case to make sure packing / unpacking works correctly across all targets, and to make sure render-test's RTTI/WitnessTable ID filling logic is correct for non-trivial cases. * Remove default-major test * Fix cyclic reference in `ExtendedTypeLayout`. * Move `lineDirectiveMode` setting to `TargetDesc`. Add `structureSize` to `TargetDesc` and `SessionDesc` for future binary compatibility. * Cleanup. Co-authored-by: Yong He <yhe@nvidia.com>
2021-06-02JSONBuilder (#1865)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP JSONWriter/JSONParser. * Checking different Layout styles for JSON. * Add slang-json-parser.h/.cpp * WIP JSONValue. * Added JSONValue::destroy/Recursive. * Improvement to JSONValue. * Improve text double conversion precision. Testing. * Simplify double parsing (just use atof). JSON comparison More testing of conversions and start of JSONValue. * Add <math.h> for isnan, isinf etc. * Small improvement with object comparison. * Fix typo in getArgsByName. * Removed use of isnan and isinf as includes don't work on linux. * Improve JSON unit test. * Added asInteger/asFloat/asBool to JSONValue. * Add SourceLoc to JSONListener. * Added ability to walk the JSONValue * JSONBuilder. * Add converting from lexemes via JSONBuilder. * Fix VS warning. * Fix warning for res not being used.
2021-06-01JSONValue / Container (#1864)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP JSONWriter/JSONParser. * Checking different Layout styles for JSON. * Add slang-json-parser.h/.cpp * WIP JSONValue. * Added JSONValue::destroy/Recursive. * Improvement to JSONValue. * Improve text double conversion precision. Testing. * Simplify double parsing (just use atof). JSON comparison More testing of conversions and start of JSONValue. * Add <math.h> for isnan, isinf etc. * Small improvement with object comparison. * Fix typo in getArgsByName. * Removed use of isnan and isinf as includes don't work on linux. * Improve JSON unit test. * Added asInteger/asFloat/asBool to JSONValue. * Change comment to trigger CI build.
2021-05-27JSON Parser and Writer (#1859)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP JSONWriter/JSONParser. * Checking different Layout styles for JSON. * Add slang-json-parser.h/.cpp
2021-05-25JSON Lexing and string encoding/decoding (#1858)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP Json lexer. * Check JSON Lex with unit test * Add JSON escaping/unescaping of strings. * Big fix encoding/decoding. * Fix typo in JSON diagnostics. * Fix typo. * Better float testing.
2021-05-25Rework shader object specialization control interface. (#1857)Yong He
2021-05-25Allow overriding specialization args via `IShaderObject`. (#1854)Yong He
* Allow overriding specialization args via `IShaderObject`. * Fixes. Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
2021-05-22Improvements in -X support (#1852)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Added SourceLoc handling for command line parsing. * Fix typo in debug. * Fix issue around the DiagnosticSink used in options parsing not having a writer available - by having DiagnosticSink parenting. * Small rename for clarity. * WIP extracting command line args for downstream tools. * Unit tests/bug fixes around extracting args. * Use DownstreamArgs in the EndToEndCompileRequest * Passing downstream compiler options downstream. * Fix issue with endToEndReq being nullptr. * Fix issue with diagnostics number change. * Small improvements to how the source line is displayed if it's too long. Default to 120, as suggested in previous review. * Make render test use x-args parsing and CommandArgReader. * Added missing diagnostics. * More DownstreamArgs to linkage so can be seen by 'components'. Added dxc-x-arg test. * Used combination of name and args instead of two Lists, which whilst equivalent was perhaps a little confusing. * Added documentation for -X support. * Added test for x-args parsing diagnostic. Improved diagnostic with list of known names. * Fix issues from merge. * Fix lookup for -matrix-layout-column-major in render test. * Remove commented out line.
2021-05-21[gfx] Support StructuredBuffer<IInterface>. (#1851)Yong He
Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
2021-05-21Downstream option handling (#1850)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Added SourceLoc handling for command line parsing. * Fix typo in debug. * Fix issue around the DiagnosticSink used in options parsing not having a writer available - by having DiagnosticSink parenting. * Small rename for clarity. * WIP extracting command line args for downstream tools. * Unit tests/bug fixes around extracting args. * Use DownstreamArgs in the EndToEndCompileRequest * Passing downstream compiler options downstream. * Fix issue with endToEndReq being nullptr. * Fix issue with diagnostics number change. * Small improvements to how the source line is displayed if it's too long. Default to 120, as suggested in previous review. Co-authored-by: T. Foley <tfoleyNV@users.noreply.github.com>
2021-05-21Overhaul the preprocessor (#1849)T. Foley
* Overhaul the preprocessor The old Slang preprocessor was based on a simple mental model that tried to unify two parts of macro expansion: * Scanning for macro invocations in a sequence of tokens * Producing the expanded tokens for a macro expansion by substituting arguments into its body The basic was that substitution of macro arguments into a macro definition is superficially similar to top-level macro expansion, just with an environment where the macro arguments act like `#define`s for the corresponding parameter names. That approach was "clever" and could conceivably have been extended to include a lot of advanced preprocessor features (e.g., a preprocessor-level `lambda` would be easy to support!), but it was basically impossible to make it correctly handle all the corner cases of the full C/C++ preprocessor. The fundamental problem with the old approach was that it conflated the two parts of expansion listed above into one implementation, while the various special cases of the C/C++ preprocessor rely on treating the two cases very differently. The new approach here (which is somewhere between a refactor and a full rewrite of the preprocessor) changes things up in a few key ways: * The abstraction still cares a lot about streams of tokens, but it now treats the top level streams (`InputFile`s) as fairly different from the lower-level streams (`InputStream`s) * Macro expansion is handled as a dedicated type of stream that wraps another stream. This allows macro expansion to be applied to anything, and supports cases where multiple rounds of macro expansion are required by the spec. * Macro *invocations* and the substitution of their arguments are now handled by a completely new system. * Macro arguments are no longer treated as if they were `#define`s * The macro body/definition is analyzed at definition time to detect various kinds of issues, and to derive a list of "ops" that make it easier to "play back" the definition at substitution time * Token pasting and stringizing are now only handled in macro definitions (rather than being allowed anywhere), and their use cases are restricted to only those that make sense (e.g., you can't stringize anythign except a macro parameter, because anything else wouldn't make sense) The key new types here are the `ExpansionInputStream` which handles scanning for macro invocations, and the `MacroInvocation` type, which handles playing back the macro body with substitutions. The `ExpansionInputStream` is the easier of the two to understand. By refactoring it to use a single token of lookahead, the one major detail it had to deal with before (abandoning expansion of a function-like macro if the macro name was not followed by `(`) is significantly easier to manage. The more subtle part is the `MacroInvocation` type, and most of the complexity there is around handling of token pasting, and the fact that either or both of the operands to a token paste might be empty. Many of the test cases that exposed the problems in the preprocessor have been moved from `current-bugs` to `preprocessor` since they now work correctly. * debugging: enable extractor command line dump * fixup * fixup
2021-05-19Glslang as DownstreamCompiler (#1846)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP Fxc as downstream compiler. * First pass FXC downstream compiler working. * GCC compile fix. * Fix FXC parsing issue. * Special case filesystem access. * Use StringUtil getSlice. * Fix isses with not emitting source for FXC. * WIP on DXC. * Small fixes for DXBC handling. * Removed DXC from ParseDiagnosticUtil (can use generic) Try to improve output for notes from DXC. * FIrst pass of Glslang as DownstreamCompiler * Fix some problems with parsing for glslang replacement. * Add slang-glslang-compiler.cpp/.h * Fix downstream for spir-v output. * dissassemble -> disassemble * Fix typo and improve some naming/comments. * Remove getSharedLibrary from DownstreamCompiler * Removed some no longer used diagnostics.
2021-05-14DXC as DownstreamCompiler (#1845)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP Fxc as downstream compiler. * First pass FXC downstream compiler working. * GCC compile fix. * Fix FXC parsing issue. * Special case filesystem access. * Use StringUtil getSlice. * Fix isses with not emitting source for FXC. * WIP on DXC. * Small fixes for DXBC handling. * Removed DXC from ParseDiagnosticUtil (can use generic) Try to improve output for notes from DXC.
2021-05-14FXC as DownstreamCompiler (#1844)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP Fxc as downstream compiler. * First pass FXC downstream compiler working. * GCC compile fix. * Fix FXC parsing issue. * Special case filesystem access. * Use StringUtil getSlice. * Fix isses with not emitting source for FXC. * Small fixes for DXBC handling.
2021-05-06Half texture support (#1836)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out StringEscapeUtil. * Added StringEscapeUtil. * Fix typo in unix quoting type. * Small comment improvements. * Try to fix linux linking issue. * Fix typo. * Attempt to fix linux link issue. * Update VS proj even though nothing really changed. * Fix another typo issue. * Fix for windows issue. Fixed bug. * Make separate Utils for escaping. * Fix typo. * Split out into StringEscapeHandler. * Windows shell does handle removing quotes (so remove code to remove them). * Handle unescaping if not initiating using the shell. * Slight improvement around shell like decoding. * Simplify command extraction. * Add shared-library category type. * Fix bug in command extraction. * Typo in transcendental category. * Enable unit-test on in smoke test category. * Make parsing failing output as a failing test. * Fixes for transcendental tests. Disable tests that do not work. * Changed category parsing. * Removed the TestResult parameter from _gatherTestsForFile. Made testsList only output. * Remove testing if all tests were disabled. * Make args of CommandLine always unescaped. * Add category. * Don't need escaping on unix/linux. * Remove some no longer used functions. * Add requireSMVersion to CUDAExtensionTracker. * half-calc.slang now works for CUDA. * bit-cast-16-bit works on CUDA. * WIP handling of CUDA vector<half> types. * Half swizzle CUDA. * Half vector test. * Fix swizzle half bug. * Fix compilation issue with narrowing to Index. * Add unary ops. * Add some vector scalar maths ops. * Add half vector conversions for CUDA. * Fix erroneous comment. * Support for half comparisons. * First pass test for half compare. * Fix bug in CUDA specialized emit control. Updated tests to have pre and post inc/dec. * Removed unneeded parts of the cuda prelude. * Half structured buffer works on CUDA. * Added name lookup for Gfx::Format * Support half texture type in test system. * Test for half reading on CUDA. * Add half formats to Vk and D3D utils. * Fix getAt for CUDA - where there might not be a .x member in a vector.
2021-05-04Cleanup work on D3D12 shader object static specialization (#1830)Tim Foley
* Cleanup work on D3D12 shader object static specialization This builds on PR #1829 in a small way (because that PR adds `getStride()` to Slang type layout reflection). The only relevant changes here are in the `render-d3d12.cpp` file. The basic idea here is to clean up the D3D12 path to be more in line with the cleanups made for D3D11 and Vulkan. The way that D3D12 shader parameter binding goes through a root signature means that some of the details that were required for those APIs (in particular, tracking both "primary" and "pending" offsets during multiple steps) are not required for the critical-path binding stuff on D3D12. There is some subtlety to the handling of the "ordinary" data buffer in the `bindAsValue()` case that I don't like, and that I'm not 100% confident I've gotten right. We may find that we have to revisit that logic as we add more tests. * fixup
2021-04-30Clean up the way we stride over sub-object ranges for D3D11 and Vulkan (#1829)Tim Foley
This change applies to the case where we have a sub-object range with more than one object in it (so arrays of constant buffers, parameter blocks, or existentials). It is worth noting that we don't really seem to have any tests covering this case right now, so it is entirely possible that things are busted already, and/or that this change doesn't work the way I think it does. When we go to actually bind the state from a sub-object range into the pipeline, we care about: * The offset to where the first object should be written * The "stride" between consecutive objects We were already capturing the offset information from Slang reflection data, and the same was true for the part of the offset that pertains to "pending" ordinary data. For other cases, though, we were manually incrementing the offset by values computed manually in `gfx` for Vulkan, and we were just skipping the offsetting step entirely for D3D11. With this change we extract more complete stride information from the Slang reflection data and use that instead of ad hoc computations. Hopefully this data is correct, and if it isn't we can consider whether it needs fixing at the Slang reflection level rather than in `gfx`. This change doesn't apply to the OpenGL back-end (which hasn't been updated to match the new approach to static specialization at all) or to the D3D12 back end (which has been updated a bit, but probably needs other cleanup work to be done first to bring it in line).
2021-04-29Update gfx back-ends to handle static specialization (#1826)Tim Foley
* Update gfx back-ends to handle static specialization The main goal here is to make the D3D11, D3D12 and Vulkan back-ends support static specialization of interface types in the case where the data for the type won't "fit" in the pre-allocated space for existential values. This includes all cases where the concrete type being specialized to has resources/samplers/etc., as well as any cases where its ordinary/uniform data exceeds the space available. (Note that the CPU and CUDA targets don't need this work since they can (in theory) support arbitrary-size data in the fixed-size existential payload by using pointer indirection. Actually supporting indirection in those cases should be a distinct change) The Slang compiler already performs layout for programs that have this kind of data that doesn't "fit," and it lays them out using an idea of "pending" type layouts. Basically, a type that contains some amount of specialized interface-type fields will produce both a "primary" type layout that just covers the data for the unspecialized case, as well as "pending" type layout that describes the layout for all the extra data needed by specialization. When laying out a `ConstantBuffer<X>` or `ParameterBlocK<X>` ("CB" or "PB"), the front-end will try to place as much of that "pending" data into the layout of the buffer/block itself as is possible. That means that both CBs and PBs will be able to allocate trailing bytes for any ordinary data in the "pending" layout. PBs will be able to allocate any trailing resources/samplers into their layout, but for CBs they will spill out to be part of the pending layout for the buffer itself. In order for the back-ends to properly handle pending data, they need to *either* assume the exact layout rules used by the front-end and try to reproduce them (e.g., by iterating over binding ranges and sub-objects in the exact same order that front-end layout would enumerate them), *or* they need to respect the reflection information produced by the front-end. This change takes the latter approach, trying to make only minimal assumptions about the layout rules being used. This choice is motivated by wanting to decouple the `gfx` implementation from the compiler front-end, especially insofar as this work has made me question whether the current layout rules are the best ones possible. A common theme across all the implementations is to have a fixed-size type that can represent "binding offsets" for the chosen back-end. The offset type has fields that depend on the API-specific way bindings are indexed; e.g., for D3D11 it has offsets for CBV, SRV, UAV, and sampler bindings. This fixed-size offset type can be filled in based on Slang reflecton information, and then used to compute derived offsets with just a few add operations. The simple offset type for each API is then extended to produce an offset type that includes both the offsets for "primary" data and also the offsets for "pending" data. Most logic that traffics in offsets doesn't have to know about this more complicated representation. Making consistent use of these offsets required that I pretty much rewrite the logic that actually applies shader objects to the API state. Doing so might be lowering the efficiency of the system in the near term, but the increase in clarity was important for getting the work done, and it seems like it will also be important if/when we start trying to perform special-case optimizations around root and entry-point parameter setting. While there are many API-specific differences, we can identify a repeated pattern where many steps, whether applying parameters to the pipeline stage or constructing signatures / layouts, can be broken down into three main operations on `ShaderObject`s or their layouts: * `*AsValue()` is the core operation, and is the one used for the `ExistentialValue` case most of the time. It ignores the ordinary data in the object, and instead processes all nested binding ranges (for resources/smaplers) and sub-objects. * `*AsConstantBuffer()` handles the `ConstntBuffer<X>` case, by dealing with the implicit buffer for ordinary data (if it is needed) and then delegates to the `*AsValue()` case. * `*AsParameterBlock()` handles the `ParameterBlock<X>` case, by allocating/preparing/etc. any descriptor tables/sets that would be required for the current object/layout and then delegating to `*AsConstantBuffer()` to do the rest The idea is that by having the parameter block case delegate to the constant buffer case, which delegates to the value/existential case, we can streamline a lot of the logic so that it doesn't seem quite as full of special cases. Note: When preparing this pull request I spent a reasonable amount of time trying to clean up the D3D11 and Vulkan implementations, so they are probably the easiest to read and understand when it comes to the new code. Doing the cleanup work also helped to work out some weird corner case bugs/issues. In contrast, the D3D12 path hasn't had as much attention given to cleanliness and comments, so it really needs some attention down the line to get things into a state that is easier to understand. * fixup: remove debugging code spotted in review
2021-04-29`gfx` DebugCallback and debug layer. (#1822)Yong He
* `gfx` DebugCallback and debug layer.
2021-04-29Simplify CommandLine by removing Escaping (#1825)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out StringEscapeUtil. * Added StringEscapeUtil. * Fix typo in unix quoting type. * Small comment improvements. * Try to fix linux linking issue. * Fix typo. * Attempt to fix linux link issue. * Update VS proj even though nothing really changed. * Fix another typo issue. * Fix for windows issue. Fixed bug. * Make separate Utils for escaping. * Fix typo. * Split out into StringEscapeHandler. * Windows shell does handle removing quotes (so remove code to remove them). * Handle unescaping if not initiating using the shell. * Slight improvement around shell like decoding. * Simplify command extraction. * Add shared-library category type. * Fix bug in command extraction. * Typo in transcendental category. * Enable unit-test on in smoke test category. * Make parsing failing output as a failing test. * Fixes for transcendental tests. Disable tests that do not work. * Changed category parsing. * Removed the TestResult parameter from _gatherTestsForFile. Made testsList only output. * Remove testing if all tests were disabled. * Make args of CommandLine always unescaped. * Add category. * Don't need escaping on unix/linux. * Remove some no longer used functions.
2021-04-29Support for escaped paths in tools (#1823)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split out StringEscapeUtil. * Added StringEscapeUtil. * Fix typo in unix quoting type. * Small comment improvements. * Try to fix linux linking issue. * Fix typo. * Attempt to fix linux link issue. * Update VS proj even though nothing really changed. * Fix another typo issue. * Fix for windows issue. Fixed bug. * Make separate Utils for escaping. * Fix typo. * Split out into StringEscapeHandler. * Windows shell does handle removing quotes (so remove code to remove them). * Handle unescaping if not initiating using the shell. * Slight improvement around shell like decoding. * Simplify command extraction. * Add shared-library category type. * Fix bug in command extraction. * Typo in transcendental category. * Enable unit-test on in smoke test category. * Make parsing failing output as a failing test. * Fixes for transcendental tests. Disable tests that do not work. * Changed category parsing. * Removed the TestResult parameter from _gatherTestsForFile. Made testsList only output. * Remove testing if all tests were disabled. * Fix typo. * Disable path canonical test on linux because CI issue.
2021-04-24Remove resource `Usage` from `gfx` interface. (#1813)Yong He
* Fix `model-viewer` crash when using Vulkan. Fixing an issue in shader object layout creation for to make sure a correct descriptor set layout is calculated for types that need an implicit constant buffer. * Fix formatting. * Fixes. * Fix memory leak in vulkan. * Remove resource `Usage` from `gfx` interface.
2021-04-23Fix `model-viewer` crash when using Vulkan. (#1804)Yong He
2021-04-23Preliminary CUDA Half support (#1808)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * WIP CUDA half support. * Working support for half on CUDA - requires cuda_fp16.h and associated files can be found. * Fix for win32 for unused funcs. * Fix for Clang. * Hack to disable unused local function warning.
2021-04-22C++ extractor improvements (#1803)jsmall-nvidia
* #include an absolute path didn't work - because paths were taken to always be relative. * Split of NodeTree. Split out FileUtil. Split out MacroWriter. * Rename slang-cpp-extractor-main.cpp -> cpp-extractor-main.cpp * First pass at extractor unit-tests * Initial parsing of enum. * Ability to disable/enable parsing of scope types. * Initial support for typedef. * Added operator== != to ArrayVIew. Added test for splitting to unit tests. * Improve comment in StringUtil. * Fix comment. * Fix typo.
2021-04-20Various fixes to make `model-viewer` example almost working. (#1801)Yong He
* Fixing `PseudoPtr` legalization and `gfx` lifetime issues. * Fixing `model-viewer` example. This change contains various fixes to bring `model-viewer` example to fully functional. These fixes include: 1. Add `spReflectionTypeLayout_getSubObjectRangeSpaceOffset` function to return the space index for a sub object referenced through a `ParameterBlock` binding. 2. Make sure `D3D12Device` specifies column major matrix order creating a Slang session. 3. Fix `platform::Window::close()` and `platform::Application::quit()`. 4. Fix memory leak during `model-viewer''s model loading. 5. Fix command buffer recording in `model-viewer`. With these changes, model viewer can now produce an image with a gray cube. The lighting is still incorrect becuase the `gfx` shader object implementation still does not handle "pending layout" resulting from global existential parameters. * Fix d3d12 root signature creation. * Use row-major matrix layout in model-viewer