| Age | Commit message (Collapse) | Author |
|---|
|
* Fields from upper to lower case in slang-ast-decl.h
* Lower camel field names in slang-ast-stmt.h
* Fix fields in slang-ast-expr.h
* slang-ast-type.h make fields lowerCamel.
* slang-ast-base.h members functions lowerCamel.
* Method names in slang-ast-type.h to lowerCamel.
* GetCanonicalType -> getCanonicalType
* Substitute -> substitute
* Equals -> equals
ToString -> toString
* ParentDecl -> parentDecl
Members -> members
* * Make hash code types explicit
* Use HashCode as return type of GetHashCode
* Added conversion from double to int64_t
* Split Stable from other hash functions
* toHash32/64 to convert a HashCode to the other styles.
GetHashCode32/64 -> getHashCode32/64
GetStableHashCode32/64 -> getStableHashCode32/64
* Other Get/Stable/HashCode32/64 fixes
* GetHashCode -> getHashCode
* Equals -> equals
* CreateCanonicalType -> createCanonicalType
* Catches of polymorphic types should be through references otherwise slicing can occur.
* Fixes for newer verison of gcc.
Fix hashing problem on gcc for Dictionary.
* Another fix for GetHashPos
* Fix signed issue around GetHashPos
|
|
* Initial work to support OptiX output for ray tracing shaders
This change represents in-progress work toward allowing Slang/HLSL ray-tracing shaders to be cross-compiled for execution on top of OptiX. The work as it exists here is incomplete, but the changes are incremental and should not disturb existing supported use cases.
One major unresolved issue in this work is that the OptiX SDK does not appear to set an environment variable
Changes include:
* Modified the premake script to support new options for adding OptiX to the build. Right now the default path to the OptiX SDK is hard-coded because the installer doesn't seem to set an environment variable. We will want to update that to have a reasonable default path for both Windows and Unix-y platforms in a later chance.
* I ran the premake generator on the project since I added new options, which resulted in a bunch of diffs to the Visual Studio project files that are unrelated to this change. Many of the diffs come from previous edits that added files using only the Visual Studio IDE rather than by re-running premake, so it is arguably better to have the checked-in project files more accurately reflect the generated files used for CI builds.
* The "downstream compiler" abstraction was extended to have an explicit notion of the kind of pipeline that shaders are being compiled for (e.g., compute vs. rasterization vs. ray tracing). This option is used to tell the NVRTC case when it needs to include the OptiX SDK headers in the search path for shader compilation (and also when it should add a `#define` to make the prelude pull in OptiX). This code again uses a hard-coded default path for the OptiX SDK; we will need to modify that to have a better discovery approach and also to support an API or command-line override.
* One note for the future is that instead of passing down a "pipeline type" we could instead pass down the list/set of stages for the kernels being compiled, and the OptiX support could be enabled whenever there is *any* ray tracing entry point present in a module. That approach would allow mixing RT and compute kernels during downstream compilation. We will need to revisit these choices when we start supporting code generation for multiple entry points at a time.
* The CUDA emit logic is currently mostly unchanged. The biggest difference is that when emitting a ray-tracing entry point we prefix the name of the generated `__global__` function with a marker for its stage type, as required by the OptiX runtime (e.g., a `__raygen__` prefix is required on all ray-generation entry points).
* The `Renderer` abstraction had a bare minimum of changes made to be able to understand that ray-tracing pipelines exist, and also that some APIs will require the name of each entry point along with its binary data in order to create a program.
* The `ShaderCompileRequest` type was updated so that only a single "source" is supported (rather than distinct source for each entry point), and also the entry points have been turned into a single list where each entry identifies its stage instead of a fixed list of fields for the supported entry-point types.
* The CUDA compute path had a lot of code added to support execution for the new ray-tracing pipeline type. The logic is mostly derived from the `optixHello` example in the OptiX SDK, and at present only supports running a single ray-generation shader with no parameters. The code here is not intended to be ready for use, but represents a signficiant amount of learning-by-doing.
* The `slang-support.cpp` file in `render-test` was updated so that instead of having separate compilation logic for compute vs. rasterization shaders (which would mean adding a third path for ray tracing), there is now a single flow to the code that works for all pipeline types and any kind of entry points.
* Implicit in the new code is dropping support for the way GLSL was being compiled for pass-through render tests, which means pass-through GLSL render tests will no longer work. It seems like we didn't have any of those to begin with, though, so it is no great loss.
* Also implicit are some new invariants about how shaders without known/default entry points need to be handled. For example, the ray tracing case intentionally does not fill in entry points on the `ShaderCompileRequest` and instead fully relies on the Slang compiler's support for discovering and enumerating entry points via reflection. As a consequence of those edits the `-no-default-entry-point` flag on `render-test` is probably not working, but it seems like we don't have any test cases that use that flag anyway.
Given the seemingly breaking changes in those last two bullets, I was surprised to find that all our current tests seem to pass with this change. If there are things that I'm missing, I hope they will come up in review.
* fixup: issues from review and CI
* Some issues noted during the review process (e.g., a missing `break`)
* Fix logic for render tests with `-no-default-entry-point`. I had somehow missed that we had tests reliant on that flag. This required a bit of refactoring to pass down the relevant flag (luckily the function in question was already being passed most of what was in `Options`, so that just passing that in directly actually simplifies the call sites a bit.
* There was a missing line of code to actually add the default compute entry points to the compile request. I think this was a problem that slipped in as part of some pre-PR refactoring/cleanup changes that I failed to re-test.
|
|
|
|
* CUDA generated first test compiles.
* WIP on enabling CUDA in render-test.
* Detect CUDA_PATH environmental variable to build build cuda support into render-test.
Added WIP cuda-compute-util.cpp/h
Added CUDA as a renderer type.
* Fix libraries needed for cuda in premake.
* Added -enable-cuda premake option. Defaults to false.
* Creates CUDA device, loads PTX and finds entry point.
* Fix some erroneous cruft from slang-cuda-prelude.h
|
|
* Initial work for "global generic value parameters"
The main new feature here is support for the `__generic_value_param` keyword, which introduces a *global generic value parameter*.
For example:
__generic_value_param kOffset : uint = 0;
This declaration introduces a global generic value parameter `kOffset` of type `uint` that has a nominal default value of zero.
The broad strokes of how this feature was added are as follows:
* A new `GlobalGenericValueParamDecl` AST node type is introduces in `slang-decl-defs.h`
* A new `parseGlobalGenericValueParamDecl` subroutine is added to `slang-parser.cpp`, and is added to the list of declaration cases as the callback for the `__generic_value_param` name.
* Cases for `GlobalGenericValueParamDecl` are added to the declaration checking passes in `slang-check-decl.cpp`, mirroring what is done for other variable declaration cases.
* A case for `GlobalGenericValueParamDecl` is aded to the `Module::_collectShaderParams` function, so that it is recognized as a kind of specialization parameter. This introduces a specialization parameter of flavor `SpecializationParam::Flavor::GenericValue` (which was already defined before this change, although it was unused).
* A case for `SpecializationParam::Flavor::GenericValue` is added in `Module::_validateSpecializationArgsImpl` to check that a specialization argument represents a compile-time-constant value (not a type).
* A case for `GlobalGenericValueParmDecl` is introduced in `slang-lower-to-ir.cpp` that introduces a global generic parameter in the IR
* The `IRBuilder` is extended to support creating `IRGlobalGenericParam`s for the distinct cases of type, witness-table, and value parameters. The same IR instruction type/opcode is used for all cases, and only the type of the IR instruction differs.
* The existing mechanisms for lowering specialization arguments to the IR, and doing specialization on the IR itself Just Work with global generic value parameters since they already support value parameters on explicit generic declarations.
That's the santized version of things, but there were also a bunch of cleanups and tweaks required along the way:
* The `SpecializationParam` type was extended to also track a `SourceLoc` to help in diagnostic messages, which meant some churn in the code that collects specialization parameters.
* The `_extractSpecializationArgs` function is tweaked to support any kind of "term" as a specialization argument (either a type or a value).
* To allow *parsing* specialization arguments that can't possibly be types (e.g., integer literals) we replace the existing `parseTypeString` routine with `parseTermString` and then in `parseTermFromSourceFile` call through to a general case of expression parsing (which can also parse types) rather than only parsing types directly.
* Right before doing back-end code generation, we check if the program we are going to emit has remaining (unspecialized) parameters, in which case we emit a diagnostic message for the parameters that haven't been specialized rather than go on to emit code that will fail to compile downstream.
* Within the `render-test` tool we collapse down the arrays that held both "generic" and "existential" specialization arguments, so that we just have *global* and *entry-point* specialization argument lists. This mirrors how Slang has worked internally for a while, but the difference hasn't been important to the test tool because no tests currently mix generic and existential specialization. The logic for parsing `TEST_INPUT` lines has been streamlined down to just the global and entry-point cases, but the pre-existing keywords are still allowed so that I don't have to tweak any test cases.
There are several significant caveats for this feature, which mean that it isn't really ready for users to hammer on just yet:
* There is no support for `Val`s of anything but integers, so there is no way to meaningfully have a generic value param with a type other than `int` or `uint`.
* We allow for a default-value expression on global generic parameters, but do not actually make use of that value for anything (e.g., to allow a programmer to omit specialization arguments), nor check that it meets the constraints of being compile-time constant.
* Global generic value parameters are *not* currently being treated the same as explicit generic parameters in terms of how they can be used for things like array sizes or other things that require constants. This will probably be relaxed at some point, but allowing a global generic to be used to size an array creates questions around layout.
* The IR optimization passes in Slang currently won't eliminate entire blocks of code based on constant values, so using a global generic value parameter to enable/disable features will *not* currently lead to us outputting drastically different HLSL or GLSL. That said, we expect most downstream compilers to be able to handle an `if(0)` well.
* Fix regression for tagged union types
The change that made specialization arguments be parsed as "terms" first, and then coerced to types meant that any special-case logic that is specific to the parsing of types would be bypassed and thus not apply.
Most of that special-case logic isn't wanted for specialization arguments, since it pertains to cases were we want to, e.g, declare a `struct` type while also declaring a variable of that type.
The one special case that *is* useful is the `__TaggedUnion(...)` syntax, which is the only way to introduce a tagged union type right now.
In order to get that case working again, all I had to do was register the existing logic for parsing `__TaggedUnion` as an expression keyword with the right callback, and the existing logic in expression parsing kicks in (that logic was already handling expression keywords like `this` and `true`).
I left in the existing logic for handling `__TaggedUnion` directly where types get parsed, rather than try to unify things.
A better long-term fix is to make the base case for type parsing route into `parseAtomicExpr` so that the two paths share the core logic.
That change should probably come as its own refactoring/cleanup, because it creates the potential for some subtle breakage.
* fixup: typo
|
|
* First pass of render-test refactor.
* Make window construction a function that can choose an implementation.
* Remove OpenGL as currently has windows dependency.
* Disable Vulkan as Renderer impl has dependency on windows.
* Pass Window in as parameter of 'update'.
* Add win-window.cpp as was missing.
* Fix warning on windows about signs during comparison.
* * Added mechanism to add random arrays as buffer inputs and select type
* Improved RenderGenerator to generate more types, and to be more careful around int32 ranges.
* Added support for security checks (for Visual Studio C++)
* Disable Execption handling being on by default when compiling kernels
* Added a 'Group' version of the entry point that will evaluate all threads in a group in a single call. In test code use this method if available.
* Added -compile-arg to be able to pass arguments to the compile within render-test
* Add documention for the _Group execution feature.
* Fix some typos in cpu-target.md
|
|
* First pass of render-test refactor.
* Make window construction a function that can choose an implementation.
* Remove OpenGL as currently has windows dependency.
* Disable Vulkan as Renderer impl has dependency on windows.
* Pass Window in as parameter of 'update'.
* Add win-window.cpp as was missing.
* Fix warning on windows about signs during comparison.
|
|
* Two fixes to avoid random crash on destruction of GLRenderer
* Use of a weak reference from objects created by GLRenderer, such that GLRenderer dtor can disable those objects assuming GLRenderer is live
* Make sure window is not destroyed before the renderer
* Used WeakSink for weak pointer.
|
|
* * Simplify some of test code around CPPCompiler
* Test using 'callable' with pass-through
* Small cpu doc improvements
* Improvements to Clang output parsing.
* Remove temporary file (base filename) .
* Improve handling of external errors - handle severity.
* On error dumping out to 'actual' file for runCPPCompilerCompile.
* Small fixes.
Set the source language type correctly for pass thru.
* Remove warning for test for clang backend c
* Preliminary work around making render-test compute potentiall work with CPU.
Made ShaderCompiler -> a stateless ShaderCompilerUtil.
Means we don't require a Renderer interface to do shader compilation.
* Refactor such that CPU test can take place in without Window or Renderer.
* Hack to look for prelude in source file directory.
Fix bug returning the SharedLibrary for HostCallable.
* Compute test running on CPU.
* Need the prelude currently in same directly as test.
* Hack to remove warning - that then produces an error on appveyor build.
Disable running render CPU test on non-windows.
* Improve handling of disabling CPU tests on linux.
* Added bit-cast.slang working on CPU.
|
|
* Prefixing source files in source/slang with slang-
* Prefix source in source/slang with slang- prefix.
* Rename core source files with slang- prefix.
* Update project files.
* Fix problems from automatic merge.
|
|
* * Fix warning in vk-swap-chain around use of Index. Rename _indexOf to _indexOfFormat.
* Rename IntSet to UIntSet and put in own files slang-uint-set.h.cpp. Use UInt as the held type.
* On UintSet setMax -> resizeAndClear. Doing so revealed bug in add.
* Closer following of conventions - use kPrefix for constants (even though held in 'enum')
* Small fixes/improvements
* * Add some documentation to UIntSet methods
* Use memset to set/reset bits
* Fix some tabbing.
Rename oldBufferSize -> oldCount
* Fix tabs.
|
|
* List made members m_
Tweaked types to closer match conventions.
* Use asserts for checking conditions on List.
Other small improvements.
* List<T>.Count() -> getSize()
* List<T>
Add -> add
First -> getFirst
Last -> getLast
RemoveLast -> removeLast
ReleaseBuffer -> detachBuffer
GetArrayView -> getArrayView
* List<T>::
AddRange -> addRange
Capacity -> getCapacity
Insert -> insert
InsertRange -> insertRange
AddRange -> addRange
RemoveRange -> removeRange
RemoveAt -> removeAt
Remove -> remove
Reverse -> reverse
FastRemove -> fastRemove
FastRemoveAt -> fastRemoveAt
Clear -> clear
* List<T>
FreeBuffer -> _deallocateBuffer
Free -> clearAndDeallocate
SwapWith -> swapWith
* List<T>
SetSize -> setSize
Reserve -> reserve
GrowToSize growToSize
* UnsafeShrinkToSize -> unsafeShrinkToSize
Compress -> compress
FindLast -> findLastIndex
FindLast -> findLastIndex
Simplify Contains
* List<T>
Removed m_allocator (wasn't used)
Swap -> swapElements
Sort -> sort
Contains -> contains
ForEach -> forEach
QuickSort -> quickSort
InsertionSort -> insertionSort
BinarySearch -> binarySearch
Max -> calcMax
Min -> calcMin
* Initializer::Initialize -> initialize
List<T>::
Allocate -> _allocate
Init -> _init
IndexOf -> indexOf
* * Put #include <assert.h> in common.h, and remove unneeded inclusions
* Small refactor of ArrayView - remove stride as not used
* getSize -> getCount
setSize -> setCount
unsafeShrinkToSize->unsafeShrinkToCount
growToSize -> growToCount
m_size -> m_count
* Some tidy up around Allocator.
* Use Index type on List.
* Refactor of IntSet.
First tentative look at using Index.
* Made Index an Int
Did preliminary fixes.
Made String use Index.
* Partial refactor of String.
* String::Buffer -> getBuffer
ToWString -> toWString
* Small improvements to String.
String::
Buffer() -> getBuffer()
Equals() -> equals
* Try to use Index where appropriate.
* Fix warnings on windows x86 builds.
|
|
* * Moved CPU determination macros to slang.h
* Determine SlangUInt/SlangInt from the pointer width (determined from CPU macros)
* Removed the UnambiguousInt and UnambigousUInt types - as a previous fragile work around
* Removed UInt/Int definition from smart-pointer.h as now in common.h
* * Remove ambiguity for PrettyWriter and ints
* Improve comment around SlangInt/UInt
* More fixes around ambiguity with PrettyWriter and integral types.
* Disable VK on OSX.
* Force CI to rebuild as spurious error.
|
|
* * Make adapter used selectable on the command line
* Added 'adapter' to Renderer::Desc with dx11, dx12, vk honoring it
* GL will check that the renderer matches, but cannot select a specific device
* Share functionality on dx adapter selection in D3DUtil
Note - that on tests that use OpenGL and the adapter doesn't match it will ignore the test (and display a message that the appropriate device couldn't be started)
* Small function name improvement.
* Variable rename to match type.
* Fix typo in Dx12 device selection.
* * Add checking if an adapter is warp
* Improve some comments
|
|
* Disable Dx12 half tests. The half-calc test runs, but is not actually doing any half maths. If the code is changed such that it is, the device fails when the shader is used. This can be seen by looking at dxil-asm.
* Fix using software driver for dx12 even when hardware is requested.
* * Refactor Dx12 _createAdapter such that it doesn't have side effects and stores desc information
* Disable half on dx12 software renderer because it crashes
* * Disable erroneous warnings from dx12
* Test for adapter creation
* Identify warp specifically
* Structured buffer test now works on dx12.
* Fix intemittent crash on dx12.
Due to if a resource was initialized with data, the actual resource constructed might be larger, for alignment issues. This led to a memcpy potentially copying from after the allocated source data and therefore a crash. Now only copies the non aligned amount of data.
* * Rename the test to use - style
* Disable TextureCube lookup in tests, as does not produce the correct result in dx12 (will fix in future PR)
* Updated hlsl.meta.slang.h that has rcp for glsl.
* * Fix bug where the SRV description was incorrect for cubemaps on dx12
* Re-enable cube map access in dx12 tests
* Slightly re-organize texture upload on dx12 to not repeatidly create and destroy upload resource for array/cube scenarios
|
|
* Disable Dx12 half tests. The half-calc test runs, but is not actually doing any half maths. If the code is changed such that it is, the device fails when the shader is used. This can be seen by looking at dxil-asm.
* Fix using software driver for dx12 even when hardware is requested.
* * Refactor Dx12 _createAdapter such that it doesn't have side effects and stores desc information
* Disable half on dx12 software renderer because it crashes
* * Disable erroneous warnings from dx12
* Test for adapter creation
* Identify warp specifically
* Structured buffer test now works on dx12.
* Fix intemittent crash on dx12.
Due to if a resource was initialized with data, the actual resource constructed might be larger, for alignment issues. This led to a memcpy potentially copying from after the allocated source data and therefore a crash. Now only copies the non aligned amount of data.
* * Rename the test to use - style
* Disable TextureCube lookup in tests, as does not produce the correct result in dx12 (will fix in future PR)
* Updated hlsl.meta.slang.h that has rcp for glsl.
|
|
Due to if a resource was initialized with data, the actual resource constructed might be larger, for alignment issues. This led to a memcpy potentially copying from after the allocated source data and therefore a crash. Now only copies the non aligned amount of data.
|
|
* Look at getting half to work on vk.
* Alter half test so can always produce consistent test results.
* First pass working half on vk.
* Improve comments for vulkan extensions around half.
* Upgraded vulkan headers to v1.1.103
https://github.com/KhronosGroup/Vulkan-Headers
* * Add getFeatures on Render interface
* Vulkan renderer determines at startup if it can support half
* Parse render-features on render-test
* Small changes to half-calc.slang test.
* Structured buffer half access works as expected for Vk, but isn't for dx12, so disable for now.
* Require the half feature for renderers for the half-structured-buffer.slang test.
* * Added ToolReturnCode to be more rigerous about how a return code is passed back from a tool
* Added support for a tool being able to pass back an 'ignored' result.
* Used enum codes to indicate meanings
* Made spawnAndWait return a ToolReturnCode
* Ignore tests that don't have required render-feature
* Fix macro line continuation usage.
* Check dx12 has half support.
* Checking for half on dx12 - if CheckFeatureSupport fails, don't fail renderer initialization.
* Fix typo.
|
|
https://github.com/KhronosGroup/Vulkan-Headers
|
|
* Fix up handling of NumElements for D3D buffer views
For everything but structured buffers we'd been setting this to the size in bytes, but that isn't really valid at all.
The `NumElements` member in the view descs is supposed to be the number of buffer elements, so it would be capped at the byte size divided by the element size.
This change fixes the computation of `NumElements` to take the size of the format into account for non-structured views.
For the "raw" case, we use a size of 4 bytes since that matches the `DXGI_FORMAT_R32_TYPELESS` format we use (which seems to be required for raw buffers).
I also added support for the raw case for SRVs where it didn't seem to be supported before (not that any of our tests cover it).
* Fix handling of size padding for D3D11 buffers
The existing code was enforcing a 256-byte-aligned size for all buffers, but this can cause problems for a structured buffer.
A structured buffer must have a size that is a multiple of the stride, so a structured buffer with a 48-byte stride and a 96-byte size would get rounded up to 256 bytes, which is not an integer multiple of 48.
This change makes it so that we only apply the padding to constant buffers.
According to MSDN, constant buffers only require padding to a 16-byte aligned size, and no other restrictions are listed for D3D11, but it is difficult to know whether those constrains are exhaustive.
I've left in the 256-byte padding for now (rather than switch to 16-byte), even though I suspect that was only needed as a band-aid for the `NumElements` issue fixed by another commit.
* Fix an IR generation bug when indxing into a strutured buffer element
The problem here arises when we have a structured buffer of matrices (an array type would likely trigger it too):
```hlsl
RWStructuredBuffer<float3x4> gMatrices;
```
and then we index into it directly, rather than copying to a temporary:
```hlsl
// CRASH:
float v = gMatrices[i][j][k];
// OKAY:
float3x4 m = gMatrices[i];
float v = m[j][k];
```
The underlying issue is that our IR lowering pass tries to defer the decision about whether to use a `get` vs. `set` vs. `ref` accessor for a subscript until as late as possible (this is to deal with the fact that sometimes D3D can provide a `ref` accessor where GLSL can only provide a `get` or `set`).
We probably need to overhaul that aspect of IR codegen sooner or later, but this change uses some of the existing machinery to try to force the `gMatrices[i]` subexpression to take the form of a pointer when doing sub-indexing like this. This fixes the present case, and hopefully shouldn't break anything else that used to work (because the subroutines I'm using to coerce the `gMatrices[i]` expression should be idempotent on the cases that were already implemented).
* Add a test case to confirm fxc/dxc layout for structured buffers of matrices
Even when row-major layout is requested globally, fxc and dxc seem to lay out a `StructuredBuffer<float3x4>` with column-major layout on the elements.
This commit adds a test that confirms that behavior.
This commit does not try to implement a fix for the issue (either fixing Slang's layout reflection information to be correct for what fxc/dxc do in practice, or fixing Slang's HLSL output to work around the fxc/dxc behavior), but just documents the status quo.
If/when we decide how we'd like to handle the issue long-term, this test can/should be updated to match the decision we make.
* fixup: build breakage on clang/gcc
This is one of those cases where the Microsoft compiler is letting through some stuff that isn't technically valid C++ ("delayed template parsing").
Fixed by just moving some declarations to earlier in the file.
|
|
Fixes issue #874
The problem here was that a pointer was being cast to `LONG` so that it could be used with the `SetWindowLongPtr` API, but that API actually expects a `LONG_PTR`. Since `LONG` is a 32-bit type on 64-bit Windows, the pointer we stored was getting truncated, leading to crashes.
I'm kind of surprised this wasn't biting more of our applications (e.g., the render tests).
|
|
* Fix typo on return type.
* * Inverted order of FlagCombiner (to make more 'nested for' like)
* On Dx12 just use D3D_FEATURE_LEVEL_11_0
* Fix typo on dll name
|
|
* Added CombinationUtil to produce combinations of flags
Used in Dx11 device creation making it fall back to release driver if debug driver is not found
* Made dx12 renderer startup similar to dx11 - testing multiple configs.
* Small improvements in naming.
* * Moved functionality to gfx from core
* Use FlagCombiner to simplify construction, and can be iterated over, without need for array
* Share DeviceCheckFlags
* Improve comments.
* Re-add the comment about combinations tested to set up dx11 device.
* More comment improvements.
|
|
* First steps toward supporting interface-type parameters on shaders
What's New
----------
From the perspective of a user, the main thing this change adds is the ability to declare top-level shader parameters (either at global scope, or in an entry-point parameter list) with interface types. For example, the following becomes possible:
```hlsl
// Define an interface to modify values
interface IModifier { float4 modify(float4 val); }
// Define some concrete implementations
struct Doubler : IModifier
{
float4 modify(float4 val) { return val + val; }
}
struct Squarer : IModifier { ... }
// Define a global shader parameter of interface type
IModifier gGlobalModifier;
// Define an entry point with an interface-type `uniform` parameter
void myShader(
unifrom IModifier entryPointModifier,
float4 inColor : COLOR,
out float4 outColor : SV_Target)
{
// Use the interface-type parameters to compute things
float4 color = inColor;
color = gGlobalModifier.modify(color);
color = entryPointModifier.modify(color);
outColor = color;
}
```
The user can specialize that shader by specifying the concrete types to use for global and entry-point parameters of interface types (e.g., plugging in `Doubler` for `gGlobalModifier` and `Squarer` for `entryPointModifier`).
The "plugging in" process is done in terms of a concept of both global and local "existential slots" which are a new `LayoutResourceKind` that represents the holes where concrete types need to be plugged in for existential/interface types.
In simple cases like the above, each interface-type parameter will yield a single existential slot in either the global or entry-point parameter layout. Users can query the start slot and number of slots for each shader parameter, just like they would for any other resource that a parameter can consume. Before generating specialized code, the user plugs in the name of the concrete type they would like to use for each slot using `spSetTypeNameForGlobalExistentialSlot` and/or `spSetTypeNameForEntryPointExistentialSlot`.
There are some major limitations to the implementation in this first change:
* Parameters must be of interface type (e.g., `IFoo`) and not an array (`IFoo[3]`), or buffer (`ConstantBuffer<IFoo>`) over an interface type. Similarly, `struct` types with interface-type fields still don't work.
* The work on interface-type function parameters still doesn't include support for `out` or `inout` parameters, nor for functions that return interface types (that isn't technically related to this change, but affects its usefullness).
* No work is being done to correctly lay out shader parameters once the concrete types for existential slots are known, so that this change really only works when the concrete type that gets plugged in is empty.
These limitations are severe enough that this feature isn't really usable as implemented in this change, and this merely represents a stepping stone toward a more complete implementation.
Implementation
--------------
The API side of thing largely mirrors what was already done to support passing strings for the type names to use for global/entry-point generic arguments, so there should be no major surprises there.
The logic in `check.cpp` computes the list of existential slots when creating unspecialized `Program`s and `EntryPoint`s (this is logically the "front end" of the compiler), and then checks the supplied argument types against what is expected in each slot when creating specialized `Program`s and `EntryPoint`s. This again mirrors how generic arguments are handled.
Type layout was extended to compute the number of existential slots that a type consumes, and will thus automatically assign ranges of slots to top-level and entry-point shader parameters in the same way it already allocates `register`s and `binding`s. The big missing feature is the ability to specialize a layout to account for the concrete types plugged into the existential-type slots.
IR generation for specialized programs and entry points was slightly extended so that it attaches information about the concrete types plugged into the existential slots, and the witness tables that show how they conform to the interface for that slot. The linking step needed some small tweaks to make sure that information gets copied over to the target-specific program when we start code generation.
The meat of the IR-level work is in `ir-bind-existentials.cpp`, which takes the information that was placed in the IR module by the generation/linking steps and uses it to rewrite shader parameters. For example, if there is a shader parameter `p` of type `IModifier`, and the corresponding existential slot has the type `Doubler` in it, we will rewrite the parameter to have type `Doubler`, and rewrite any uses of `p` to instead use `makeExistential(p, /*witness that Doubler conforms to IModifier*/)`.
Once the replacement is done on the parameters, the existing work for specializing existential-based code when the input type(s) are known kicks in and does the rest.
Testing
-------
A single compute test is added to validate that this feature works. It is narrowly tailored to not require any of the features not supported by the initial implementation (e.g., all of the concrete types used have no members).
The test case *does* include use of an associated type through one of these existential-type parameters, which has exposed a subtle bug in how "opening" of existential values is implemented in the front-end. Rather than fix the underlying problem, I cleaned up the code in the front-end to special case when the existential value being opened is a variable bound with `let`, to directly use a reference to that variable rather than introduce a temporary. Similarly, in the IR generation step, I added an optimization to make variables declared with `let` skip introducing an IR-level variable and just use the SSA value of their initializer directly instead.
* fixup: missing files
* fixup: incorrect type for unreachable return
* fixup: actually comment ir-bind-existentials.cpp
|
|
* Split front- and back-ends
This change is a major refactor of several of the types that provide the behind-the-scenes implementation of the public C API.
The goal of this refactor is primarily to allow for future API services that let the user operate both the front- and back-ends of the compiler in a more complex fashion.
For example, as user should be able to compile a bunch of source code into modules, look up types, functions, etc. in those modules, specialize generic types/functions to the types they've looked up, and then finally request target code to be gernerated for specialized entry points.
The back-end code generation they trigger should re-use the front-end compilation work (parsing, semantic checking, IR generation) that was already performed.
The most visible change is that `CompileRequest` has been split up into several smaller types that take responsibility for parts of what it did:
* The `Linkage` type owns the storage for `import`ed modules, and well as the `TargetRequest`s that represent code-generation targets. The intention is that an application could use a single `Linkage` for the duration of its runtime (so long as it was okay with the memory usage), so that each `import`ed module only gets loaded once. For now, this type needs to manage the search paths, file system, and source manager, because of its responsibility for loading files.
* A `FrontEndCompileRequest` owns the stuff related to parsing, semantic checking, and initial IR generation. This most notably includes the `TranslationUnitRequest`s and the `FrontEndEntryPointRequest`s (which used to be just `EntryPointRequest`s). It's main job is to produce AST and IR modules for each translation unit, and to find and validate the entry points. The front-end request does *not* interact with generic arguments for global or entry-point generic parameters.
* The main output of both `import` operations and front-end translation units is the `Module` type, which is just a simple container for both the AST module (to service the reflection/layout APIs, and also for semantic checking of code that `import`s the module) and the IR module (for linking and code generation). This type captures the commonalities between the old `LoadedModule` (which is now just an alias for `Module`) and `TranslationUnitRequest` (which now owns a `Module`).
* The secondary output of front-end compilation is a `Program`, which comprises a list of referenced `Module`s and validated `EntryPoint`s that will be used together. Layout and code generation both need a `Program` to tell them what modules and entry points will be used together (we don't want to just code-gen everythin that has ever been loaded into the linakge). The `Program`s created by the front-end do not include generic arguments, so they may provide incomplete layout information and/or be unsuitable for code generation.
* A `BackEndCompileRequest` owns stuff related to turning a `Program` into output kernels for the targets of a `Linkage`. Most of the data it owns beyond the `Program` to be compiled is minor, so this is a good candidate for demotion from a heap-allocated object to just a `struct` of options that gets passed around.
* The `CompileRequestBase` type is an attempt to wrap up the common functionality of both front-end and back-end compile requests. Most of it is just exposing the availability of a linkage and `DiagnosticSink`, so this type is a good candidate for subsequent removal. The main interesting thing it has is the flags related to dumping and validation of IR, so there is probably a good refactoring still to be made around deciding how options should be handled going forward.
* Behind the scenes, the `Program` type is set up to handle some level of on-line compilation and layout work. The `Program` knows the `Linkage` it belongs to, and allows for a `TargetProgram` to be looked up based on a specific `TargetRequest`. A `TargetProgram` then allows layout information and compiled kernel code to be asked for on-demand, in order to support eventual "live" compilation scenarios.
* The `EndToEndCompileRequest` type is a composition/coordination type that replaces the old `CompileRequest` in a way that uses the services of the various other types. It owns a few pieces of state that only make sense in the context of an end-to-end compile (e.g., there is really no way to "pass through" code when the front- and back-ends are run separately) or a command-line compile (everything to do with specifying output paths for files is really just for the benefit of `slangc`, and might even be moved there over time).
* One important detail is that the `EndToEndCompilRequest` owns all of the string-based generic arguments for both global and entry-point generic parameters. The logic in `check.cpp` for dealing with those arguments has been heavily refactored to separate out the parsings steps that are specific to end-to-end compilation with string-based type arguments, and the semantic checking steps that result in a specialized `Program` (which can be exposed through new APIs that aren't tied to end-to-end compilation).
It is perhaps not surprising that this change had a lot of consequences, so I'll briefly run over some of the main categories of changes required:
* I changed the way that global generic arguments are passed via API (use `spSetGlobalGenericArgs` instead of the generic arguments for `spAddEntryPointEx`, which are not just for entry-point generics), which has been a change that we've needed for a long time. This is technically a breaking API change, although we should have very few client applications that care about it.
* A bunch of places that used to take "big" objects like `CompileRequest` now just take the sub-pieces they care about (e.g., a function might have only needed a `Linkage` and a `DiagnosticSink`). This makes many subroutines or "context" struct types more generally useful, at the cost of taking more parameters.
* In a few cases the conceptually clean separation of the layers breaks down (often for edge-case or compatibility features), and so we may pass along additional objects that are allowed to be null, but are used when present. A big example of this is how the back-end code generation routines accept an `EndToEndCompileRequest` that is optional, and only used to check whether "pass through" compilation is needed. We should probably look into cleaning this kind of logic up over time so that we don't need to violate the apparent separation of phases of compilation.
* In cases where separation of layers was being broken for the sake of GLSL features, I went ahead and ripped them out, since all of that should be dead code anyway.
* In many cases I increased the encapsulation of data in the core types to help track down use sites and make sure they are following invariants better.
* In cases where code was doing, e.g., `context->shared->compileRequest->session->getThing()` I have tried to introduce convenience routines so that the usage site is just `context->getThing()` to improve encapsulation and allow changes to be made more easily going forward.
* The `noteInternalErrorLoc` functionality was moved off of the compile request and into `DiagnosticSink`, since that is the one type you can rely on having around when you want to note an internal error. We may consider going forward if (and how) it should reset the counter used for noting locations on internal errors.
* A few APIs now take `DiagnosticSink*` arguments where they didn't before, and as a result some public APIs need to create `DiagnosticSink`s to pass in, before going ahead and ignoring the messages. In the future there should be variations of these APIs that accept an `ISlangBlob**` parameter for the output.
* fixup: missing include for compilers with accurate template checking (non-VS)
* fixup: review feedback
|
|
* Re-enable warnings around null this.
* Remove testing for nullptr in Substitution::Equals tests
* Fix ref counting problem in vulkan render.
* * Remove SLANG_ASSERT(this) in mthods
* Place asserts conservatively at method call sites where appropriate.
|
|
* Remove circular reference to renderer on Vk & D3D12 DescriptorSetImpl
* Refactor Stbi image loading such that memory is correctly freed when goes out of scope.
Added Crt memory dump at termination.
Reduced erroneous reporting by scoping TestContext.
* Used capitalized acronym for STBImage to keep Tim happy.
|
|
* Try using hardware device before reference on dx11
* Output error string on renderer construction failure
|
|
|
|
* Premake work in progress for linux.
* Added dump function.
* Remove examples on linux
Small warning fix.
* * Don't build render-test on linux
* Removed work around virtual destructor warning, and just used virtual dtor for simplicity
* Git ignore obj directories
* Fix premake working on windows.
* * Fix sprintf_s functions
* Make generates arg parsing more robust
* Added FloatIntUnion to avoid type punning/strong aliasing issues, and repeated union definitions.
* Work around problems building on linux with getClass claiming a strict aliasing issue.
* Fix for targetBlock appearing potentiall used unintialized to gcc.
* Linux slang link options -fPIC to make dll.
* Add -fPIC to build options on linux.
* Add -ldl for linux on slang.
* Fixes to try and get premake working with .so on linux.
* Make core compile with -fPIC
* Try to fix linux linking with --no-as-needed before -ldl
* Add rpath back.
* Remove render-gl from linux build.
* Re-add location for linux.
* Don't include <malloc.h> except on windows.
* Remove unused line to fix warning on osx.
* Remove ambiguity on OSX for operator <<.
* Fixing ambiguity with operator overloading and Int types for OSX.
* Fix ambiguity around UInt and operator
* Fix ambiguity of UInt conversion for OSX.
* Added UnambiguousInt and UnambiguousUInt to make it easier to work around OSX integer coercion for UInt/Int types.
|
|
* * Remove dispose from IRInst
* Use MemoryArena instead of MemoryPool
* Make all IRInst not require Dtor - by having ref counted array store ptrs that need freeing
* Increase block size - typically compilation is 2Mb of IR space(!)
* Fix issues around StringRepresentation::equal because null has special meaning.
* Don't bother to construct as String to compare StringRepresentation, just used UnownedStringSlice.
* Added fromLiteral support to UnownedStringSlice and use instead of strlen version.
* Use more conventional way to test StringRepresentation against a String.
* Fix gcc/clang template problem with cast.
* Fix warnings.
|
|
* Improve model-viewer support for lights
The main visible change here is that the model-viewer example supports
multiple light sources, with a basic UI for adding new light sources to
the scene, and for manipulating the ones that are there.
Along the way I also refactored the `IMaterial` decomposition to be a
bit less naive, while still only including a completely naive
Blinn-Phong implementation.
I also went ahead and spruced up the `cube.obj` file so that it has
multiple materials, although it is still a completely uninteresting
asset.
* Fixup: Windows SDK version
|
|
This isn't being made visible just yet, but it will allow us to have a simple UI for loading models into the model-viewer example.
In order to support rendering with IMGUI I had to add the following to the `Renderer` layer:
* viewports
* scissor rects
* blend support
These are really only fully implemented for D3D11, but adding them to the other back-ends should be a reasonably small task.
|
|
The original goal here was to bring up a second example program: `model-viewer`.
While the existing `hello-world` example is enough to get somebody up to speed with the basics of the Slang API (as a drop-in replacement for `D3DCompile` or similar), it doesn't really show any of the big-picture stuff that Slang is meant to enable.
There wasn't any use of D3D12/Vulkan descriptor tables/sets, and there wasn't any use of interfaces, generics, or `ParameterBlock`s in the shader code.
The `model-viewer` example addresses these issues. Its shader code involves generics, interfaces, and multiple `ParameterBlock`s, and the host-side code demonstrates a few key things for working with Slang:
* There is an application-level abstraction for parameter blocks, that combines the graphics-API descriptor set object with Slang type information
* There is a shader cache layer used to look up an appropriate variant of a rendering effect by using parameter block types to "plug in" global type variables
* There is a clear separation between the phases of compilation: a first phase that does semantic checking and enables reflection-based allocation of graphics API objects, followed by one or more code generation passes for specialized kernels.
This example is certainly not perfect, and it will need to be revamped more going forward. In particular:
* The output picture is ugly as sin. We need a plan for how to get this to load better content, perhaps even popping up an error message to note that the required input data isn't present in the basic repository.
* The shader code is too simplistic. There isn't any real material variety, and the `IMaterial` abstraction is completely wrong.
* The use of parameter blocks is facile because there are no resource parameters right now. Fixing that will likely expose issues around interfacing with Slang's reflection API.
* The whole example exposes the issue that Slang's current APIs aren't really designed for the benefit of two-phase compilation (since our many client application has been stuck on one-phase compilation).
* Global type parameters are actually a Bad Idea that we only did for compatibility with existing codebases. We should not be showing them off in an example of the Right Way to use Slang, but the language support for type parameters on entry points is still not complete.
Of course, the majority of the changes here are *not* inside the example applications, and instead involve a major overhaul of the `Renderer` abstraction that is used for both tests and examples. The main thrust of the change is to make the abstraction layer be closer to the D3D12/Vulkan model than to a D3D11-style model. This is important for the `model-viewer` example, since it aspires to show how Slang can be incorporated into a renderer that targets a modern API. The most important bit is actually the use of descriptor sets and "pipeline layouts" a la Vulkan, since without these Slang's `ParameterBlock` abstraction won't make a lot of sense.
Implementation of the abstraction for the various APIs has very much been on an as-needed basis. The current implementation is just enough for the two examples to work, plus enough to get all the tests to pass in both debug and release builds on Windows.
A big missing feature in the API abstraction right now is memory lifetime management. The code had been trending toward something D3D11-like where a constant buffer could be mapped per-frame with the implementation doing behind-the-scenes allocation for targets like D3D12/Vulkan. I'd like to shift more toward a model of just exposing "transient" allocations that are only valid for one frame, because these are more representation of how an efficient renderer for next-generation APIs will work. That transition isn't actually complete, though, so there are problems with the existing examples where `hello-world` is actually scribbling into memory that the GPU might still be using, while `model-viewer` is doing full-on heavy-weight allocations on a per-frame basis with no real concern for the performance implications.
All together, there are a lot of things here that need more work, but this branch has been way too long-lived already, and so I'd like to get this checked in as long as all the tests pass.
|
