| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
vertex/fragment shader pair, but instead of comparing the resulting framebuffer, it expects the test shader to write results into a UAV, and compares the pixel shader UAV output to the reference output.
|
|
|
|
|
|
|
|
inputs for running test shaders with arbitrary parameter definitions.
This commit contains the parser of the resource input definition.
|
|
* Fix up emission of shader parameter semantics when using IR
- Make sure to propagate entry point parameter layouts down to IR parameters when doing the initial cloning to form target-specific IR
- When layout information is present on an IR node, prefer to use that over the original high-level declaration for outputting semantics in final HLSL
- Fix up test runner to generate `.actual` files when running compute tests, in cases where the `render-test` application errors out (e.g., because of a Slang compilation error)
- Add a first test of generics functionality, to show that they generate valid code through the IR
- Right now this test is *not* using any "interesting" operations on the type parameter, so this is not a test that can confirm that interface constraints work
* fixup: skip compute tests when running on Linux
|
|
testing framework.
|
|
There are two big changes here:
- Add logic during the initial IR cloning pass for an entry point + target that tries to pick the best possible version of any target-overloaded function. This allows us to pick the intrinsic version of `saturate()` when compiling for HLSL output, but then pick the non-intrinsic version (that is implemented in terms of `clamp()`) when targetting GLSL.
- Add an initial specialization pass that tries to deal with generics. This required some fixing work to IR generation, so that we correctly generate explicit operations to specialize a generic for specific types (this is currently implemented as a `specialize` instruction that takes the generic to specialize plus a declaration-reference that represents the specialized form). With that work in place, we can scan for `specialize` instructions inside of non-generic functions, and use them to trigger generation of specialized code. We rely on the name-mangling scheme to help us find pre-existing specializations when possible.
There are also a bunch of cleanups encountered along the way:
- Don't use the explicit `layout(offset=...)` for uniforms, because it isn't supported by all current drivers. For now we will just assume that our layout rules compute the same values that the driver would for un-marked-up code. We can come back later and try to implement a workaround in the cases where this doesn't apply (e.g., by re-running the layout logic as part of emission, and dropping layout modifiers from variables that don't need explicit layout).
- Fix some issues in IR dump printing so that we print function declarations more nicely.
- Testing: print out failing pixel when image-diff fails
|
|
Move reflection JSON generation into separate test fixture
|
|
When outputting GLSL from a Slang or HLSL entry point, we need to translate any parameters or results of an entry-point function into global declarations of `in` or `out` parameters, as needed by GLSL.
This change adds these transformations at the IR level, so that they don't need to complicate the emit logic.
More detailed changes:
- Make `render0` test use IR. It passes out of the box.
- Fix test runner to not always dump diffs on failures
I accidentally initialized an option to `true` instead of `false` when working on debugging the Travis CI failures.
- Special-case output for component-wise multiplication to handle GLSL `matrixCompMul()`
- Handle GLSL vs. HLSL output for calls to `mul()`
- Output proper `layout(std140)` on GLSL constant buffer declarations
- Require appropriate GLSL extension when emitting explicit `layout(offset = ...)` on constant buffer members
- TODO: Need to avoid requiring this extension in cases where the offsets are what would be computed anyway.
Realistically, should probably be emitting code with explicit padding, etc. to guarantee layouts.
- Add an IR-based pass to translate entry point functions by eliminating their input/output parameters and replacing them with global variables.
- Demangle names when calling target intrinsics
The lowering to the IR will turn a call like `sin(foo)` into a call to a function declaration with a mangled name like `_S3sin...`. This works fine when the user is calling their own functions, since the name mangling will apply to both the definition and use sites, but for builtin functions it obviously isn't what we want.
This change makes it so that we demangle the name of an instrinsic function just enough so that we can extract the original simple name, and make a call using that.
These changes do nor provide 100% of what we need when translating to GLSL, so the `cross-compile-entry-point` test *still* hasn't been flipped over to use the IR (even though that is the test case I've been using to develop these changes).
|
|
* Attempt to fix subprocess handling for Linux
Our CI builds are sometimes hanging on Travis, and I suspect it might be something to do with how we are waiting for subprocesses to complete. I'm trying to following the manpage for the `wait()` and `waitpid()` calls a bit better here.
* fixup: try to use poll() instead of select()
* fixup: missing header
* fixup
* fixup
* fixup: try to emit test output when tests fail on Travis
|
|
* Get tests running/passing under Linux
- Fix up `dlopen` abstraction
- Fix up some test cases to request hlsl (rather than default to dxbc) so they can run on non-Windows targets
- Fix up test runner ignore tests that can't run on current platform (and not count those as failure)
- Fix file handle leeak in process spawner absttraction
- Get additional test-related applications building
- More tweaks to Travis script; in theory deployment is set up now (yeah, right)
* fixup
* fixup: Travis environment variable syntax
* fixup: Buffer->begin
* fixup: actually run full tests on one config
* fixup: add build status badge for Travis
|
|
At a high level, this commit adds two things:
1. A "bytecode" format for serializing Slang IR instructions and related structure (functions, "registers")
2. A virtual machine that can load and then execute code in that bytecode format.
The reason for kicking off this work right now is that we *need* a way to run tests on Slang code generation that doesn't rely on having a GPU present (given that our CI runs on VM instances without GPUs), nor on textual comparison to the output of other compilers. With these features I've implemented a slapdash `slang-eval-test` test fixture that can run a (trivial) compute shader to very our compilation flow through to bytecode.
Some key design constraints/challenges:
- The bytecode format should be "position independent" so that a user can just load a blob of data and then inspect it without having to deserialize into another format, allocate memory, etc. Eventually the bytecode format might be a replacement for out current reflection API (we used to base reflection off a similar format, but the cost/benefit wasn't there at the time and we switched to just using the AST).
- The VM should be able to execute bytecode functions without doing any per-operation translation, JIT, etc. (translation of more coarse-grained symbols is okay). For now the VM is just being used to run tests, but eventually I'd like it to be viable for:
- Running Slang-based code in the context of the compiler itself. This starts with stuff like constant-folding in the front-end, but could expand to more general metaprogramming features.
- Running Slang-based ocde within a runtime application (e.g., a game engine) that wants to be able to run things like "parameter shader" code, or even just evaluate compute-like code on CPU (e.g., when supporting particles on both CPU and GPU).
- Finally, the bytecode format should ideally be able to round-trip back to the IR without unacceptable loss of information. This requirement and the previous one play off of each other, because things like a traditional SSA phi operation is ugly when you have to actually *execute* it. This doesn't matter right now when we don't have SSA yet, but it might be part of the decision-making here.
The actual implementation is centralized in `bytecode.{h,cpp}` and `vm.{h.cpp}`.
Big picture notes:
- The space of opcodes is shared between IR and bytecode (BC), with the hope that this makes translation of operations between the two easy.
- The actual bytecode instruction stream relies on a variable-length encoding for integer values, including opcodes and operand numbers, so that the common case is single-byte encoding.
- In the long term I intend to have a rule that if you use a single-byte encoding for an opcode, then all operands are required to use single-byte encodings too. Operations that need multi-byte operands would then be forced to use a multi-byte encoding of the op, and would be sent down a slower path in the interpeter.
- The "bytecode"'s outer structure is based on ordinary data structures linked with pointers, but they are "relative pointers" so the actual structure is position-independent.
- There are two main kinds of operands: registers and "constants." An operand is a signed integer where non-negatie values indicate registers (with `index == operandVal`) and negative values indicate constants (with `index == ~operandVal`).
- Registers are stored in the "stack frame" for a VM function call, and each has a fixed offset based on the size of the type and those that come before it. Conceptually, registers are allowed to overlap if they aren't live at the same time, and we manage this with a simple stack model: every register is supposed to identify the register that comes directly before it (this isn't implemented yet).
- "Constants" are more realistically a representation of "captured" values, but they are currently also how constants come in. Basically we can use a compact range of indices in the bytecode for a function, and each of these indices indirectly refers to some value in the next outer scope.
- The actual encoding of bytecode instructions right now is largely ad-hoc and very wasteful (we encode the type on everything, and we also encode everything as if it had varargs).
- In some cases, an instruction needs to know the types of the values involved (e.g., because it needs to load an array element, which means copying a number of bytes based on the size). The way the VM works we have types attached to our registers, so we currently get sneaky and look at those types in some ops. Longer term is makes sense to encode the required type info directly in the BC.
- There's a whole lot of hand-waving going on with how the actual top-level bytecode module gets loaded, because of the way we currently treat the top-level module as an instruction stream in the IR. This means that we try to represent the loaded module as a "stack frame" for a call to the module as a function, but that approach as serious problems, and isn't realistically what we want to do.
|
|
|
|
Fixes #104
- Map HLSL `nointerpolation` to GLSL `flat`
- When lowering a `struct` type varying input/output, look for interpolation modifiers along the "chain" from the leaf field up to the original shader input variable (and take the first one found)
- Not sure if this is strictly needed, but it seems like a reasonable policy
- Add `flat` to varying input of integer type, with no other interpolation modifier
- Note: I do *not* do anything to ignore a manually imposed interpolation modifier that might be incorrect
|
|
- `RefPtr` no longer tries to have distinct cases for interal-vs-external reference counts. Instead we always require an internal reference count.
- Types the used `RefPtr` but weren't `RefObject` were made to inherit `RefObject`
- The `ReferenceCounted` base class was removed, so that only `RefObject` remains
- Implicit conversion from `RefPtr<T>` to `T*` added
- This created some complicates for other types that relied on implicit conversions, so this isn't a net cleanup right now
- The main type that got messed up by the above was `String`, which previously held a `RefPtr<char, ...>`. This change thus *also* includes a major overhaul of `String`:
- `String` now holds all its data via indirection, using a `StringRepresentation` that is a `RefObject`. This object holds a length, capacity, and directly stores the character data in its allocation. This means that `sizeof(String)==sizeof(void*)`
- It is now possible to directly mutate a `String` by appending to its representation (we just need to ensure it has a reference count of `1`, possibly by cloning it). This means that `StringBuilder` is now basically just an idomatic use of `String`
- A couple operations that just return sub-ranges of a `String` now return `StringSlice` to avoid allocation when it isn't needed. This required more work.
- Indices into strings changed from `int` to `UInt` (which is pointer-sized). This had a bunch of follow-on changes because the value `-1` sometimes needs to be special-cased in code that uses indices. Further cleanups are probably needed here.
|
|
|
|
|
|
Getting rid of more namespace complexity and stripping things down to the basics.
This also gets rid of some dead code in the "core" library.
|
|
This includes a bunch of related changes:
- `slang-test`
- Add a notion of an "output mode" that specifies whether we output to console (the default), or invoke the apprpriate AppVeyor command to update test status
- Add a notion of test categories, so that tests can be tagged with categories, and then we can invoke only those tets in a given category, or choose to *exclude* tests with specific categories
- Allow the `OSProcessSpawner` to look up an executable by "path" (meaning a full path is expected) or by "name" (meaning it should be allowed to look in the current directory, `PATH` environment variable, etc.). This was important to make sure that I can run `appveyor` without having to know its absolute path.
- AppVeyor configuration
- Change badge to reflect new build account for organization (rather than a single-user account)
- Remove attempt to set AppVeyor build version in a clever way, since it breaks links from GitHub to AppVeyor
- Change order or configurations in the build matrix to front-load the Release build (which has the main tests)
- Turn on `fast_finish` flag so we don't have to wait as long for failed builds
- Turn on `parallel` builds
- Set `verbosity: minimal` to avoid getting build spew about Xamarin stuff I'm not using
- Add custom `test_script` to invoke `test.bat`
- Sets the test category based on teh build configuration, so we don't run the full test suite on every input.
- `test.bat`
- Allow for `-platform` and `-configuration` arguments
- Rewrute a platform of `Win32` over to `x86` to match how the output directories are named
- Futz around with how the directories are being passed along to work around annoying `.bat` file quoting behavior (I still don't get how batch files work)
- Tests
- Mark a bunch of tests as `smoke` tests
- Mark the relevant tests as `render` tests
(these get filtered out for AppVeyor builds)
|
|
Now if there are multiple tests listed for an input file like `foo.slang`, they will be output as:
passed test: 'foo.slang'
passed test: 'foo.slang.1'
passed test: 'foo.slang.2'
This isn't a perfect solution, but it should work well enough for now.
This change doesn't make any tests actually use the new capability.
|
|
This is a large change that contains many pieces:
- Update the `cross-compile0` test to actually make use of cross compilation.
Now the `cross-compile0.hlsl` file contains both HLSL and GLSL source code, and then imports code from `cross-compile0.slang`, which provides a "library" (one function) that can be shared between both the HLSL and GLSL version of things.
- Fixed a bug in the support for backslash-escaped newlines.
- Added a new `__import` declaration type (replaces the `using` directive that was still around in a vestigial form)
An `__import` causes the compiler to look for a Slang source file (currently using the ordinary `#include` lookup logic), and then parse/check the found file as an additional module ("translation unit"), before making its declarations visible in the current scope.
- Refactored the main compilation flow to be simpler. There were the `ShaderCompiler` and `ShaderCompilerImpl` classes that weren't relaly doing anything, but added complexity to the whole workflow.
- The `render-test` application has been heavily modified to better support testing cross-compilation workflows. At the most basic level we are starting to distinguish pass-through vs. rewriter workflows, and are passing various `#define`s down to the compiler(s) to let the source code be customized as needed for each case.
Several annoying corner cases are caused here by having to support the GLSL compilation model, which really wants each entry point in its own specific translation unit, whereas we really want to keep things nicely contained in single files.
- Added support for `__intrinsic` operations to have target-specific behavior.
This allows a function to be given a different name for some specific target (so a call gets emitted as a call to that other operation).
More generally, the library writer can put together an arbitrary format string that will be used in place of expressions that call the given function, e.g.:
__intrinsic(hlsl, "$1 - $0") __intrinsic int foo(int a, int b);
Given this declaration, a call like `foo(x,y)` will code generate as `x - y` for HLSL, and as `foo(x,y)` for all other targets.
Annoying things still to be dealt with:
- The way that I'm filtering the user-provided options when passing things down to the compilation of dynamically loaded modules is a bit ad hoc. It would be good to have a systematic notion of which options will be inherited and which won't. There is also more code duplication than I'd like, so we risk having the compiler behave differently when compiling a file at the top level, vs. because of `__import`.
- Adding target-specific behavior to intrinsics is all well and good, but the current approach means we can only add this to the original declaration, which limits the ability to easily extend the set of targets.
A better approach long-term would be to add a more robust notion of target-based overload resolution (which would happen after semantic checking). Then one mechanism would be used to find the right target-specific overload to use for an operation, and then each (target-specific) definition could use a simpler attribute to intercept code-generation behavior.
Note that we might eventually need a similar notion to deal with stage- or profile-specific functions and the overloading behavior around them, so using this for intrinsics doesn't seem like a bad idea.
|
|
The test case that is there right now is nominally a cross-compilation test, but for right now it uses the preprocessor to present completely different code for HLSL and GLSL compilation.
This change is really just fleshing out the OpenGL side of `render-test` enough that it can produce images using OpenGL to enable further testing.
|
|
|
