summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2017-07-17Handle `flat` interpolation cases in cross compilationTim Foley
Fixes #104 - Map HLSL `nointerpolation` to GLSL `flat` - When lowering a `struct` type varying input/output, look for interpolation modifiers along the "chain" from the leaf field up to the original shader input variable (and take the first one found) - Not sure if this is strictly needed, but it seems like a reasonable policy - Add `flat` to varying input of integer type, with no other interpolation modifier - Note: I do *not* do anything to ignore a manually imposed interpolation modifier that might be incorrect
2017-07-14Don't use "auto locations" mode in glslangTim Foley
Fixes #88 The code was using `glslang::TShader::setAutoMapLocations` as a workaround for an old glslang issue, but apparently this mode has a bug where it ends up applying a `location` layout to the implicitly-created `gl_PerVertex` definition (which shouldn't be allowed). This change drops the call to `setAutoMapLocations` (and `setAutoMapBindings`) since it should no longer be required.
2017-07-13Add support for dumping intermediates for debugging.Tim Foley
Calling: spSetDumpIntermedites(compileRequest, true); will set up a mode where Slang tries to dump every intermediate HLSL, GLSL, DXBC, SPIR-V, etc. file it generates. If SPIR-V or DXBC is requested then we also dump assembly of those. Right now the files are all named as `slang-<counter>.<ext>`, and get dropped in whatever the working directory is, but I'm open to ideas on how to improve that. Note: this change introduces a new binary interface to `glslang`, so pulling it requires an updated `glslang.dll`.
2017-07-10Removed spGetTranslationUnitCode; Unified ↵Kai-Hwa Yao
EntryPointResult/TranslationUnitResult, added helper functionality; Ensure null termination when printing raw data
2017-07-10Don't assume vector contains contentsKai-Hwa Yao
2017-07-10Allow glslang wrapper to output regular SPIRV before disassemblyKai-Hwa Yao
2017-07-06Fix many warnings-as-errors issues.Tim Foley
The code should now compile cleanly with warnings as errors for VS2015 with `W3`. Most of the changes had to do with propagating a real pointer-sized integer type through code that had been using `int`.
2017-07-06Start to support cross-compilation via "lowering" passTim Foley
- The big change here is the introduction of a "lowering" pass that takes an input AST from the semantic checker, and produces an output AST suitable for emitting. The intention is that he lowering pass is responsible for: - Stripping out unused code (when we have enough information to do so), by only outputting declarations that are transitively references from an entry point - When cross-compiling to GLSL, generating a suitable `void main()` entry point to wrap the user-written entry-point function - (Eventually) legalizing types in the program, by scalarizing aggregate types that mix uniform and resource types - (Eventually) instantiating generic declarations so that the resulting code only deals with fully specialized declarations - (Eventually) de-sugaring OOP constructs into basic "structs and functions" form - (Eventually) instantiating code that depends on interface types at the concrete types chosen - It is clear that there is still a lot of work to be done there, to this change is really about getting infrastructure in place without breaking the existing test cases. - One cleanup here is that we get rid of the idea of whole-translation-unit output, since that was specific to HLSL output, and there is really no strong reason for keeping it. Users should now just ask for the output for each entry point that they wanted to generate. - The biggest source of complexity for the lowering process is that it needs to produce the same AST structure as the input, to deal with the complexity of the rewriter case. That is, we need the output to be able to reproduce the input exactly in the case where we are rewriting and nothing needs to change, so the output format needs at least the degrees of freedom of the input. - As a result, we end up having to distinguish "rewriter" and "full" modes in both lowering and code-emit steps, so that we can react appropriately. - Generating a GLSL `main()` also adds a lot of complexity. Right now I'm using the simplest approach, where we always output the Slang/HLSL entry point as an ordinary function (as written) and then emit a simple GLSL `main()` to call it. I generate globals for all the shader inputs/outputs (these need to be scalarized and have explicit `location`s attached), and then collect these into the `struct` types of the original parameters as needed. - This approach will start to have some major down-sides once we have to deal with "arrayed" input/output - A long-term question here is how to replace entry-point parameter types with scalarized and/or "transposed" versions, while still letting the original code work as written (including copying those inputs to temporary arrays) - Split `BlockStatementSyntaxNode` into: - `BlockStmt` which just provides a scope around a `body` statement - `SeqStmt` which just allows multiple statements to be treated as one - Change how we emit `for` loops, to deal with the case where the initialization part might expand into multiple statements - Basically `for(A;B;C) {D}` becomes `{A; for(;B;C) {D}}`, so we can handle arbitrary statements for `A` - As an additional wrinkle, when we are rewriting HLSL, we just generate `A; for(;B;C) {D}` to deal with the broken scoping there - This change is needed because the lowering pass was sometimes expanding the original initialization statement `A` into a block `{A}`. Certainly if it declared multiple variables we'd need to handle it, and this seemed the easiest way - A more significant challenge for lowering would come if/when we ever wanted to support true short-circuiting behavior for `&&` and `||` - For right now I'm not changing the behavior of the "rewriter" mode, so we still have `UnparsedStmt` instances being generated, but it is clear that eventually we need to parse *all* input, even if we can't type-check 100% of it. This is required so that we can rewrite user code that might refer to a shader input with interface type.
2017-06-29Overhaul `RefPtr` and `String`Tim Foley
- `RefPtr` no longer tries to have distinct cases for interal-vs-external reference counts. Instead we always require an internal reference count. - Types the used `RefPtr` but weren't `RefObject` were made to inherit `RefObject` - The `ReferenceCounted` base class was removed, so that only `RefObject` remains - Implicit conversion from `RefPtr<T>` to `T*` added - This created some complicates for other types that relied on implicit conversions, so this isn't a net cleanup right now - The main type that got messed up by the above was `String`, which previously held a `RefPtr<char, ...>`. This change thus *also* includes a major overhaul of `String`: - `String` now holds all its data via indirection, using a `StringRepresentation` that is a `RefObject`. This object holds a length, capacity, and directly stores the character data in its allocation. This means that `sizeof(String)==sizeof(void*)` - It is now possible to directly mutate a `String` by appending to its representation (we just need to ensure it has a reference count of `1`, possibly by cloning it). This means that `StringBuilder` is now basically just an idomatic use of `String` - A couple operations that just return sub-ranges of a `String` now return `StringSlice` to avoid allocation when it isn't needed. This required more work. - Indices into strings changed from `int` to `UInt` (which is pointer-sized). This had a bunch of follow-on changes because the value `-1` sometimes needs to be special-cased in code that uses indices. Further cleanups are probably needed here.
2017-06-26typoTim Foley
2017-06-26Turn on some debug output for AppVeyor test failure.Tim Foley
2017-06-20Overhaul handling of entry points and translation units.Tim Foley
The main user-visible change here is that instead of `spAddTranslationUnitEntryPoint` we have `spAddEntryPoint`, to reflect that the list of entry points is "global" to a compile request. As a result, `spGetEntryPointSource` now only needs the entry point index, and not the translation unit index. There are a bunch more behind-the-scenes changes, though, reflecting a streamlining of the concepts related to compilation into a smaller number of classes. Now there is: - `Session` (unchanged) to manage the lifetimes of shared stuff like the stdlib - `CompileRequest` (merges in `CompileOptions`) to handle all the lifetime related to a single invocation of the compiler - `TranslationUnitRequest` (merges `TranslationUnitOptions`, `CompileUnit`) to represent a single translation unit ("module") that the user is trying to compile. This is a single file for HLSL/GLSL, but can be multiple files for Slang. - `EntryPointRequest` (merges `EntryPointOption` and a bit of `EntryPointResult`) to track a single entry point that the user is asking to compile (that entry point always comes from a single translation unit) A lot of functions used to take some combination of these and end up with really long signatures. I've given most of the objects "parent" pointers so that they can get back to all the context they need, so most functions don't need as many parameters. It may eventually be important to tease these apart again, in particular: - The code-generation side of things (the `*Result` types) might need to be pulled out in case we want to codegen multiple times from the same AST - Similarly, the layout stuff may also need to be pulled out, in case we want to lay things out multiple times with different rules.
2017-06-19Allow for automatic importing of Slang codeTim Foley
The basic idea of this change is that user code can just write: #include "foo.h" and then if `foo.h` gets found in a list of registered directories for "auto-import," then it actually gets interpreted as if the user had writte, more or less: __import foo; That is, the code in `foo.h` will be treated as Slang, and will be fully parsed and checked (no matter what the source language had been), and the scoping rules will be those of `__import` instead of `#include`. This is a really big hammer, and I could imagine it smashing fingers if used poorly. I'm not sure this feature will pan out, but we need to try things to know. One big piece of that that I'll likely keep in either case is an overhaul of command-line options parsing for `slangc`. In particular, this logic has been moved into the core `slang` library (so that users can just pass options in via the API), and it is all done on UTF-8 strings rather than wide strings (which was always going to be Windows-specific).
2017-06-15Rename `CoreLib::*` to `Slang`Tim Foley
Getting rid of more namespace complexity and stripping things down to the basics. This also gets rid of some dead code in the "core" library.
2017-06-14AppVeyor: Run tests as part of AppVeyor buildsTim Foley
This includes a bunch of related changes: - `slang-test` - Add a notion of an "output mode" that specifies whether we output to console (the default), or invoke the apprpriate AppVeyor command to update test status - Add a notion of test categories, so that tests can be tagged with categories, and then we can invoke only those tets in a given category, or choose to *exclude* tests with specific categories - Allow the `OSProcessSpawner` to look up an executable by "path" (meaning a full path is expected) or by "name" (meaning it should be allowed to look in the current directory, `PATH` environment variable, etc.). This was important to make sure that I can run `appveyor` without having to know its absolute path. - AppVeyor configuration - Change badge to reflect new build account for organization (rather than a single-user account) - Remove attempt to set AppVeyor build version in a clever way, since it breaks links from GitHub to AppVeyor - Change order or configurations in the build matrix to front-load the Release build (which has the main tests) - Turn on `fast_finish` flag so we don't have to wait as long for failed builds - Turn on `parallel` builds - Set `verbosity: minimal` to avoid getting build spew about Xamarin stuff I'm not using - Add custom `test_script` to invoke `test.bat` - Sets the test category based on teh build configuration, so we don't run the full test suite on every input. - `test.bat` - Allow for `-platform` and `-configuration` arguments - Rewrute a platform of `Win32` over to `x86` to match how the output directories are named - Futz around with how the directories are being passed along to work around annoying `.bat` file quoting behavior (I still don't get how batch files work) - Tests - Mark a bunch of tests as `smoke` tests - Mark the relevant tests as `render` tests (these get filtered out for AppVeyor builds)
2017-06-14Testing: Add support for multiple tests per input file.Tim Foley
Now if there are multiple tests listed for an input file like `foo.slang`, they will be output as: passed test: 'foo.slang' passed test: 'foo.slang.1' passed test: 'foo.slang.2' This isn't a perfect solution, but it should work well enough for now. This change doesn't make any tests actually use the new capability.
2017-06-13First pass at support for cross-compilationTim Foley
This is a large change that contains many pieces: - Update the `cross-compile0` test to actually make use of cross compilation. Now the `cross-compile0.hlsl` file contains both HLSL and GLSL source code, and then imports code from `cross-compile0.slang`, which provides a "library" (one function) that can be shared between both the HLSL and GLSL version of things. - Fixed a bug in the support for backslash-escaped newlines. - Added a new `__import` declaration type (replaces the `using` directive that was still around in a vestigial form) An `__import` causes the compiler to look for a Slang source file (currently using the ordinary `#include` lookup logic), and then parse/check the found file as an additional module ("translation unit"), before making its declarations visible in the current scope. - Refactored the main compilation flow to be simpler. There were the `ShaderCompiler` and `ShaderCompilerImpl` classes that weren't relaly doing anything, but added complexity to the whole workflow. - The `render-test` application has been heavily modified to better support testing cross-compilation workflows. At the most basic level we are starting to distinguish pass-through vs. rewriter workflows, and are passing various `#define`s down to the compiler(s) to let the source code be customized as needed for each case. Several annoying corner cases are caused here by having to support the GLSL compilation model, which really wants each entry point in its own specific translation unit, whereas we really want to keep things nicely contained in single files. - Added support for `__intrinsic` operations to have target-specific behavior. This allows a function to be given a different name for some specific target (so a call gets emitted as a call to that other operation). More generally, the library writer can put together an arbitrary format string that will be used in place of expressions that call the given function, e.g.: __intrinsic(hlsl, "$1 - $0") __intrinsic int foo(int a, int b); Given this declaration, a call like `foo(x,y)` will code generate as `x - y` for HLSL, and as `foo(x,y)` for all other targets. Annoying things still to be dealt with: - The way that I'm filtering the user-provided options when passing things down to the compilation of dynamically loaded modules is a bit ad hoc. It would be good to have a systematic notion of which options will be inherited and which won't. There is also more code duplication than I'd like, so we risk having the compiler behave differently when compiling a file at the top level, vs. because of `__import`. - Adding target-specific behavior to intrinsics is all well and good, but the current approach means we can only add this to the original declaration, which limits the ability to easily extend the set of targets. A better approach long-term would be to add a more robust notion of target-based overload resolution (which would happen after semantic checking). Then one mechanism would be used to find the right target-specific overload to use for an operation, and then each (target-specific) definition could use a simpler attribute to intercept code-generation behavior. Note that we might eventually need a similar notion to deal with stage- or profile-specific functions and the overloading behavior around them, so using this for intrinsics doesn't seem like a bad idea.
2017-06-12GLSL: get GLSL limping in `render-test`Tim Foley
The test case that is there right now is nominally a cross-compilation test, but for right now it uses the preprocessor to present completely different code for HLSL and GLSL compilation. This change is really just fleshing out the OpenGL side of `render-test` enough that it can produce images using OpenGL to enable further testing.
2017-06-09Build: more fixes to get `msbuild` to work from command line.Tim Foley
All of this is just related to cruft left over from the old project setup.
2017-06-09glslang: Fixups for buildTim Foley
Some files seem to have been removed since I last sync'ed the glslang code.
2017-06-09Initial import of code.Tim Foley