Making it easier to work with shaders https://git.yummers.dev/slang.git/atom?h=master 2019-05-31T21:20:37+00:00 2019-05-31T21:20:37+00:00 jsmall-nvidia jsmall@nvidia.com 2019-05-31T21:20:37+00:00 urn:sha1:6cbc3929a54d37bd23cb5efa8e3320ba02f78b2f * Prefixing source files in source/slang with slang- * Prefix source in source/slang with slang- prefix. * Rename core source files with slang- prefix. * Update project files. * Fix problems from automatic merge. 2019-01-22T22:57:25+00:00 Tim Foley tfoleyNV@users.noreply.github.com 2019-01-22T22:57:25+00:00 urn:sha1:a08a3140716b89146bf0a22dc014c5470e90e910 The AST class hierarchy for variable declarations had a few messy bits. First, there are more subclasses of `VarDeclBase` than seem strictly necessary; especially for stuff like `struct` member variable which use `StructField` even for `static` fields (which are effectively globals). Second, the AST node type for the "cases" within an `enum` was made a subclass of `VarDeclBase` for expediency, but this isn't really semantically accurate (and doesn't seem to be paying off much in deduplication of code). This change tries to address both of those problems. First, we replace the existing `Variable` and `StructField` cases with a single `VarDecl` case that covers globals, locals, and member variables. I haven't gone so far as to replace function parameters or generic value parameters, but that might be worth considering as a further clean-up. Second, we change `EnumCaseDecl` to inherit directly from `Decl` instead of `VarDeclBase` and add an explicit case for handling them where they were previously handled as if they were variable declarations (this was done by manually surveying all locations in the code that referenced `VarDeclBase`). 2017-08-09T17:23:09+00:00 Tim Foley tfoley@nvidia.com 2017-08-09T17:13:40+00:00 urn:sha1:695c2700de54a5fec72ce7214c137a1dc3a02d7b - `ExpressionSyntaxNode` becomes `Expr` - `StatementSyntaxNode` becomes `Stmt` - `StructSyntaxNode` becomes `StructDecl` - `ProgramSyntaxNode` becomes `ModuleDecl` - `ExpressionType` becomes `Type` - Existing fields names `Type` become `type` - There might be some collateral damage here if there were, e.g., `enum`s named `Type`, but I can live with that for now and fix those up as a I see them 2017-07-18T21:51:12+00:00 Tim Foley tfoley@nvidia.com 2017-07-18T21:51:12+00:00 urn:sha1:2476c035ec15d3ee22239ebd1fe10e6e8c1e01e3 The basic syntax is: $for(i in Range(0,99)) { /* stuff goes here */ } Note that the exact form is very restrictive. All that you are allowed to change is `i`, `0`, `99` or `/* stuff goes here */`. As a tiny bit of syntax sugar, the following should work: $for(i in Range(99)) { /* stuff goes here */ } Note that the range given is half-open (C++ iterator `[begin,end)` style). Both the beginning and end of the range must be compile-time constant expressions that Slang knows how to constant-fold. The implementation will basically generate code for `/* stuff goes here */` N times, once for each value in the half-open range. Each time, the variable `i` will be replaced with a different compile-time-constant expression. While I was working on a test case for this, I also found that our build of glslang had an issue with resource limits, so I fixed that. Clients will need to build a new glslang to use the fix. 2017-07-09T01:22:26+00:00 Tim Foley tfoley@nvidia.com 2017-07-09T00:47:31+00:00 urn:sha1:780a0bcd3724cad77cb65f931f111273776c9ca4 If the user doesn't use any `import` declarations, there is no reason to parse their code at all, so having the option of falling back to `UnparsedStmt` can potentially save us some headaches down the road. The new rule now is that if you have the "no checking" flag on, *and* the parser hasn't yet seen any `import` declarations, then it still used `UnparsedStmt` to avoid touching function bodies. Otherwise, I go ahead and parse function bodies, and assume I can rewrite any code I can semantically understand. 2017-07-09T01:22:26+00:00 Tim Foley tfoley@nvidia.com 2017-07-09T00:21:37+00:00 urn:sha1:a40b6679931f672a911070fcc7eeb41c52e8b8fd HLSL has the bad scoping behavior for `for` loops, and we need to respect that. But, we need to have correct scoping for GLSL, and we'd like it for Slang. We also need to ensure that `for` loops written in a "correct" language get the correct behavior when emitted as HLSL. There was already code to handle this in the emit pass, but it was unfortunately using an `isRewrite` flag to try to tell if the HLSL behavior was wanted. This doesn't work when the code being emitted might come from a mix of languages. This change adds a distinct `UnscopedForStmt` syntax node type, and uses that when parsing HLSL input (bot not for other languages). We make sure to preserve this node type through lowering, and then specialize our emit logic on this case. With this, there are no more remaining uses of `isRewrite` in the emit logic, which is good because it didn't mean what I needed it to mean any more (since we now emit only a single module, that was merged during lowering). 2017-07-09T01:18:36+00:00 Tim Foley tfoley@nvidia.com 2017-07-07T21:30:26+00:00 urn:sha1:f69bc6cdb10aab2d1b202668cb7ecbcc0ddf33f2 This is in anticipation of needing to have more complete knowledge to be able to handle user code that `import`s library functionality. The big picture of this change is just to remove the `UnparsedStmt` class that was used to hold the bodies of user functions as opaque token streams, and thus to let the full parser and compiler loose on that code. That is the easy part, of course, and the hard part is all the fixes that this requires in the rest of the compielr to make this even remotely work. Subsequent commit address a lot of other issues, so this particular commit mostly represents work-in-progress. One detail is that this change puts a conditional around nearly every diagnostic message in `check.cpp` to suppress thing when in rewriter mode. I have yet to check how that works out if there are errors in anything we actually need to understand for the purposes of generating reflection data. 2017-07-06T16:17:04+00:00 Tim Foley tfoley@nvidia.com 2017-06-28T20:34:38+00:00 urn:sha1:f145e09a6dcbcf326f782b3e6a76dbf291c792cf - The big change here is the introduction of a "lowering" pass that takes an input AST from the semantic checker, and produces an output AST suitable for emitting. The intention is that he lowering pass is responsible for: - Stripping out unused code (when we have enough information to do so), by only outputting declarations that are transitively references from an entry point - When cross-compiling to GLSL, generating a suitable `void main()` entry point to wrap the user-written entry-point function - (Eventually) legalizing types in the program, by scalarizing aggregate types that mix uniform and resource types - (Eventually) instantiating generic declarations so that the resulting code only deals with fully specialized declarations - (Eventually) de-sugaring OOP constructs into basic "structs and functions" form - (Eventually) instantiating code that depends on interface types at the concrete types chosen - It is clear that there is still a lot of work to be done there, to this change is really about getting infrastructure in place without breaking the existing test cases. - One cleanup here is that we get rid of the idea of whole-translation-unit output, since that was specific to HLSL output, and there is really no strong reason for keeping it. Users should now just ask for the output for each entry point that they wanted to generate. - The biggest source of complexity for the lowering process is that it needs to produce the same AST structure as the input, to deal with the complexity of the rewriter case. That is, we need the output to be able to reproduce the input exactly in the case where we are rewriting and nothing needs to change, so the output format needs at least the degrees of freedom of the input. - As a result, we end up having to distinguish "rewriter" and "full" modes in both lowering and code-emit steps, so that we can react appropriately. - Generating a GLSL `main()` also adds a lot of complexity. Right now I'm using the simplest approach, where we always output the Slang/HLSL entry point as an ordinary function (as written) and then emit a simple GLSL `main()` to call it. I generate globals for all the shader inputs/outputs (these need to be scalarized and have explicit `location`s attached), and then collect these into the `struct` types of the original parameters as needed. - This approach will start to have some major down-sides once we have to deal with "arrayed" input/output - A long-term question here is how to replace entry-point parameter types with scalarized and/or "transposed" versions, while still letting the original code work as written (including copying those inputs to temporary arrays) - Split `BlockStatementSyntaxNode` into: - `BlockStmt` which just provides a scope around a `body` statement - `SeqStmt` which just allows multiple statements to be treated as one - Change how we emit `for` loops, to deal with the case where the initialization part might expand into multiple statements - Basically `for(A;B;C) {D}` becomes `{A; for(;B;C) {D}}`, so we can handle arbitrary statements for `A` - As an additional wrinkle, when we are rewriting HLSL, we just generate `A; for(;B;C) {D}` to deal with the broken scoping there - This change is needed because the lowering pass was sometimes expanding the original initialization statement `A` into a block `{A}`. Certainly if it declared multiple variables we'd need to handle it, and this seemed the easiest way - A more significant challenge for lowering would come if/when we ever wanted to support true short-circuiting behavior for `&&` and `||` - For right now I'm not changing the behavior of the "rewriter" mode, so we still have `UnparsedStmt` instances being generated, but it is clear that eventually we need to parse *all* input, even if we can't type-check 100% of it. This is required so that we can rewrite user code that might refer to a shader input with interface type. 2017-06-30T19:22:33+00:00 Tim Foley tfoley@nvidia.com 2017-06-29T15:55:09+00:00 urn:sha1:cab694dcead92a554654d7fa3f08909d519425f0 - The big change here is that all the definitions for syntax-node classes have been macro-ized, to that we can do light metaprogramming over them - The use of macros for this has big down-sides, but I'm not quite ready to do anything more heavy-weight right now - The macro-ized definitions can be included multiple times, to generate different declarations/code as needed - The first example of using this meta-programming facility is a new visitor system - The actual visitor base classes and the dispatch logic are all generated from the meta-files - There was only one visitor left in the code: the semantics checker, so that was ported to the new system. - All current test cases pass, so *of course* that means all is well.