| Age | Commit message (Collapse) | Author |
|---|
|
The lexer was creating an `unsigned long long` value, and then the AST was storing it in an `int`.
This change makes both use a `long long`.
This is obviously still a stopgap until I can get arbitrary precisions in here.
|
|
Fix handling of literal suffixes
|
|
- Add logic to extract the value and suffix from a numeric literal
- This duplicates some of the lexing logic, but this is hard to avoid without redundant runtime work
- Note that I'm not using and stdlib string-to-number code. This should be more robust once it is working, but it is obviously error prone in the near term. The main up-sides to this are:
- We can handle binary integer literals
- We can handle hexadecimal floating-point literals without stdlib support
- We can hypothetically support digit separators, if we ever wanted
- The parser looks at the suffix characters sliced off by the lexer, and tries to pick a type to use for a literal
- It uses `NULL` if there is no suffix, to avoid some nasty order dependencies where the stdlib might need to parse a number before it has seen the definition of `int`
- Right now I only handle a few cases, so there may be bugs lurking here
- The emit logic needs to handle the fact that a literal node in the AST might have a non-default type attached.
- Right now I just quickly check for the most likely types, and emit the literal with a matching suffix. This doesn't seem robust if any source language supports a suffix for a type where a target has no corresponding suffix. In the long term some amount of casting is probably required.
|
|
These had a typo (`Literial`), so they needed a fix eventually.
I also went ahead and made things a bit more verbose (`IntegerLiteral`, `FloatingPointLiteral`) because these names don't get used often enough for the brevity to pay off.
|
|
Allow "dotted" import paths
|
|
The code:
__import foo.bar;
will try to import from a file matching "foo/bar.slang".
I also went ahead and allowed a raw string literal in and import:
__import "foo/bar";
(In the latter case, an explicit `/` must be used instead of `.`)
|
|
Allow for re-export of an `import` declaration
|
|
If module `A.slang` contains `__exported __import B;` then any declarations from `B.slang` will be visible to any client code that does `__import A;`.
This allows a user to make a single "umbrella" file that encompases a bunch of code files.
Note that this really only affects scoping during Slang compilation/checking; at code generation time everything always gets emitted as raw HLSL/GLSL so that names will be visible whether we want them to be or not.
|
|
Emit global-scope parameters from imported files.
|
|
This fixes up a bug in the earlier change that provided reflection for imported parameters; I'd failed to confirm that the code generation logic can handle imported parameters correctly.
The main fix was to have an `import` declaration automatically use the global-scope layout already determined, sine imported parameters will in general appear there.
|
|
Fix parsing of string literals.
|
|
String literals can be used as part of attributes, but we lacked an actual AST representation for them.
This change adds basic parsing for string literals, as well as emit logic for them.
I also included a fix for parsing of chained right-associative operators.
To test these fixes, I've re-enabled one of the HLSL tests I disabled a while back. It would be good to go through and see how many of those we can re-enable now.
|
|
Add support for `#import`
|
|
Previously the code checked for a duplicate `#import` using a data structure attached to the compile request, but this would fail for nested imports.
It also wouldn't work for a combination of `#import` and `__import`.
This change makes it so that we instead track a set of already-imported modules in the semantic checking visitor, which is instantiated once per translation unit.
We also key this set on the actual module (AST) imported, rather than on path/name/whatever, so hopefully it will be robust to the same thing getting imported multiple ways.
|
|
With this change, there is now a meaningful semantic difference between `__import` and `#import`.
An `__import` compiles the target file in a fresh environment, only providing it any macro definitions passed via command line or API. Any macros defined in the imported file are not made visible at the import site. One can think of an `__import` as a bit like `using namespace` in C++.
A `#import` will tokenize the input in the same preprocessor environment as the importing file, and any macros defined along the way will be visible in the parent file.
It is a *bit* like a `#include` with two big differences:
- The imported code is always parsed as Slang, and as its own module with default flags, etc. (so semantic checks are on even if we are in "rewriter" mode). It is pulled into the outer namespace just as for `__import`.
- A given file will only get `#import`ed once for a translation unit, so it behaves a bit like there is an implicit `#pragma once` in the target file
|
|
Right now `#import` only differs from `#include` in that it takes a string literal for a file name instead of a raw identifier (to which `.slang` gets appended).
The next step is to make `#import` respect preprocessor state, while `__import` doesn't.
|
|
Reflect imported code
|
|
This is the cause of the test failures on AppVeyor.
|
|
|
|
|
|
- The basic idea is simple: be sure to enumerate code in `__import`ed modules when generating reflection info
- Note that we don't currently allow an entry point to appear in an imported module, so we only consider globlal-scope parameters
- Although there isn't currently a real implementation of namespacing, I went ahead and ensured that parameters in imported modules are treated as distinct from parameters in the user's code, even if they have the same name.
|
|
Standard library additions + supporting features
|
|
These are annoyingly subtle.
|
|
The operator was being declared as `IntrinsicOp::Not` when it should be `IntrinsicOp::BitNot`
|
|
These were being passed over by the emit logic because I didn't have tests that used them.
|
|
The catch with these operations is that they return a vector based on the scalar of the element type of the texture.
That is, given `Texture2D<float> t` the operation `t.GatherRed(...)` should return a `float4`.
The ideal way to solve this would use associated types, but we aren't there yet, so I am using extension declarations.
An extension can "capture" the identity of the element type, like so:
__generic<T, let N : int> __extension Texture2D<vector<T,N> > { ... }
That extension will match `Texture2D<float3>` and correctly capture `T == float`, so that we can use it in other operations.
Getting this working required a bunch of changes:
- Actually emit the relevant extension declarations in the stdlib
- Fix the parser to be able to parse `Texture2D<vector<T,N> >` (that is, a nested generic app).
- I actually went ahead and significantly overhauled the expression parser while I was there, because I just couldn't deal with the existing code any longer.
- Added support for general-case lookup to look through `__extension` declarations. I had logic in place to special-case this for looking up "constructors" but hadn't done anything for general member lookup yet.
- This required some annoying holes to be punched through the layers, because lookup might need to invoke semantic analysis to ensure that an extension has been checked.
- There is some first-pass code trying to support looking up a `typedef` nested inside the `vector` type. This is a nice idea in principle, but the problem is that the `Texture2D<T>` definition would be looking up `T.Element` and not `float4.Element`, and that means we'd need machinery for doing lookup *through* interface conformances for a type parameter like `T`
The big gotcha here is that none of this logic applies to `Texture2D<float>` (the original case I mentioned) because I am matching vector types and not scalars.
Matching scalars *should* be as easy as:
__generic<T : __BuitlinScalarType> __extension Texture2D<T> { ... }
But I'd need to confirm that interface constraints like that actually work, or else that extension would *also* apply to `Texture2D<float4>` and break everything.
|
|
- Vector constructors that take two vectors that add up to the target size (`float4(float2, float2)`)
- I now realize I implemented the general case here, but there really is only the one case...
- Geometry shader output stream types now have `Append()` and `RestartStrip()` methods
|
|
Falcor work
|
|
If the user imports a module along more than one path, we need to make sure we don't emit the code twice.
I handle this by keeping a set of already-emitted modules.
Down the line, a more robust code generation strategy for non-"rewriter" use cases would be handling this at the per-declaration level, and this logic wouldn't really be needed.
|
|
This was just missing logic in the parser.
|
|
This helps ensure that we pull things into scope at the right time.
|
|
Overhaul handling of entry points and translation units.
|
|
The main user-visible change here is that instead of `spAddTranslationUnitEntryPoint` we have `spAddEntryPoint`, to reflect that the list of entry points is "global" to a compile request.
As a result, `spGetEntryPointSource` now only needs the entry point index, and not the translation unit index.
There are a bunch more behind-the-scenes changes, though, reflecting a streamlining of the concepts related to compilation into a smaller number of classes.
Now there is:
- `Session` (unchanged) to manage the lifetimes of shared stuff like the stdlib
- `CompileRequest` (merges in `CompileOptions`) to handle all the lifetime related to a single invocation of the compiler
- `TranslationUnitRequest` (merges `TranslationUnitOptions`, `CompileUnit`) to represent a single translation unit ("module") that the user is trying to compile. This is a single file for HLSL/GLSL, but can be multiple files for Slang.
- `EntryPointRequest` (merges `EntryPointOption` and a bit of `EntryPointResult`) to track a single entry point that the user is asking to compile (that entry point always comes from a single translation unit)
A lot of functions used to take some combination of these and end up with really long signatures.
I've given most of the objects "parent" pointers so that they can get back to all the context they need, so most functions don't need as many parameters.
It may eventually be important to tease these apart again, in particular:
- The code-generation side of things (the `*Result` types) might need to be pulled out in case we want to codegen multiple times from the same AST
- Similarly, the layout stuff may also need to be pulled out, in case we want to lay things out multiple times with different rules.
|
|
Fix types for `InputPatch` and `OutputPatch`
|
|
Fixes #34.
I'd declared these as if they were `InputPatch<T>`, but they are really `InputPatch<T,N>`.
This change fixes the declarations, and makes these types no longer inherit from the contrived `BuiltinGenericType`.
Instead they are more-or-less ordinary `DeclRefType`s using the same approach that `MatrixExpressionType` uses.
|
|
Line directives
|
|
Fixes for preprocessor conditionals that use macros
|
|
Bug fix for newline escaping.
|
|
The basic underlying problem here is that the preprocessor tries to linger at the end of an input stream until it is sure it is time to advance.
An input stream can include raw input files, or the expansion of a macro or macro argument.
This was originally done to deal with not getting good end-of-line tokens when in directives (that issue has been fixed), but it is now a legacy issue that should probably be removed (but I am wary of making such a sweeping change).
The problem that arises is that some code depends on what the actual input stream is (e.g., when turning conditionals on/off), and so we need to be careful.
The bugs that this change affects arise when a `#if` or `#elseif` conditional expression *ends* with a macro expansion:
#define FOO 2
#if 2 == FOO
...
#endif
When we try to start the preprocessor conditional block the "active" stream is still the expansion of `FOO`, when we needed it to be the input file.
We fix this for now by snapshotting the input stream at the start of the directive, but a better long term fix would be to fix up this weird end-of-input behavior.
|
|
The previous changes had left out logic for "scrubbing" a token value that includes an escaped newline, because I expected it would only occur within whitespace. Unfortunately, some user code looked like this:
```
a + b
```
That is, there was a token at the very start of the line, after the escaped newline.
As a result, after consuming the leading whitespace (which didn't end up consuming the escaped newline - but we could consider making it do so in future), the lexer started to lex a token that *starts* with an escaped newline, but turns out to be an identifer (which gets an invalid name).
This change adds some ad-hoc code to "scrub" the value of *every* token, which wasteful but at least solves the problem.
|
|
We now output a line directive for (nearly) every declaration, statement, and modifier, so that hopefully there will be fewer cases where a downstream error doesn't point to the correct line.
This exposes a lot of issues where we can/should clean up the simplicity of the code we emit (e.g., not output redundant parens; tracking source locations for types better).
These kinds of issues will need to be addressed in follow-on changes.
A few big ones:
- Because GLSL doesn't allow for file names in `#line` directives, we really need to expose some data that can clean up error messages (or can be used by an application to do the same) so that they know which file is which.
- We really need a command line option (and an equivalent API flag) to turn off emission of `#line` directies, so that the user can get moderately clean code as output.
|
|
If the line number for the next token is within a small range, then go ahead and output newlines to get caught up, rather than emit a `#line` directive. This saves a small amount of clutter, and in the particular case where the number of lines is 1, it stops our current behavior of putting a directive on each line.
|
|
Make sure that semantic checks always apply to Slang files
|
|
That is, even if hte user specified the `-no-checking` option (or the equivalent via API), we still want/need to apply full semantic checks to Slang code, so that cross-compilation will be possible.
|
|
Allow for automatic importing of Slang code
|
|
The basic idea of this change is that user code can just write:
#include "foo.h"
and then if `foo.h` gets found in a list of registered directories for "auto-import," then it actually gets interpreted as if the user had writte, more or less:
__import foo;
That is, the code in `foo.h` will be treated as Slang, and will be fully parsed and checked (no matter what the source language had been), and the scoping rules will be those of `__import` instead of `#include`.
This is a really big hammer, and I could imagine it smashing fingers if used poorly.
I'm not sure this feature will pan out, but we need to try things to know.
One big piece of that that I'll likely keep in either case is an overhaul of command-line options parsing for `slangc`. In particular, this logic has been moved into the core `slang` library (so that users can just pass options in via the API), and it is all done on UTF-8 strings rather than wide strings (which was always going to be Windows-specific).
|
|
Bug fix: handle unchecked operator application in emit logic
|
|
When in rewriter mode, the emit logic will never see function applications inside function bodies, but it *will* see function application expressions at global scope, and some of these expressions might be unchecked.
The challenge here is that even simple math operations now show up as function calls, so we need a bit of special-case logic to detect unchecked calls and then emit them using the syntax they were written with (e.g., use infix syntax if they were written as an infix expression).
|
|
Replace `DeclRef` approach
|
|
For context: a `DeclRef` is supposed to capture both a pointer to a particualr declaration, and also any information needed to specialize that declaration for a context (e.g., generic parameter substitutions).
The existing approach had a hiearchy of specialized decl-ref types that mirrored the AST hierarchy, but that led to a lot of boilerplate where you had to recapitulate the exact same hierarchy.
The new appraoch basically treats `DeclRef<T>` as a sort of "smart pointer" in that it wraps a pointer to a `T` (the declaration), plus a side field for the specialization info, and then allows it to be cast as needed to other types (where the pointer cast would be allowed), while carrying along the side info.
To enable this, all the things that used to be member functions of declaration-reference types are now free functions that take a `DeclRef<T>` for some specific `T` as a parameter.
|
