| Age | Commit message (Collapse) | Author |
|---|
|
- Add logic to extract the value and suffix from a numeric literal
- This duplicates some of the lexing logic, but this is hard to avoid without redundant runtime work
- Note that I'm not using and stdlib string-to-number code. This should be more robust once it is working, but it is obviously error prone in the near term. The main up-sides to this are:
- We can handle binary integer literals
- We can handle hexadecimal floating-point literals without stdlib support
- We can hypothetically support digit separators, if we ever wanted
- The parser looks at the suffix characters sliced off by the lexer, and tries to pick a type to use for a literal
- It uses `NULL` if there is no suffix, to avoid some nasty order dependencies where the stdlib might need to parse a number before it has seen the definition of `int`
- Right now I only handle a few cases, so there may be bugs lurking here
- The emit logic needs to handle the fact that a literal node in the AST might have a non-default type attached.
- Right now I just quickly check for the most likely types, and emit the literal with a matching suffix. This doesn't seem robust if any source language supports a suffix for a type where a target has no corresponding suffix. In the long term some amount of casting is probably required.
|
|
These had a typo (`Literial`), so they needed a fix eventually.
I also went ahead and made things a bit more verbose (`IntegerLiteral`, `FloatingPointLiteral`) because these names don't get used often enough for the brevity to pay off.
|
|
The previous changes had left out logic for "scrubbing" a token value that includes an escaped newline, because I expected it would only occur within whitespace. Unfortunately, some user code looked like this:
```
a + b
```
That is, there was a token at the very start of the line, after the escaped newline.
As a result, after consuming the leading whitespace (which didn't end up consuming the escaped newline - but we could consider making it do so in future), the lexer started to lex a token that *starts* with an escaped newline, but turns out to be an identifer (which gets an invalid name).
This change adds some ad-hoc code to "scrub" the value of *every* token, which wasteful but at least solves the problem.
|
|
This gets rid of one unecessary namespace.
|
|
This is really messy and I'm not entirely happy with the result, but at least we handle a couple more corner cases.
|
|
This is mostly to allow for the idiomatic style of defining a multi-line macro in C:
#define FOO(a,b) \
x(a) \
y(b) \
/* end */
The handling is reasonably general: in the lexer whenever we need to consume or "peek" the next code point, we check if we are at the start of a backslash-newline sequence, and if so we skip past that to find what we were looking for.
However, the way I'm handling things right now there is no step taken to "clean" a token and remove the backslash or newline from its value, so downstream code that actually inspects token values will probably break if users start putting escaped newlines in the middle of names.
We can fix that issue when (if) it comes up.
|
|
|
