Lexer: handle escaped newlines

This is mostly to allow for the idiomatic style of defining a multi-line macro in C:

    #define FOO(a,b) \
        x(a)         \
        y(b)         \
        /* end */

The handling is reasonably general: in the lexer whenever we need to consume or "peek" the next code point, we check if we are at the start of a backslash-newline sequence, and if so we skip past that to find what we were looking for.

However, the way I'm handling things right now there is no step taken to "clean" a token and remove the backslash or newline from its value, so downstream code that actually inspects token values will probably break if users start putting escaped newlines in the middle of names.

We can fix that issue when (if) it comes up.

0 files changed, 0 insertions, 0 deletions