Preprocessor: fix bug with multi-argument macros.

There was a subtle bug when a function-like macro with multiple arguments expands to use the arguments one after the other:

    #define FOO(a,b) a b

    FOO(int, x);

During expansion, the input streams look something like (using `.` to represent the cursor):

    // macro invocation:
    FOO(int, x) . ;

    // macro expansion of `FOO(int, x)`
    a . b

    // macro argument `a`

    int .

That is, we are at the end of the first argument's tokens.
When "peeking" the next token, we correctly work up the list of active streams until we find one that isn't at its end, and that gives us the token `b`.

But then we need to look up `b` in an appropriate environment to find what it is bound to.
Each of the streams above has an environment asociated with it, and in particular, `b` is only defined in the middle environment, because that is where the macro arguments were registered.

The simple fix here is to make the lookup logic for finding an environment follow the same logic as finding the next token.

A more complete fix down the line could involve getting rid of the approach of allowing an input stream to be "active" but at its end.
I believe this was originally required to handle some error cases in directives, where we'd want to keep the input stream for one file active until we are done parsing a full directive from it (e.g., if a directive is on the last line of the file).
Now that we generate an explicit "end of directive" token, that may not be required.

1 files changed, 28 insertions, 1 deletions

diff --git a/source/slang/preprocessor.cpp b/source/slang/preprocessor.cpp
index cdde2591d..60329c275 100644
--- a/source/slang/preprocessor.cpp
+++ b/source/slang/preprocessor.cpp
@@ -450,8 +450,35 @@ static PreprocessorMacro* LookupMacro(PreprocessorEnvironment* environment, Stri
 
 static PreprocessorEnvironment* GetCurrentEnvironment(Preprocessor* preprocessor)
 {
+    // The environment we will use for looking up a macro is assocaited
+    // with the current input stream (because it may include entries
+    // for macro arguments).
+    //
+    // We need to be careful, though, when we are at the end of an
+    // input stream (e.g., representing one argument), so that we
+    // don't use its environment.
+
     PreprocessorInputStream* inputStream = preprocessor->inputStream;
-    return inputStream ? inputStream->environment : &preprocessor->globalEnv;
+
+    for(;;)
+    {
+        // If there is no input stream that isn't at its end,
+        // then fall back to the global environment.
+        if (!inputStream)
+            return &preprocessor->globalEnv;
+
+        // If the current input stream is at its end, then
+        // fall back to its parent stream.
+        if (inputStream->tokenReader.PeekTokenType() == TokenType::EndOfFile)
+        {
+            inputStream = inputStream->parent;
+            continue;
+        }
+
+        // If we've found an active stream that isn't at its end,
+        // then use that for lookup.
+        return inputStream->environment;
+    }
 }
 
 static PreprocessorMacro* LookupMacro(Preprocessor* preprocessor, String const& name)