diff options
| author | Tim Foley <tfoley@nvidia.com> | 2017-06-12 12:26:12 -0700 |
|---|---|---|
| committer | Tim Foley <tfoley@nvidia.com> | 2017-06-13 13:53:47 -0700 |
| commit | d81c347e0edcbbf181885baf2b13978c28dfc9a8 (patch) | |
| tree | ed48ba12c7db141d517b29343b9127692e72ad8c /source | |
| parent | a14f3c50ef2cf7d3e34e729dfe10ce1cec880955 (diff) | |
First pass at support for cross-compilation
This is a large change that contains many pieces:
- Update the `cross-compile0` test to actually make use of cross compilation.
Now the `cross-compile0.hlsl` file contains both HLSL and GLSL source code, and then imports code from `cross-compile0.slang`, which provides a "library" (one function) that can be shared between both the HLSL and GLSL version of things.
- Fixed a bug in the support for backslash-escaped newlines.
- Added a new `__import` declaration type (replaces the `using` directive that was still around in a vestigial form)
An `__import` causes the compiler to look for a Slang source file (currently using the ordinary `#include` lookup logic), and then parse/check the found file as an additional module ("translation unit"), before making its declarations visible in the current scope.
- Refactored the main compilation flow to be simpler. There were the `ShaderCompiler` and `ShaderCompilerImpl` classes that weren't relaly doing anything, but added complexity to the whole workflow.
- The `render-test` application has been heavily modified to better support testing cross-compilation workflows. At the most basic level we are starting to distinguish pass-through vs. rewriter workflows, and are passing various `#define`s down to the compiler(s) to let the source code be customized as needed for each case.
Several annoying corner cases are caused here by having to support the GLSL compilation model, which really wants each entry point in its own specific translation unit, whereas we really want to keep things nicely contained in single files.
- Added support for `__intrinsic` operations to have target-specific behavior.
This allows a function to be given a different name for some specific target (so a call gets emitted as a call to that other operation).
More generally, the library writer can put together an arbitrary format string that will be used in place of expressions that call the given function, e.g.:
__intrinsic(hlsl, "$1 - $0") __intrinsic int foo(int a, int b);
Given this declaration, a call like `foo(x,y)` will code generate as `x - y` for HLSL, and as `foo(x,y)` for all other targets.
Annoying things still to be dealt with:
- The way that I'm filtering the user-provided options when passing things down to the compilation of dynamically loaded modules is a bit ad hoc. It would be good to have a systematic notion of which options will be inherited and which won't. There is also more code duplication than I'd like, so we risk having the compiler behave differently when compiling a file at the top level, vs. because of `__import`.
- Adding target-specific behavior to intrinsics is all well and good, but the current approach means we can only add this to the original declaration, which limits the ability to easily extend the set of targets.
A better approach long-term would be to add a more robust notion of target-based overload resolution (which would happen after semantic checking). Then one mechanism would be used to find the right target-specific overload to use for an operation, and then each (target-specific) definition could use a simpler attribute to intercept code-generation behavior.
Note that we might eventually need a similar notion to deal with stage- or profile-specific functions and the overloading behavior around them, so using this for intrinsics doesn't seem like a bad idea.
Diffstat (limited to 'source')
| -rw-r--r-- | source/slang/check.cpp | 50 | ||||
| -rw-r--r-- | source/slang/compiler.cpp | 857 | ||||
| -rw-r--r-- | source/slang/compiler.h | 56 | ||||
| -rw-r--r-- | source/slang/emit.cpp | 250 | ||||
| -rw-r--r-- | source/slang/parser.cpp | 74 | ||||
| -rw-r--r-- | source/slang/slang-stdlib.cpp | 79 | ||||
| -rw-r--r-- | source/slang/slang.cpp | 792 | ||||
| -rw-r--r-- | source/slang/syntax-visitors.h | 18 | ||||
| -rw-r--r-- | source/slang/syntax.cpp | 7 | ||||
| -rw-r--r-- | source/slang/syntax.h | 67 |
10 files changed, 1331 insertions, 919 deletions
diff --git a/source/slang/check.cpp b/source/slang/check.cpp index 7d50c5978..a1b7393fb 100644 --- a/source/slang/check.cpp +++ b/source/slang/check.cpp @@ -49,15 +49,18 @@ namespace Slang ProgramSyntaxNode * program = nullptr; FunctionSyntaxNode * function = nullptr; CompileOptions const* options = nullptr; + CompileRequest* request = nullptr; // lexical outer statements List<StatementSyntaxNode*> outerStmts; public: SemanticsVisitor( DiagnosticSink * pErr, - CompileOptions const& options) + CompileOptions const& options, + CompileRequest* request) : SyntaxVisitor(pErr) , options(&options) + , request(request) { } @@ -686,7 +689,7 @@ namespace Slang int argIndex = 0; - for(auto& fieldDeclRef : toStructDeclRef.GetMembersOfType<FieldDeclRef>()) + for(auto fieldDeclRef : toStructDeclRef.GetMembersOfType<FieldDeclRef>()) { if(argIndex >= argCount) { @@ -1083,7 +1086,7 @@ namespace Slang RefPtr<Modifier> checkModifier( RefPtr<Modifier> m, - Decl* decl) + Decl* /*decl*/) { if(auto hlslUncheckedAttribute = m.As<HLSLUncheckedAttribute>()) { @@ -1878,7 +1881,7 @@ namespace Slang if (!funcDeclRefExpr) return nullptr; auto funcDeclRef = funcDeclRefExpr->declRef; - auto intrinsicMod = funcDeclRef.GetDecl()->FindModifier<IntrinsicModifier>(); + auto intrinsicMod = funcDeclRef.GetDecl()->FindModifier<IntrinsicOpModifier>(); if (!intrinsicMod) return nullptr; // Let's not constant-fold operations with more than a certain number of arguments, for simplicity @@ -4892,13 +4895,44 @@ namespace Slang return expr; } + + virtual void visitImportDecl(ImportDecl* decl) override + { + // We need to look for a module with the specified name + // (whether it has already been loaded, or needs to + // be loaded), and then put its declarations into + // the current scope. + + auto name = decl->nameToken.Content; + auto scope = decl->scope; + + // Try to load a module matching the name + auto importedModuleDecl = findOrImportModule(request, name, decl->nameToken.Position); + + // If we didn't find a matching module, then bail out + if (!importedModuleDecl) + return; + + // Record the module that was imported, so that we can use + // it later during code generation. + decl->importedModuleDecl = importedModuleDecl; + + // Create a new sub-scope to wire the module + // into our lookup chain. + auto subScope = new Scope(); + subScope->containerDecl = importedModuleDecl.Ptr(); + + subScope->nextSibling = scope->nextSibling; + scope->nextSibling = subScope; + } }; - SyntaxVisitor * CreateSemanticsVisitor( - DiagnosticSink * err, - CompileOptions const& options) + SyntaxVisitor* CreateSemanticsVisitor( + DiagnosticSink* err, + CompileOptions const& options, + CompileRequest* request) { - return new SemanticsVisitor(err, options); + return new SemanticsVisitor(err, options, request); } // diff --git a/source/slang/compiler.cpp b/source/slang/compiler.cpp index e78de9133..d02e5d10b 100644 --- a/source/slang/compiler.cpp +++ b/source/slang/compiler.cpp @@ -53,606 +53,501 @@ namespace Slang // - int compilerInstances = 0; - - class ShaderCompilerImpl : public ShaderCompiler + String EmitHLSL(ExtraContext& context) { - public: - - // Actual context for compilation... :( - struct ExtraContext + if (context.getOptions().passThrough != PassThroughMode::None) { - CompileOptions const* options = nullptr; - TranslationUnitOptions const* translationUnitOptions = nullptr; - - CompileResult* compileResult = nullptr; - - RefPtr<ProgramSyntaxNode> programSyntax; - ProgramLayout* programLayout; - - String sourceText; - String sourcePath; - - CompileOptions const& getOptions() { return *options; } - TranslationUnitOptions const& getTranslationUnitOptions() { return *translationUnitOptions; } - }; - - - String EmitHLSL(ExtraContext& context) + return context.sourceText; + } + else { - if (context.getOptions().passThrough != PassThroughMode::None) - { - return context.sourceText; - } - else - { - // TODO(tfoley): probably need a way to customize the emit logic... - return emitProgram( - context.programSyntax.Ptr(), - context.programLayout, - CodeGenTarget::HLSL); - } + // TODO(tfoley): probably need a way to customize the emit logic... + return emitProgram( + context.programSyntax.Ptr(), + context.programLayout, + CodeGenTarget::HLSL); } + } - String emitGLSLForEntryPoint(ExtraContext& context, EntryPointOption const& entryPoint) + String emitGLSLForEntryPoint(ExtraContext& context, EntryPointOption const& /*entryPoint*/) + { + if (context.getOptions().passThrough != PassThroughMode::None) { - if (context.getOptions().passThrough != PassThroughMode::None) - { - return context.sourceText; - } - else - { - // TODO(tfoley): probably need a way to customize the emit logic... - return emitProgram( - context.programSyntax.Ptr(), - context.programLayout, - CodeGenTarget::GLSL); - } + return context.sourceText; } + else + { + // TODO(tfoley): probably need a way to customize the emit logic... + return emitProgram( + context.programSyntax.Ptr(), + context.programLayout, + CodeGenTarget::GLSL); + } + } - char const* GetHLSLProfileName(Profile profile) + char const* GetHLSLProfileName(Profile profile) + { + switch(profile.raw) { - switch(profile.raw) - { - #define PROFILE(TAG, NAME, STAGE, VERSION) case Profile::TAG: return #NAME; - #include "profile-defs.h" + #define PROFILE(TAG, NAME, STAGE, VERSION) case Profile::TAG: return #NAME; + #include "profile-defs.h" - default: - // TODO: emit an error here! - return "unknown"; - } + default: + // TODO: emit an error here! + return "unknown"; } + } #ifdef _WIN32 - void* GetD3DCompilerDLL() - { - // TODO(tfoley): let user specify version of d3dcompiler DLL to use. - static HMODULE d3dCompiler = LoadLibraryA("d3dcompiler_47"); - // TODO(tfoley): handle case where we can't find it gracefully - assert(d3dCompiler); - return d3dCompiler; - } + void* GetD3DCompilerDLL() + { + // TODO(tfoley): let user specify version of d3dcompiler DLL to use. + static HMODULE d3dCompiler = LoadLibraryA("d3dcompiler_47"); + // TODO(tfoley): handle case where we can't find it gracefully + assert(d3dCompiler); + return d3dCompiler; + } - List<uint8_t> EmitDXBytecodeForEntryPoint( - ExtraContext& context, - EntryPointOption const& entryPoint) + List<uint8_t> EmitDXBytecodeForEntryPoint( + ExtraContext& context, + EntryPointOption const& entryPoint) + { + static pD3DCompile D3DCompile_ = nullptr; + if (!D3DCompile_) { - static pD3DCompile D3DCompile_ = nullptr; - if (!D3DCompile_) - { - HMODULE d3dCompiler = (HMODULE)GetD3DCompilerDLL(); - assert(d3dCompiler); + HMODULE d3dCompiler = (HMODULE)GetD3DCompilerDLL(); + assert(d3dCompiler); - D3DCompile_ = (pD3DCompile)GetProcAddress(d3dCompiler, "D3DCompile"); - assert(D3DCompile_); - } + D3DCompile_ = (pD3DCompile)GetProcAddress(d3dCompiler, "D3DCompile"); + assert(D3DCompile_); + } - // The HLSL compiler will try to "canonicalize" our input file path, - // and we don't want it to do that, because they it won't report - // the same locations on error messages that we would. - // - // To work around that, we prepend a custom `#line` directive. + // The HLSL compiler will try to "canonicalize" our input file path, + // and we don't want it to do that, because they it won't report + // the same locations on error messages that we would. + // + // To work around that, we prepend a custom `#line` directive. - String rawHlslCode = EmitHLSL(context); + String rawHlslCode = EmitHLSL(context); - StringBuilder hlslCodeBuilder; - hlslCodeBuilder << "#line 1 \""; - for(auto c : context.sourcePath) + StringBuilder hlslCodeBuilder; + hlslCodeBuilder << "#line 1 \""; + for(auto c : context.sourcePath) + { + char buffer[] = { c, 0 }; + switch(c) { - char buffer[] = { c, 0 }; - switch(c) - { - default: - hlslCodeBuilder << buffer; - break; + default: + hlslCodeBuilder << buffer; + break; - case '\\': - hlslCodeBuilder << "\\\\"; - } + case '\\': + hlslCodeBuilder << "\\\\"; } - hlslCodeBuilder << "\"\n"; - hlslCodeBuilder << rawHlslCode; - - auto hlslCode = hlslCodeBuilder.ProduceString(); - - ID3DBlob* codeBlob; - ID3DBlob* diagnosticsBlob; - HRESULT hr = D3DCompile_( - hlslCode.begin(), - hlslCode.Length(), - context.sourcePath.begin(), - nullptr, - nullptr, - entryPoint.name.begin(), - GetHLSLProfileName(entryPoint.profile), - 0, - 0, - &codeBlob, - &diagnosticsBlob); - List<uint8_t> data; - if (codeBlob) - { - data.AddRange((uint8_t const*)codeBlob->GetBufferPointer(), (int)codeBlob->GetBufferSize()); - codeBlob->Release(); - } - if (diagnosticsBlob) + } + hlslCodeBuilder << "\"\n"; + hlslCodeBuilder << rawHlslCode; + + auto hlslCode = hlslCodeBuilder.ProduceString(); + + ID3DBlob* codeBlob; + ID3DBlob* diagnosticsBlob; + HRESULT hr = D3DCompile_( + hlslCode.begin(), + hlslCode.Length(), + context.sourcePath.begin(), + nullptr, + nullptr, + entryPoint.name.begin(), + GetHLSLProfileName(entryPoint.profile), + 0, + 0, + &codeBlob, + &diagnosticsBlob); + List<uint8_t> data; + if (codeBlob) + { + data.AddRange((uint8_t const*)codeBlob->GetBufferPointer(), (int)codeBlob->GetBufferSize()); + codeBlob->Release(); + } + if (diagnosticsBlob) + { + // TODO(tfoley): need a better policy for how we translate diagnostics + // back into the Slang world (although we should always try to generate + // HLSL that doesn't produce any diagnostics...) + String diagnostics = (char const*) diagnosticsBlob->GetBufferPointer(); + fprintf(stderr, "%s", diagnostics.begin()); + OutputDebugStringA(diagnostics.begin()); + diagnosticsBlob->Release(); + } + if (FAILED(hr)) + { + // TODO(tfoley): What to do on failure? + } + return data; + } + + List<uint8_t> EmitDXBytecode( + ExtraContext& context) + { + if(context.getTranslationUnitOptions().entryPoints.Count() != 1) + { + if(context.getTranslationUnitOptions().entryPoints.Count() == 0) { - // TODO(tfoley): need a better policy for how we translate diagnostics - // back into the Slang world (although we should always try to generate - // HLSL that doesn't produce any diagnostics...) - String diagnostics = (char const*) diagnosticsBlob->GetBufferPointer(); - fprintf(stderr, "%s", diagnostics.begin()); - OutputDebugStringA(diagnostics.begin()); - diagnosticsBlob->Release(); + // TODO(tfoley): need to write diagnostics into this whole thing... + fprintf(stderr, "no entry point specified\n"); } - if (FAILED(hr)) + else { - // TODO(tfoley): What to do on failure? + fprintf(stderr, "multiple entry points specified\n"); } - return data; + return List<uint8_t>(); } - List<uint8_t> EmitDXBytecode( - ExtraContext& context) + return EmitDXBytecodeForEntryPoint(context, context.getTranslationUnitOptions().entryPoints[0]); + } + + String EmitDXBytecodeAssemblyForEntryPoint( + ExtraContext& context, + EntryPointOption const& entryPoint) + { + static pD3DDisassemble D3DDisassemble_ = nullptr; + if (!D3DDisassemble_) { - if(context.getTranslationUnitOptions().entryPoints.Count() != 1) - { - if(context.getTranslationUnitOptions().entryPoints.Count() == 0) - { - // TODO(tfoley): need to write diagnostics into this whole thing... - fprintf(stderr, "no entry point specified\n"); - } - else - { - fprintf(stderr, "multiple entry points specified\n"); - } - return List<uint8_t>(); - } + HMODULE d3dCompiler = (HMODULE)GetD3DCompilerDLL(); + assert(d3dCompiler); - return EmitDXBytecodeForEntryPoint(context, context.getTranslationUnitOptions().entryPoints[0]); + D3DDisassemble_ = (pD3DDisassemble)GetProcAddress(d3dCompiler, "D3DDisassemble"); + assert(D3DDisassemble_); } - String EmitDXBytecodeAssemblyForEntryPoint( - ExtraContext& context, - EntryPointOption const& entryPoint) + List<uint8_t> dxbc = EmitDXBytecodeForEntryPoint(context, entryPoint); + if (!dxbc.Count()) { - static pD3DDisassemble D3DDisassemble_ = nullptr; - if (!D3DDisassemble_) - { - HMODULE d3dCompiler = (HMODULE)GetD3DCompilerDLL(); - assert(d3dCompiler); - - D3DDisassemble_ = (pD3DDisassemble)GetProcAddress(d3dCompiler, "D3DDisassemble"); - assert(D3DDisassemble_); - } - - List<uint8_t> dxbc = EmitDXBytecodeForEntryPoint(context, entryPoint); - if (!dxbc.Count()) - { - return ""; - } + return ""; + } - ID3DBlob* codeBlob; - HRESULT hr = D3DDisassemble_( - &dxbc[0], - dxbc.Count(), - 0, - nullptr, - &codeBlob); + ID3DBlob* codeBlob; + HRESULT hr = D3DDisassemble_( + &dxbc[0], + dxbc.Count(), + 0, + nullptr, + &codeBlob); - String result; - if (codeBlob) - { - result = String((char const*) codeBlob->GetBufferPointer()); - codeBlob->Release(); - } - if (FAILED(hr)) - { - // TODO(tfoley): need to figure out what to diagnose here... - } - return result; + String result; + if (codeBlob) + { + result = String((char const*) codeBlob->GetBufferPointer()); + codeBlob->Release(); + } + if (FAILED(hr)) + { + // TODO(tfoley): need to figure out what to diagnose here... } + return result; + } - String EmitDXBytecodeAssembly( - ExtraContext& context) + String EmitDXBytecodeAssembly( + ExtraContext& context) + { + if(context.getTranslationUnitOptions().entryPoints.Count() == 0) { - if(context.getTranslationUnitOptions().entryPoints.Count() == 0) - { - // TODO(tfoley): need to write diagnostics into this whole thing... - fprintf(stderr, "no entry point specified\n"); - return ""; - } - - StringBuilder sb; - for (auto entryPoint : context.getTranslationUnitOptions().entryPoints) - { - sb << EmitDXBytecodeAssemblyForEntryPoint(context, entryPoint); - } - return sb.ProduceString(); + // TODO(tfoley): need to write diagnostics into this whole thing... + fprintf(stderr, "no entry point specified\n"); + return ""; } - - HMODULE getGLSLCompilerDLL() + StringBuilder sb; + for (auto entryPoint : context.getTranslationUnitOptions().entryPoints) { - // TODO(tfoley): let user specify version of glslang DLL to use. - static HMODULE glslCompiler = LoadLibraryA("glslang"); - // TODO(tfoley): handle case where we can't find it gracefully - assert(glslCompiler); - return glslCompiler; + sb << EmitDXBytecodeAssemblyForEntryPoint(context, entryPoint); } + return sb.ProduceString(); + } - String emitSPIRVAssemblyForEntryPoint( - ExtraContext& context, - EntryPointOption const& entryPoint) - { - String rawGLSL = emitGLSLForEntryPoint(context, entryPoint); + HMODULE getGLSLCompilerDLL() + { + // TODO(tfoley): let user specify version of glslang DLL to use. + static HMODULE glslCompiler = LoadLibraryA("glslang"); + // TODO(tfoley): handle case where we can't find it gracefully + assert(glslCompiler); + return glslCompiler; + } - static glslang_CompileFunc glslang_compile = nullptr; - if (!glslang_compile) - { - HMODULE glslCompiler = getGLSLCompilerDLL(); - assert(glslCompiler); - glslang_compile = (glslang_CompileFunc)GetProcAddress(glslCompiler, "glslang_compile"); - assert(glslang_compile); - } + String emitSPIRVAssemblyForEntryPoint( + ExtraContext& context, + EntryPointOption const& entryPoint) + { + String rawGLSL = emitGLSLForEntryPoint(context, entryPoint); - StringBuilder diagnosticBuilder; - StringBuilder outputBuilder; + static glslang_CompileFunc glslang_compile = nullptr; + if (!glslang_compile) + { + HMODULE glslCompiler = getGLSLCompilerDLL(); + assert(glslCompiler); - auto outputFunc = [](char const* text, void* userData) - { - *(StringBuilder*)userData << text; - }; + glslang_compile = (glslang_CompileFunc)GetProcAddress(glslCompiler, "glslang_compile"); + assert(glslang_compile); + } - glslang_CompileRequest request; - request.sourcePath = context.sourcePath.begin(); - request.sourceText = rawGLSL.begin(); - request.slangStage = (SlangStage) entryPoint.profile.GetStage(); + StringBuilder diagnosticBuilder; + StringBuilder outputBuilder; - request.diagnosticFunc = outputFunc; - request.diagnosticUserData = &diagnosticBuilder; + auto outputFunc = [](char const* text, void* userData) + { + *(StringBuilder*)userData << text; + }; - request.outputFunc = outputFunc; - request.outputUserData = &outputBuilder; + glslang_CompileRequest request; + request.sourcePath = context.sourcePath.begin(); + request.sourceText = rawGLSL.begin(); + request.slangStage = (SlangStage) entryPoint.profile.GetStage(); - int err = glslang_compile(&request); + request.diagnosticFunc = outputFunc; + request.diagnosticUserData = &diagnosticBuilder; - String diagnostics = diagnosticBuilder.ProduceString(); - String output = outputBuilder.ProduceString(); + request.outputFunc = outputFunc; + request.outputUserData = &outputBuilder; - if(err) - { - OutputDebugStringA(diagnostics.Buffer()); - fprintf(stderr, "%s", diagnostics.Buffer()); - exit(1); - } + int err = glslang_compile(&request); - return output; + String diagnostics = diagnosticBuilder.ProduceString(); + String output = outputBuilder.ProduceString(); + + if(err) + { + OutputDebugStringA(diagnostics.Buffer()); + fprintf(stderr, "%s", diagnostics.Buffer()); + exit(1); } + + return output; + } #endif - String emitSPIRVAssembly( - ExtraContext& context) + String emitSPIRVAssembly( + ExtraContext& context) + { + if(context.getTranslationUnitOptions().entryPoints.Count() == 0) { - if(context.getTranslationUnitOptions().entryPoints.Count() == 0) - { - // TODO(tfoley): need to write diagnostics into this whole thing... - fprintf(stderr, "no entry point specified\n"); - return ""; - } - - StringBuilder sb; - for (auto entryPoint : context.getTranslationUnitOptions().entryPoints) - { - sb << emitSPIRVAssemblyForEntryPoint(context, entryPoint); - } - return sb.ProduceString(); + // TODO(tfoley): need to write diagnostics into this whole thing... + fprintf(stderr, "no entry point specified\n"); + return ""; } - // Do emit logic for a single entry point - EntryPointResult emitEntryPoint(ExtraContext& context, EntryPointOption& entryPoint) + StringBuilder sb; + for (auto entryPoint : context.getTranslationUnitOptions().entryPoints) { - EntryPointResult result; + sb << emitSPIRVAssemblyForEntryPoint(context, entryPoint); + } + return sb.ProduceString(); + } - switch (context.getOptions().Target) + // Do emit logic for a single entry point + EntryPointResult emitEntryPoint(ExtraContext& context, EntryPointOption& entryPoint) + { + EntryPointResult result; + + switch (context.getOptions().Target) + { + case CodeGenTarget::GLSL: { - case CodeGenTarget::GLSL: - { - String code = emitGLSLForEntryPoint(context, entryPoint); - result.outputSource = code; - } - break; + String code = emitGLSLForEntryPoint(context, entryPoint); + result.outputSource = code; + } + break; - case CodeGenTarget::DXBytecode: - { - auto code = EmitDXBytecodeForEntryPoint(context, entryPoint); + case CodeGenTarget::DXBytecode: + { + auto code = EmitDXBytecodeForEntryPoint(context, entryPoint); - // TODO(tfoley): Need to figure out an appropriate interface - // for returning binary code, in addition to source. + // TODO(tfoley): Need to figure out an appropriate interface + // for returning binary code, in addition to source. #if 0 - if (context.compileResult) - { - StringBuilder sb; - sb.Append((char*) code.begin(), code.Count()); + if (context.compileResult) + { + StringBuilder sb; + sb.Append((char*) code.begin(), code.Count()); - String codeString = sb.ProduceString(); - result.outputSource = codeString; - } - else + String codeString = sb.ProduceString(); + result.outputSource = codeString; + } + else #endif + { + int col = 0; + for(auto ii : code) { - int col = 0; - for(auto ii : code) - { - if(col != 0) fputs(" ", stdout); - fprintf(stdout, "%02X", ii); - col++; - if(col == 8) - { - fputs("\n", stdout); - col = 0; - } - } - if(col != 0) + if(col != 0) fputs(" ", stdout); + fprintf(stdout, "%02X", ii); + col++; + if(col == 8) { fputs("\n", stdout); + col = 0; } } - return result; - } - break; - - case CodeGenTarget::DXBytecodeAssembly: - { - String code = EmitDXBytecodeAssemblyForEntryPoint(context, entryPoint); - result.outputSource = code; - } - break; - - case CodeGenTarget::SPIRVAssembly: - { - String code = emitSPIRVAssemblyForEntryPoint(context, entryPoint); - result.outputSource = code; + if(col != 0) + { + fputs("\n", stdout); + } } - break; - - // Note(tfoley): We currently hit this case when compiling the stdlib - case CodeGenTarget::Unknown: - break; - - default: - throw "unimplemented"; + return result; } + break; - return result; - - - } - - TranslationUnitResult emitTranslationUnitEntryPoints(ExtraContext& context) - { - TranslationUnitResult result; - - for (auto& entryPoint : context.getTranslationUnitOptions().entryPoints) + case CodeGenTarget::DXBytecodeAssembly: { - EntryPointResult entryPointResult = emitEntryPoint(context, entryPoint); - - result.entryPoints.Add(entryPointResult); + String code = EmitDXBytecodeAssemblyForEntryPoint(context, entryPoint); + result.outputSource = code; } + break; - // The result for the translation unit will just be the concatenation - // of the results for each entry point. This doesn't actually make - // much sense, but it is good enough for now. - StringBuilder sb; - for (auto& entryPointResult : result.entryPoints) + case CodeGenTarget::SPIRVAssembly: { - sb << entryPointResult.outputSource; + String code = emitSPIRVAssemblyForEntryPoint(context, entryPoint); + result.outputSource = code; } + break; - result.outputSource = sb.ProduceString(); + // Note(tfoley): We currently hit this case when compiling the stdlib + case CodeGenTarget::Unknown: + break; - return result; + default: + throw "unimplemented"; } - // Do emit logic for an entire translation unit, which might - // have zero or more entry points - TranslationUnitResult emitTranslationUnit(ExtraContext& context) - { - // Most of our code generation targets will require us - // to proceed through one entry point at a time, but - // in some cases we can emit an entire translation unit - // in one go. + return result; - switch (context.getOptions().Target) - { - default: - // The default behavior is going to loop over all the entry - // points, and then collect an aggregate result. - return emitTranslationUnitEntryPoints(context); - case CodeGenTarget::HLSL: - // When targetting HLSL, we can emit the entire translation unit - // as a single HLSL program, and include all the entry points. - { - - String hlsl = EmitHLSL(context); - - TranslationUnitResult result; - result.outputSource = hlsl; + } - // Because the user might ask for per-entry-point source, - // we will just attach the same string as the result for - // each entry point. - for( auto& entryPoint : context.getTranslationUnitOptions().entryPoints ) - { - (void)entryPoint; + TranslationUnitResult emitTranslationUnitEntryPoints(ExtraContext& context) + { + TranslationUnitResult result; - EntryPointResult entryPointResult; - entryPointResult.outputSource = hlsl; - result.entryPoints.Add(entryPointResult); - } + for (auto& entryPoint : context.getTranslationUnitOptions().entryPoints) + { + EntryPointResult entryPointResult = emitEntryPoint(context, entryPoint); - return result; - } - break; - } + result.entryPoints.Add(entryPointResult); } - TranslationUnitResult DoNewEmitLogic(ExtraContext& context) + // The result for the translation unit will just be the concatenation + // of the results for each entry point. This doesn't actually make + // much sense, but it is good enough for now. + StringBuilder sb; + for (auto& entryPointResult : result.entryPoints) { - TranslationUnitResult result = emitTranslationUnit(context); - - // As a bit of a hack, we include a mode where we just - // print things to standard output, so that we can see them - // - // TODO(tfoley): Is this path ever needed/used? - if( !context.compileResult ) - { - fprintf(stdout, "%s", result.outputSource.Buffer()); - } - - return result; + sb << entryPointResult.outputSource; } - void DoNewEmitLogic( - ExtraContext& context, - CollectionOfTranslationUnits* collectionOfTranslationUnits) - { - switch (context.getOptions().Target) - { - default: - // For most targets, we will do things per-translation-unit - for( auto translationUnit : collectionOfTranslationUnits->translationUnits ) - { - ExtraContext innerContext = context; - innerContext.translationUnitOptions = &translationUnit.options; - innerContext.programSyntax = translationUnit.SyntaxNode; - innerContext.sourcePath = "slang"; // don't have this any more! - innerContext.sourceText = ""; - - TranslationUnitResult translationUnitResult = DoNewEmitLogic(innerContext); - context.compileResult->translationUnits.Add(translationUnitResult); - } - break; + result.outputSource = sb.ProduceString(); - case CodeGenTarget::ReflectionJSON: - { - String reflectionJSON = emitReflectionJSON(context.programLayout); + return result; + } - // HACK(tfoley): just print it out since that is what people probably expect. - // TODO: need a way to control where output gets routed across all possible targets. - fprintf(stdout, "%s", reflectionJSON.begin()); - } - break; - } - } + // Do emit logic for an entire translation unit, which might + // have zero or more entry points + TranslationUnitResult emitTranslationUnit(ExtraContext& context) + { + // Most of our code generation targets will require us + // to proceed through one entry point at a time, but + // in some cases we can emit an entire translation unit + // in one go. - virtual void Compile( - CompileResult& result, - CollectionOfTranslationUnits* collectionOfTranslationUnits, - const CompileOptions& options) override + switch (context.getOptions().Target) { - RefPtr<SyntaxVisitor> visitor = CreateSemanticsVisitor(result.GetErrorWriter(), options); - try + default: + // The default behavior is going to loop over all the entry + // points, and then collect an aggregate result. + return emitTranslationUnitEntryPoints(context); + + case CodeGenTarget::HLSL: + // When targetting HLSL, we can emit the entire translation unit + // as a single HLSL program, and include all the entry points. { - for( auto& translationUnit : collectionOfTranslationUnits->translationUnits ) - { - visitor->setSourceLanguage(translationUnit.options.sourceLanguage); - translationUnit.SyntaxNode->Accept(visitor.Ptr()); - } - if (result.GetErrorCount() > 0) - return; + String hlsl = EmitHLSL(context); - // Do binding generation, and then reflection (globally) - // before we move on to any code-generation activites. - GenerateParameterBindings(collectionOfTranslationUnits); + TranslationUnitResult result; + result.outputSource = hlsl; + // Because the user might ask for per-entry-point source, + // we will just attach the same string as the result for + // each entry point. + for( auto& entryPoint : context.getTranslationUnitOptions().entryPoints ) + { + (void)entryPoint; - // HACK(tfoley): for right now I just want to pretty-print an AST - // into another language, so the whole compiler back-end is just - // getting in the way. - // - // I'm going to bypass it for now and see what I can do: + EntryPointResult entryPointResult; + entryPointResult.outputSource = hlsl; + result.entryPoints.Add(entryPointResult); + } - ExtraContext extra; - extra.options = &options; - extra.programLayout = collectionOfTranslationUnits->layout.Ptr(); - extra.compileResult = &result; - DoNewEmitLogic(extra, collectionOfTranslationUnits); - } - catch (int) - { + return result; } - catch (...) - { - throw; - } - return; + break; } + } - ShaderCompilerImpl() - { - if (compilerInstances == 0) - { - BasicExpressionType::Init(); - } - compilerInstances++; - } + TranslationUnitResult generateOutput(ExtraContext& context) + { + TranslationUnitResult result = emitTranslationUnit(context); + return result; + } - ~ShaderCompilerImpl() + void generateOutput( + ExtraContext& context, + CollectionOfTranslationUnits* collectionOfTranslationUnits) + { + switch (context.getOptions().Target) { - compilerInstances--; - if (compilerInstances == 0) + default: + // For most targets, we will do things per-translation-unit + for( auto translationUnit : collectionOfTranslationUnits->translationUnits ) { - BasicExpressionType::Finalize(); - SlangStdLib::Finalize(); + ExtraContext innerContext = context; + innerContext.translationUnitOptions = &translationUnit.options; + innerContext.programSyntax = translationUnit.SyntaxNode; + innerContext.sourcePath = "slang"; // don't have this any more! + innerContext.sourceText = ""; + + TranslationUnitResult translationUnitResult = generateOutput(innerContext); + context.compileResult->translationUnits.Add(translationUnitResult); } - } + break; - virtual TranslationUnitResult PassThrough( - String const& sourceText, - String const& sourcePath, - const CompileOptions & options, - TranslationUnitOptions const& translationUnitOptions) override - { - ExtraContext extra; - extra.options = &options; - extra.translationUnitOptions = &translationUnitOptions; - extra.sourcePath = sourcePath; - extra.sourceText = sourceText; + case CodeGenTarget::ReflectionJSON: + { + String reflectionJSON = emitReflectionJSON(context.programLayout); - return DoNewEmitLogic(extra); + // HACK(tfoley): just print it out since that is what people probably expect. + // TODO: need a way to control where output gets routed across all possible targets. + fprintf(stdout, "%s", reflectionJSON.begin()); + } + break; } + } - }; - - ShaderCompiler * CreateShaderCompiler() + TranslationUnitResult passThrough( + String const& sourceText, + String const& sourcePath, + const CompileOptions & options, + TranslationUnitOptions const& translationUnitOptions) { - return new ShaderCompilerImpl(); + ExtraContext extra; + extra.options = &options; + extra.translationUnitOptions = &translationUnitOptions; + extra.sourcePath = sourcePath; + extra.sourceText = sourceText; + + return generateOutput(extra); } } diff --git a/source/slang/compiler.h b/source/slang/compiler.h index f35d6603c..01db7874f 100644 --- a/source/slang/compiler.h +++ b/source/slang/compiler.h @@ -17,6 +17,7 @@ namespace Slang { class ILConstOperand; struct IncludeHandler; + struct CompileRequest; enum class CompilerMode { @@ -87,28 +88,31 @@ namespace Slang // The source file(s) that will be compiled to form this translation unit List<RefPtr<SourceFile> > sourceFiles; + + // Preprocessor definitions to use for this translation unit only + // (whereas the ones on `CompileOptions` will be shared) + Dictionary<String, String> preprocessorDefinitions; }; class CompileOptions { public: - CompilerMode Mode = CompilerMode::ProduceShader; + // What target language are we compiling to? CodeGenTarget Target = CodeGenTarget::Unknown; - StageTarget stage = StageTarget::Unknown; - EnumerableDictionary<String, String> BackendArguments; - String SymbolToCompile; - String outputName; - List<String> TemplateShaderArguments; + // Directories to search for `#include` files or `import`ed modules List<String> SearchDirectories; - Dictionary<String, String> PreprocessorDefinitions; + // Definitions to provide during preprocessing + Dictionary<String, String> preprocessorDefinitions; + + // Translation units we are being asked to compile List<TranslationUnitOptions> translationUnits; - // the code generation profile we've been asked to use + // The code generation profile we've been asked to use. Profile profile; - // should we just pass the input to another compiler? + // Should we just pass the input to another compiler? PassThroughMode passThrough = PassThroughMode::None; // Flags supplied through the API @@ -133,13 +137,34 @@ namespace Slang RefPtr<ProgramLayout> layout; }; + // Context information for code generation + struct ExtraContext + { + CompileOptions const* options = nullptr; + TranslationUnitOptions const* translationUnitOptions = nullptr; + + CompileResult* compileResult = nullptr; + + RefPtr<ProgramSyntaxNode> programSyntax; + ProgramLayout* programLayout; + + String sourceText; + String sourcePath; + + CompileOptions const& getOptions() { return *options; } + TranslationUnitOptions const& getTranslationUnitOptions() { return *translationUnitOptions; } + }; + +#if 0 + class ShaderCompiler : public CoreLib::Basic::Object { public: virtual void Compile( CompileResult& result, CollectionOfTranslationUnits* collectionOfTranslationUnits, - const CompileOptions& options) = 0; + const CompileOptions& options, + CompileRequest* request) = 0; virtual TranslationUnitResult PassThrough( String const& sourceText, @@ -150,6 +175,17 @@ namespace Slang }; ShaderCompiler * CreateShaderCompiler(); +#endif + + TranslationUnitResult passThrough( + String const& sourceText, + String const& sourcePath, + const CompileOptions & options, + TranslationUnitOptions const& translationUnitOptions); + + void generateOutput( + ExtraContext& context, + CollectionOfTranslationUnits* collectionOfTranslationUnits); } } diff --git a/source/slang/emit.cpp b/source/slang/emit.cpp index 6b488a68f..f11757594 100644 --- a/source/slang/emit.cpp +++ b/source/slang/emit.cpp @@ -25,6 +25,13 @@ struct EmitContext // A set of words reserved by the target Dictionary<String, String> reservedWords; + + // For GLSL output, we can't emit traidtional `#line` directives + // with a file path in them, so we maintain a map that associates + // each path with a unique integer, and then we output those + // instead. + Dictionary<String, int> mapGLSLSourcePathToID; + int glslSourceIDCount = 0; }; // @@ -267,7 +274,7 @@ static bool MaybeEmitParens(EmitContext* context, int outerPrec, int prec) // might have introduced, but which interfere with our ability // to use it effectively in the target language static RefPtr<ExpressionSyntaxNode> prepareLValueExpr( - EmitContext* context, + EmitContext* /*context*/, RefPtr<ExpressionSyntaxNode> expr) { for(;;) @@ -370,6 +377,77 @@ static void EmitUnaryAssignExpr( emitUnaryExprImpl(context, outerPrec, prec, preOp, postOp, expr, true); } +// Determine if a target intrinsic modifer is applicable to the target +// we are currently emitting code for. +static bool isTargetIntrinsicModifierApplicable( + EmitContext* context, + RefPtr<TargetIntrinsicModifier> modifier) +{ + auto const& targetToken = modifier->targetToken; + + // If no target name was specified, then the modifier implicitly + // applies to all targets. + if(targetToken.Type == TokenType::Unknown) + return true; + + // Otherwise, we need to check if the target name matches what + // we expect. + auto const& targetName = targetToken.Content; + + switch(context->target) + { + default: + assert(!"unexpected"); + return false; + + case CodeGenTarget::GLSL: return targetName == "glsl"; + case CodeGenTarget::HLSL: return targetName == "hlsl"; + } +} + +// Find an intrinsic modifier appropriate to the current compilation target. +// +// If there are multiple such modifiers, this should return the best one. +static RefPtr<TargetIntrinsicModifier> findTargetIntrinsicModifier( + EmitContext* context, + RefPtr<ModifiableSyntaxNode> syntax) +{ + RefPtr<TargetIntrinsicModifier> bestModifier; + for(auto m : syntax->GetModifiersOfType<TargetIntrinsicModifier>()) + { + if(!isTargetIntrinsicModifierApplicable(context, m)) + continue; + + // For now "better"-ness is defined as: a modifier + // with a specified target is better than one without + // (it is more specific) + if(!bestModifier || bestModifier->targetToken.Type == TokenType::Unknown) + { + bestModifier = m; + } + } + + return bestModifier; +} + +static String getStringOrIdentifierTokenValue( + Token const& token) +{ + switch(token.Type) + { + default: + assert(!"unexpected"); + return ""; + + case TokenType::Identifier: + return token.Content; + + case TokenType::StringLiterial: + return getStringLiteralTokenValue(token); + break; + } +} + static void emitCallExpr( EmitContext* context, RefPtr<InvokeExpressionSyntaxNode> callExpr, @@ -380,9 +458,9 @@ static void emitCallExpr( { auto funcDeclRef = funcDeclRefExpr->declRef; auto funcDecl = funcDeclRef.GetDecl(); - if (auto intrinsicModifier = funcDecl->FindModifier<IntrinsicModifier>()) + if (auto intrinsicOpModifier = funcDecl->FindModifier<IntrinsicOpModifier>()) { - switch (intrinsicModifier->op) + switch (intrinsicOpModifier->op) { #define CASE(NAME, OP) case IntrinsicOp::NAME: EmitBinExpr(context, outerPrec, kPrecedence_##NAME, #OP, callExpr); return CASE(Mul, *); @@ -473,7 +551,69 @@ static void emitCallExpr( default: break; } + } + else if(auto targetIntrinsicModifier = findTargetIntrinsicModifier(context, funcDecl)) + { + if(targetIntrinsicModifier->definitionToken.Type != TokenType::Unknown) + { + auto name = getStringOrIdentifierTokenValue(targetIntrinsicModifier->definitionToken); + + if(name.IndexOf('$') < 0) + { + // Simple case: it is just an ordinary name, so we call it like a builtin. + // + // TODO: this case could probably handle things like operators, for generality? + + emit(context, name); + Emit(context, "("); + int argCount = callExpr->Arguments.Count(); + for (int aa = 0; aa < argCount; ++aa) + { + if (aa != 0) Emit(context, ", "); + EmitExpr(context, callExpr->Arguments[aa]); + } + Emit(context, ")"); + return; + } + else + { + // General case: we are going to emit some more complex text. + int argCount = callExpr->Arguments.Count(); + + Emit(context, "("); + + char const* cursor = name.begin(); + char const* end = name.end(); + while(cursor != end) + { + char c = *cursor++; + if( c != '$' ) + { + // Not an escape sequence + emitRawTextSpan(context, &c, &c+1); + continue; + } + + assert(cursor != end); + + char d = *cursor++; + assert(('0' <= d) && (d <= '9')); + + int argIndex = d - '0'; + assert((0 <= argIndex) && (argIndex < argCount)); + Emit(context, "("); + EmitExpr(context, callExpr->Arguments[argIndex]); + Emit(context, ")"); + } + + Emit(context, ")"); + } + + return; + } + + // TODO: emit as approperiate for this target // We might be calling an intrinsic subscript operation, // and should desugar it accordingly @@ -1178,23 +1318,51 @@ static void EmitLoopAttributes(EmitContext* context, RefPtr<StatementSyntaxNode> } } -static void advanceToSourceLocation( +// Emit a `#line` directive to the output. +// Doesn't udpate state of source-location tracking. +static void emitLineDirective( EmitContext* context, CodePosition const& sourceLocation) { - // If we are currently emitting code at a source location with - // a differnet file or line, *or* if the source location is - // somehow later on the line than what we want to emit, - // then we need to emit a new `#line` directive. - if(sourceLocation.FileName != context->loc.FileName - || sourceLocation.Line != context->loc.Line - || sourceLocation.Col < context->loc.Col) + emitRawText(context, "\n#line "); + + char buffer[16]; + sprintf(buffer, "%d", sourceLocation.Line); + emitRawText(context, buffer); + + emitRawText(context, " "); + + if(context->target == CodeGenTarget::GLSL) { - emitRawText(context, "\n#line "); + auto path = sourceLocation.FileName; + + // GLSL doesn't support the traditional form of a `#line` directive without + // an extension. Rather than depend on that extension we will output + // a directive in the traditional GLSL fashion. + // + // TODO: Add some kind of configuration where we require the appropriate + // extension and then emit a traditional line directive. - char buffer[16]; - sprintf(buffer, "%d", sourceLocation.Line); + int id = 0; + if(!context->mapGLSLSourcePathToID.TryGetValue(path, id)) + { + id = context->glslSourceIDCount++; + context->mapGLSLSourcePathToID.Add(path, id); + } + + sprintf(buffer, "%d", id); emitRawText(context, buffer); + } + else + { + // The simple case is to emit the path for the current source + // location. We need to be a little bit careful with this, + // because the path might include backslash characters if we + // are on Windows, and we want to canonicalize those over + // to forward slashes. + // + // TODO: Canonicalization like this should be done centrally + // in a module that tracks source files. emitRawText(context, "\""); for(auto c : sourceLocation.FileName) @@ -1213,11 +1381,39 @@ static void advanceToSourceLocation( break; } } - emitRawText(context, "\"\n"); + emitRawText(context, "\""); + } + + emitRawText(context, "\n"); +} + +// Emit a `#line` directive to the output, and also +// ensure that source location tracking information +// is correct based on the directive we just output. +static void emitLineDirectiveAndUpdateSourceLocation( + EmitContext* context, + CodePosition const& sourceLocation) +{ + emitLineDirective(context, sourceLocation); - context->loc.FileName = sourceLocation.FileName; - context->loc.Line = sourceLocation.Line; - context->loc.Col = 1; + context->loc.FileName = sourceLocation.FileName; + context->loc.Line = sourceLocation.Line; + context->loc.Col = 1; +} + +static void advanceToSourceLocation( + EmitContext* context, + CodePosition const& sourceLocation) +{ + // If we are currently emitting code at a source location with + // a differnet file or line, *or* if the source location is + // somehow later on the line than what we want to emit, + // then we need to emit a new `#line` directive. + if(sourceLocation.FileName != context->loc.FileName + || sourceLocation.Line != context->loc.Line + || sourceLocation.Col < context->loc.Col) + { + emitLineDirectiveAndUpdateSourceLocation(context, sourceLocation); } // Now indent up to the appropriate column, so that error messages @@ -2279,6 +2475,24 @@ static void EmitDeclImpl(EmitContext* context, RefPtr<Decl> decl, RefPtr<VarLayo { return; } + else if( auto importDecl = decl.As<ImportDecl>()) + { + // When in "rewriter" mode, we need to emit the code of the imported + // module in-place at the `import` site. + + auto moduleDecl = importDecl->importedModuleDecl; + + // TODO: do we need to modify the code generation environment at + // all when doing this recursive emit? + // + // TODO: what if we import the same module along two different + // paths? Probably need logic to avoid emitting the same + // module more than once. + + EmitDeclsInContainer(context, moduleDecl); + + return; + } else if( auto emptyDecl = decl.As<EmptyDecl>() ) { EmitModifiers(context, emptyDecl); diff --git a/source/slang/parser.cpp b/source/slang/parser.cpp index 773e3b74b..edd77c9ae 100644 --- a/source/slang/parser.cpp +++ b/source/slang/parser.cpp @@ -616,26 +616,51 @@ namespace Slang #undef CASE - else if (AdvanceIf(parser, "__intrinsic")) + else if (AdvanceIf(parser, "__intrinsic_op")) { - auto modifier = new IntrinsicModifier(); + auto modifier = new IntrinsicOpModifier(); modifier->Position = loc; - if (AdvanceIf(parser, TokenType::LParent)) + parser->ReadToken(TokenType::LParent); + if (parser->LookAheadToken(TokenType::IntLiterial)) + { + modifier->op = (IntrinsicOp)StringToInt(parser->ReadToken().Content); + } + else { - if (parser->LookAheadToken(TokenType::IntLiterial)) + modifier->opToken = parser->ReadToken(TokenType::Identifier); + + modifier->op = findIntrinsicOp(modifier->opToken.Content.Buffer()); + + if (modifier->op == IntrinsicOp::Unknown) { - modifier->op = (IntrinsicOp)StringToInt(parser->ReadToken().Content); + parser->sink->diagnose(loc, Diagnostics::unimplemented, "unknown intrinsic op"); } - else - { - modifier->opToken = parser->ReadToken(TokenType::Identifier); + } + + parser->ReadToken(TokenType::RParent); - modifier->op = findIntrinsicOp(modifier->opToken.Content.Buffer()); + AddModifier(&modifierLink, modifier); + } + + else if (AdvanceIf(parser, "__intrinsic")) + { + auto modifier = new TargetIntrinsicModifier(); + modifier->Position = loc; - if (modifier->op == IntrinsicOp::Unknown) + if (AdvanceIf(parser, TokenType::LParent)) + { + modifier->targetToken = parser->ReadToken(TokenType::Identifier); + + if( AdvanceIf(parser, TokenType::Comma) ) + { + if( parser->LookAheadToken(TokenType::StringLiterial) ) { - parser->sink->diagnose(loc, Diagnostics::unimplemented, "unknown intrinsic op"); + modifier->definitionToken = parser->ReadToken(); + } + else + { + modifier->definitionToken = parser->ReadToken(TokenType::Identifier); } } @@ -760,21 +785,18 @@ namespace Slang } } - static RefPtr<Decl> ParseUsing( + static RefPtr<Decl> parseImportDecl( Parser* parser) { - parser->ReadToken("using"); - if (parser->tokenReader.PeekTokenType() == TokenType::StringLiterial) - { - auto usingDecl = new UsingFileDecl(); - usingDecl->fileName = parser->ReadToken(TokenType::StringLiterial); - parser->ReadToken(TokenType::Semicolon); - return usingDecl; - } - else - { - unexpected(); - } + parser->ReadToken("__import"); + + auto decl = new ImportDecl(); + decl->nameToken = parser->ReadToken(TokenType::Identifier); + decl->scope = parser->currentScope; + + parser->ReadToken(TokenType::Semicolon); + + return decl; } static Token ParseDeclName( @@ -2123,8 +2145,6 @@ parser->ReadToken(TokenType::Comma); decl = ParseDeclaratorDecl(parser, containerDecl); else if (parser->LookAheadToken("typedef")) decl = ParseTypeDef(parser); - else if (parser->LookAheadToken("using")) - decl = ParseUsing(parser); else if (parser->LookAheadToken("cbuffer") || parser->LookAheadToken("tbuffer")) decl = ParseHLSLBufferDecl(parser); else if (parser->LookAheadToken("__generic")) @@ -2141,6 +2161,8 @@ parser->ReadToken(TokenType::Comma); decl = ParseTraitDecl(parser); else if(parser->LookAheadToken("__modifier")) decl = parseModifierDecl(parser); + else if(parser->LookAheadToken("__import")) + decl = parseImportDecl(parser); else if (AdvanceIf(parser, TokenType::Semicolon)) { decl = new EmptyDecl(); diff --git a/source/slang/slang-stdlib.cpp b/source/slang/slang-stdlib.cpp index f74fcd603..494a32e4d 100644 --- a/source/slang/slang-stdlib.cpp +++ b/source/slang/slang-stdlib.cpp @@ -8,16 +8,16 @@ #define LINE_STRING STRINGIZE(__LINE__) enum { kLibIncludeStringLine = __LINE__+1 }; -const char * LibIncludeStringChunks[] = { R"( +const char * LibIncludeStringChunks[] = { R"=( typedef uint UINT; -__generic<T> __intrinsic(Assign) T operator=(out T left, T right); +__generic<T> __intrinsic_op(Assign) T operator=(out T left, T right); -__generic<T,U> __intrinsic(Sequence) U operator,(T left, U right); +__generic<T,U> __intrinsic_op(Sequence) U operator,(T left, U right); -__generic<T> __intrinsic(Select) T operator?:(bool condition, T ifTrue, T ifFalse); -__generic<T, let N : int> __intrinsic(Select) vector<T,N> operator?:(vector<bool,N> condition, vector<T,N> ifTrue, vector<T,N> ifFalse); +__generic<T> __intrinsic_op(Select) T operator?:(bool condition, T ifTrue, T ifFalse); +__generic<T, let N : int> __intrinsic_op(Select) vector<T,N> operator?:(vector<bool,N> condition, vector<T,N> ifTrue, vector<T,N> ifFalse); __generic<T> __magic_type(HLSLAppendStructuredBufferType) struct AppendStructuredBuffer { @@ -234,7 +234,7 @@ __generic<T> __magic_type(HLSLPointStreamType) struct PointStream {}; __generic<T> __magic_type(HLSLLineStreamType) struct LineStream {}; __generic<T> __magic_type(HLSLLineStreamType) struct TriangleStream {}; -)", R"( +)=", R"=( // Note(tfoley): Trying to systematically add all the HLSL builtins @@ -503,7 +503,7 @@ __generic<T : __BuiltinFloatingPointType> __intrinsic T fwidth(T x); __generic<T : __BuiltinFloatingPointType, let N : int> __intrinsic vector<T,N> fwidth(vector<T,N> x); __generic<T : __BuiltinFloatingPointType, let N : int, let M : int> __intrinsic matrix<T,N,M> fwidth(matrix<T,N,M> x); -)", R"( +)=", R"=( // Get number of samples in render target __intrinsic uint GetRenderTargetSampleCount(); @@ -615,27 +615,27 @@ __intrinsic uint4 msad4(uint reference, uint2 source, uint4 accum); // General inner products // scalar-scalar -__generic<T : __BuiltinArithmeticType> __intrinsic(Mul_Scalar_Scalar) T mul(T x, T y); +__generic<T : __BuiltinArithmeticType> __intrinsic_op(Mul_Scalar_Scalar) T mul(T x, T y); // scalar-vector and vector-scalar -__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic(Mul_Vector_Scalar) vector<T,N> mul(vector<T,N> x, T y); -__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic(Mul_Scalar_Vector) vector<T,N> mul(T x, vector<T,N> y); +__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic_op(Mul_Vector_Scalar) vector<T,N> mul(vector<T,N> x, T y); +__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic_op(Mul_Scalar_Vector) vector<T,N> mul(T x, vector<T,N> y); // scalar-matrix and matrix-scalar -__generic<T : __BuiltinArithmeticType, let N : int, let M :int> __intrinsic(Mul_Matrix_Scalar) matrix<T,N,M> mul(matrix<T,N,M> x, T y); -__generic<T : __BuiltinArithmeticType, let N : int, let M :int> __intrinsic(Mul_Scalar_Matrix) matrix<T,N,M> mul(T x, matrix<T,N,M> y); +__generic<T : __BuiltinArithmeticType, let N : int, let M :int> __intrinsic_op(Mul_Matrix_Scalar) matrix<T,N,M> mul(matrix<T,N,M> x, T y); +__generic<T : __BuiltinArithmeticType, let N : int, let M :int> __intrinsic_op(Mul_Scalar_Matrix) matrix<T,N,M> mul(T x, matrix<T,N,M> y); // vector-vector (dot product) -__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic(InnerProduct_Vector_Vector) T mul(vector<T,N> x, vector<T,N> y); +__generic<T : __BuiltinArithmeticType, let N : int> __intrinsic_op(InnerProduct_Vector_Vector) T mul(vector<T,N> x, vector<T,N> y); // vector-matrix -__generic<T : __BuiltinArithmeticType, let N : int, let M : int> __intrinsic(InnerProduct_Vector_Matrix) vector<T,M> mul(vector<T,N> x, matrix<T,N,M> y); +__generic<T : __BuiltinArithmeticType, let N : int, let M : int> __intrinsic_op(InnerProduct_Vector_Matrix) vector<T,M> mul(vector<T,N> x, matrix<T,N,M> y); // matrix-vector -__generic<T : __BuiltinArithmeticType, let N : int, let M : int> __intrinsic(InnerProduct_Matrix_Vector) vector<T,N> mul(matrix<T,N,M> x, vector<T,M> y); +__generic<T : __BuiltinArithmeticType, let N : int, let M : int> __intrinsic_op(InnerProduct_Matrix_Vector) vector<T,N> mul(matrix<T,N,M> x, vector<T,M> y); // matrix-matrix -__generic<T : __BuiltinArithmeticType, let R : int, let N : int, let C : int> __intrinsic(InnerProduct_Matrix_Matrix) matrix<T,R,C> mul(matrix<T,R,N> x, matrix<T,N,C> y); +__generic<T : __BuiltinArithmeticType, let R : int, let N : int, let C : int> __intrinsic_op(InnerProduct_Matrix_Matrix) matrix<T,R,C> mul(matrix<T,R,N> x, matrix<T,N,C> y); // noise (deprecated) __intrinsic float noise(float x); @@ -759,9 +759,17 @@ __generic<T : __BuiltinFloatingPointType, let N : int> __intrinsic vector<T,N> r __generic<T : __BuiltinFloatingPointType, let N : int, let M : int> __intrinsic matrix<T,N,M> rsqrt(matrix<T,N,M> x); // Clamp value to [0,1] range -__generic<T : __BuiltinFloatingPointType> __intrinsic T saturate(T x); -__generic<T : __BuiltinFloatingPointType, let N : int> __intrinsic vector<T,N> saturate(vector<T,N> x); -__generic<T : __BuiltinFloatingPointType, let N : int, let M : int> __intrinsic matrix<T,N,M> saturate(matrix<T,N,M> x); +__generic<T : __BuiltinFloatingPointType> +__intrinsic(glsl, "clamp($0, 0, 1)") __intrinsic +T saturate(T x); + +__generic<T : __BuiltinFloatingPointType, let N : int> +__intrinsic(glsl, "clamp($0, 0, 1)") __intrinsic +vector<T,N> saturate(vector<T,N> x); + +__generic<T : __BuiltinFloatingPointType, let N : int, let M : int> +__intrinsic(glsl, "clamp($0, 0, 1)") __intrinsic +matrix<T,N,M> saturate(matrix<T,N,M> x); // Extract sign of value @@ -769,7 +777,7 @@ __generic<T : __BuiltinSignedArithmeticType> __intrinsic int sign(T x); __generic<T : __BuiltinSignedArithmeticType, let N : int> __intrinsic vector<int,N> sign(vector<T,N> x); __generic<T : __BuiltinSignedArithmeticType, let N : int, let M : int> __intrinsic matrix<int,N,M> sign(matrix<T,N,M> x); -)", R"( +)=", R"=( // Sine @@ -853,7 +861,7 @@ __generic<T : __BuiltinFloatingPointType, let N : int> __intrinsic vector<T,N> t __generic<T : __BuiltinFloatingPointType, let N : int, let M : int> __intrinsic matrix<T,N,M> trunc(matrix<T,N,M> x); -)", R"( +)=", R"=( // Shader model 6.0 stuff @@ -930,13 +938,13 @@ __generic<T : __BuiltinType, let N : int> __intrinsic vector<T,N> WaveReadLaneAt __generic<T : __BuiltinType, let N : int, let M : int> __intrinsic matrix<T,N,M> WaveReadLaneAt(matrix<T,N,M> expr, int laneIndex); -)", R"( +)=", R"=( // `typedef`s to help with the fact that HLSL has been sorta-kinda case insensitive at various points typedef Texture2D texture2D; #line default -)" }; +)=" }; using namespace CoreLib::Basic; @@ -1522,19 +1530,6 @@ namespace Slang sb << "__intrinsic mat4 operator * (mat4, mat4);\n"; #endif -#if 0 - sb << "__intrinsic(And) bool operator && (bool, bool);\n"; - sb << "__intrinsic(Or) bool operator || (bool, bool);\n"; - - for (auto type : intTypes) - { - sb << "__intrinsic(And) bool operator && (bool, " << type << ");\n"; - sb << "__intrinsic(Or) bool operator || (bool, " << type << ");\n"; - sb << "__intrinsic(And) bool operator && (" << type << ", bool);\n"; - sb << "__intrinsic(Or) bool operator || (" << type << ", bool);\n"; - } -#endif - for (auto op : unaryOps) { for (auto type : kBaseTypes) @@ -1547,17 +1542,17 @@ namespace Slang // scalar version sb << fixity; - sb << "__intrinsic(" << int(op.opCode) << ") " << type.name << " operator" << op.opName << "(" << qual << type.name << " value);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") " << type.name << " operator" << op.opName << "(" << qual << type.name << " value);\n"; // vector version sb << "__generic<let N : int> "; sb << fixity; - sb << "__intrinsic(" << int(op.opCode) << ") vector<" << type.name << ",N> operator" << op.opName << "(" << qual << "vector<" << type.name << ",N> value);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << type.name << ",N> operator" << op.opName << "(" << qual << "vector<" << type.name << ",N> value);\n"; // matrix version sb << "__generic<let N : int, let M : int> "; sb << fixity; - sb << "__intrinsic(" << int(op.opCode) << ") matrix<" << type.name << ",N,M> operator" << op.opName << "(" << qual << "matrix<" << type.name << ",N,M> value);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << type.name << ",N,M> operator" << op.opName << "(" << qual << "matrix<" << type.name << ",N,M> value);\n"; } } @@ -1580,15 +1575,15 @@ namespace Slang // TODO: handle `SHIFT` // scalar version - sb << "__intrinsic(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << leftQual << leftType << " left, " << rightType << " right);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << leftQual << leftType << " left, " << rightType << " right);\n"; // vector version sb << "__generic<let N : int> "; - sb << "__intrinsic(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << leftQual << "vector<" << leftType << ",N> left, vector<" << rightType << ",N> right);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << leftQual << "vector<" << leftType << ",N> left, vector<" << rightType << ",N> right);\n"; // matrix version sb << "__generic<let N : int, let M : int> "; - sb << "__intrinsic(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, matrix<" << rightType << ",N,M> right);\n"; + sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, matrix<" << rightType << ",N,M> right);\n"; } } diff --git a/source/slang/slang.cpp b/source/slang/slang.cpp index 2f37981c5..1de94bb14 100644 --- a/source/slang/slang.cpp +++ b/source/slang/slang.cpp @@ -2,9 +2,11 @@ #include "../core/slang-io.h" #include "../slang/slang-stdlib.h" +#include "parameter-binding.h" #include "../slang/parser.h" #include "../slang/preprocessor.h" #include "../slang/reflection.h" +#include "syntax-visitors.h" #include "../slang/type-layout.h" #ifdef _WIN32 @@ -19,128 +21,142 @@ using namespace CoreLib::Basic; using namespace CoreLib::IO; using namespace Slang::Compiler; -namespace SlangLib +namespace Slang { +namespace Compiler { + +static void stdlibDiagnosticCallback( + char const* message, + void* userData) { - static void stdlibDiagnosticCallback( - char const* message, - void* userData) - { - fputs(message, stderr); - fflush(stderr); + fputs(message, stderr); + fflush(stderr); #ifdef WIN32 - OutputDebugStringA(message); + OutputDebugStringA(message); #endif - } +} - class Session - { - public: - bool useCache = false; - CoreLib::String cacheDir; +class Session +{ +public: + bool useCache = false; + CoreLib::String cacheDir; - RefPtr<ShaderCompiler> compiler; + RefPtr<Scope> slangLanguageScope; + RefPtr<Scope> hlslLanguageScope; + RefPtr<Scope> glslLanguageScope; - RefPtr<Scope> slangLanguageScope; - RefPtr<Scope> hlslLanguageScope; - RefPtr<Scope> glslLanguageScope; + List<RefPtr<ProgramSyntaxNode>> loadedModuleCode; - List<RefPtr<ProgramSyntaxNode>> loadedModuleCode; + Session(bool /*pUseCache*/, CoreLib::String /*pCacheDir*/) + { + // Initialize global state + // TODO: move this into the session instead + BasicExpressionType::Init(); - Session(bool /*pUseCache*/, CoreLib::String /*pCacheDir*/) - { - compiler = CreateShaderCompiler(); + // Create scopes for various language builtins. + // + // TODO: load these on-demand to avoid parsing + // stdlib code for languages the user won't use. - // Create scopes for various language builtins. - // - // TODO: load these on-demand to avoid parsing - // stdlib code for languages the user won't use. + slangLanguageScope = new Scope(); - slangLanguageScope = new Scope(); + hlslLanguageScope = new Scope(); + hlslLanguageScope->parent = slangLanguageScope; - hlslLanguageScope = new Scope(); - hlslLanguageScope->parent = slangLanguageScope; + glslLanguageScope = new Scope(); + glslLanguageScope->parent = slangLanguageScope; - glslLanguageScope = new Scope(); - glslLanguageScope->parent = slangLanguageScope; + addBuiltinSource(slangLanguageScope, "stdlib", SlangStdLib::GetCode()); + addBuiltinSource(glslLanguageScope, "glsl", getGLSLLibraryCode()); + } - addBuiltinSource(slangLanguageScope, "stdlib", SlangStdLib::GetCode()); - addBuiltinSource(glslLanguageScope, "glsl", getGLSLLibraryCode()); - } + ~Session() + { + // We need to clean up the strings for the standard library + // code that we might have allocated and loaded into static + // variables (TODO: don't use `static` variables for this stuff) - ~Session() - { - // We need to clean up the strings for the standard library - // code that we might have allocated and loaded into static - // variables (TODO: don't use `static` variables for this stuff) + SlangStdLib::Finalize(); - SlangStdLib::Finalize(); + // Ditto for our type represnetation stuff - // Ditto for our type represnetation stuff + ExpressionType::Finalize(); + } - ExpressionType::Finalize(); - } + CompileUnit createPredefUnit() + { + CompileUnit translationUnit; - CompileUnit createPredefUnit() - { - CompileUnit translationUnit; + RefPtr<ProgramSyntaxNode> translationUnitSyntax = new ProgramSyntaxNode(); - RefPtr<ProgramSyntaxNode> translationUnitSyntax = new ProgramSyntaxNode(); + TranslationUnitOptions translationUnitOptions; + translationUnit.options = translationUnitOptions; + translationUnit.SyntaxNode = translationUnitSyntax; - TranslationUnitOptions translationUnitOptions; - translationUnit.options = translationUnitOptions; - translationUnit.SyntaxNode = translationUnitSyntax; + return translationUnit; + } - return translationUnit; - } + void addBuiltinSource( + RefPtr<Scope> const& scope, + String const& path, + String const& source); +}; - void addBuiltinSource( - RefPtr<Scope> const& scope, - String const& path, - String const& source); - }; +struct CompileRequest +{ + // Pointer to parent session + Session* mSession; - struct CompileRequest - { - // Pointer to parent session - Session* mSession; + // Input options + CompileOptions Options; + + // Output stuff + DiagnosticSink mSink; + String mDiagnosticOutput; - // Input options - CompileOptions Options; + RefPtr<CollectionOfTranslationUnits> mCollectionOfTranslationUnits; - // Output stuff - DiagnosticSink mSink; - String mDiagnosticOutput; + RefPtr<ProgramLayout> mReflectionData; - RefPtr<CollectionOfTranslationUnits> mCollectionOfTranslationUnits; + CompileResult mResult; - RefPtr<ProgramLayout> mReflectionData; + List<String> mDependencyFilePaths; - CompileResult mResult; + CompileRequest(Session* session) + : mSession(session) + {} - List<String> mDependencyFilePaths; + ~CompileRequest() + {} - CompileRequest(Session* session) - : mSession(session) - {} + struct IncludeHandlerImpl : IncludeHandler + { + CompileRequest* request; - ~CompileRequest() - {} + List<String> searchDirs; - struct IncludeHandlerImpl : IncludeHandler + virtual bool TryToFindIncludeFile( + CoreLib::String const& pathToInclude, + CoreLib::String const& pathIncludedFrom, + CoreLib::String* outFoundPath, + CoreLib::String* outFoundSource) override { - CompileRequest* request; + String path = Path::Combine(Path::GetDirectoryName(pathIncludedFrom), pathToInclude); + if (File::Exists(path)) + { + *outFoundPath = path; + *outFoundSource = File::ReadAllText(path); + + request->mDependencyFilePaths.Add(path); - List<String> searchDirs; + return true; + } - virtual bool TryToFindIncludeFile( - CoreLib::String const& pathToInclude, - CoreLib::String const& pathIncludedFrom, - CoreLib::String* outFoundPath, - CoreLib::String* outFoundSource) override + for (auto & dir : searchDirs) { - String path = Path::Combine(Path::GetDirectoryName(pathIncludedFrom), pathToInclude); + path = Path::Combine(dir, pathToInclude); if (File::Exists(path)) { *outFoundPath = path; @@ -150,303 +166,453 @@ namespace SlangLib return true; } + } + return false; + } + }; - for (auto & dir : searchDirs) - { - path = Path::Combine(dir, pathToInclude); - if (File::Exists(path)) - { - *outFoundPath = path; - *outFoundSource = File::ReadAllText(path); - request->mDependencyFilePaths.Add(path); + CompileUnit parseTranslationUnit( + TranslationUnitOptions const& translationUnitOptions, + CompileOptions& options) + { + IncludeHandlerImpl includeHandler; + includeHandler.request = this; - return true; - } - } - return false; - } - }; + CompileUnit translationUnit; + + RefPtr<Scope> languageScope; + switch (translationUnitOptions.sourceLanguage) + { + case SourceLanguage::HLSL: + languageScope = mSession->hlslLanguageScope; + break; + + case SourceLanguage::GLSL: + languageScope = mSession->glslLanguageScope; + break; + + case SourceLanguage::Slang: + default: + languageScope = mSession->slangLanguageScope; + break; + } + Dictionary<String, String> preprocessorDefinitions; + for(auto& def : options.preprocessorDefinitions) + preprocessorDefinitions.Add(def.Key, def.Value); + for(auto& def : translationUnitOptions.preprocessorDefinitions) + preprocessorDefinitions.Add(def.Key, def.Value); - CompileUnit parseTranslationUnit( - TranslationUnitOptions const& translationUnitOptions) + RefPtr<ProgramSyntaxNode> translationUnitSyntax = new ProgramSyntaxNode(); + + for (auto sourceFile : translationUnitOptions.sourceFiles) { - auto& options = Options; + auto sourceFilePath = sourceFile->path; + + auto searchDirs = options.SearchDirectories; + searchDirs.Reverse(); + searchDirs.Add(Path::GetDirectoryName(sourceFilePath)); + searchDirs.Reverse(); + includeHandler.searchDirs = searchDirs; + + String source = sourceFile->content; + + auto tokens = preprocessSource( + source, + sourceFilePath, + mResult.GetErrorWriter(), + &includeHandler, + preprocessorDefinitions, + translationUnitSyntax.Ptr()); + + parseSourceFile( + translationUnitSyntax.Ptr(), + options, + tokens, + mResult.GetErrorWriter(), + sourceFilePath, + languageScope); + } - IncludeHandlerImpl includeHandler; - includeHandler.request = this; + translationUnit.options = translationUnitOptions; + translationUnit.SyntaxNode = translationUnitSyntax; - CompileUnit translationUnit; + return translationUnit; + } - RefPtr<Scope> languageScope; - switch( translationUnitOptions.sourceLanguage ) - { - case SourceLanguage::HLSL: - languageScope = mSession->hlslLanguageScope; - break; - - case SourceLanguage::GLSL: - languageScope = mSession->glslLanguageScope; - break; - - case SourceLanguage::Slang: - default: - languageScope = mSession->slangLanguageScope; - break; - } + CompileUnit parseTranslationUnit( + TranslationUnitOptions const& translationUnitOptions) + { + return parseTranslationUnit(translationUnitOptions, Options); + } + void checkTranslationUnit( + CompileUnit& translationUnit, + RefPtr<SyntaxVisitor> visitor) + { + visitor->setSourceLanguage(translationUnit.options.sourceLanguage); + translationUnit.SyntaxNode->Accept(visitor.Ptr()); + } - auto& preprocesorDefinitions = options.PreprocessorDefinitions; + void checkTranslationUnit( + CompileUnit& translationUnit, + CompileOptions& options) + { + RefPtr<SyntaxVisitor> visitor = CreateSemanticsVisitor( + mResult.GetErrorWriter(), + options, + this); - RefPtr<ProgramSyntaxNode> translationUnitSyntax = new ProgramSyntaxNode(); + checkTranslationUnit(translationUnit, visitor); + } - for( auto sourceFile : translationUnitOptions.sourceFiles ) - { - auto sourceFilePath = sourceFile->path; + void checkCollectionOfTranslationUnits( + RefPtr<CollectionOfTranslationUnits> collectionOfTranslationUnits) + { + RefPtr<SyntaxVisitor> visitor = CreateSemanticsVisitor( + mResult.GetErrorWriter(), + Options, + this); + + for( auto& translationUnit : collectionOfTranslationUnits->translationUnits ) + { + checkTranslationUnit(translationUnit, visitor); + } + } - auto searchDirs = options.SearchDirectories; - searchDirs.Reverse(); - searchDirs.Add(Path::GetDirectoryName(sourceFilePath)); - searchDirs.Reverse(); - includeHandler.searchDirs = searchDirs; + void generateOutputForCollectionOfTranslationUnits( + RefPtr<CollectionOfTranslationUnits> collectionOfTranslationUnits) + { + // Do binding generation, and then reflection (globally) + // before we move on to any code-generation activites. + GenerateParameterBindings(collectionOfTranslationUnits.Ptr()); - String source = sourceFile->content; - auto tokens = preprocessSource( - source, - sourceFilePath, - mResult.GetErrorWriter(), - &includeHandler, - preprocesorDefinitions, - translationUnitSyntax.Ptr()); - - parseSourceFile( - translationUnitSyntax.Ptr(), - options, - tokens, - mResult.GetErrorWriter(), - sourceFilePath, - languageScope); - } + // HACK(tfoley): for right now I just want to pretty-print an AST + // into another language, so the whole compiler back-end is just + // getting in the way. + // + // I'm going to bypass it for now and see what I can do: - translationUnit.options = translationUnitOptions; - translationUnit.SyntaxNode = translationUnitSyntax; + ExtraContext extra; + extra.options = &Options; + extra.programLayout = collectionOfTranslationUnits->layout.Ptr(); + extra.compileResult = &mResult; - return translationUnit; - } + generateOutput(extra, collectionOfTranslationUnits.Ptr()); + } - int executeCompilerDriverActions() + int executeCompilerDriverActions() + { + // If we are being asked to do pass-through, then we need to do that here... + if (Options.passThrough != PassThroughMode::None) { - // If we are being asked to do pass-through, then we need to do that here... - if (Options.passThrough != PassThroughMode::None) + for (auto& translationUnitOptions : Options.translationUnits) { - for( auto& translationUnitOptions : Options.translationUnits ) + switch (translationUnitOptions.sourceLanguage) { - switch( translationUnitOptions.sourceLanguage ) - { // We can pass-through code written in a native shading language - case SourceLanguage::GLSL: - case SourceLanguage::HLSL: - break; + case SourceLanguage::GLSL: + case SourceLanguage::HLSL: + break; // All other translation units need to be skipped - default: - continue; - } - - auto sourceFile = translationUnitOptions.sourceFiles[0]; - auto sourceFilePath = sourceFile->path; - String source = sourceFile->content; - - mSession->compiler->PassThrough( - source, - sourceFilePath, - Options, - translationUnitOptions); + default: + continue; } - return 0; - } - // TODO: load the stdlib + auto sourceFile = translationUnitOptions.sourceFiles[0]; + auto sourceFilePath = sourceFile->path; + String source = sourceFile->content; - mCollectionOfTranslationUnits = new CollectionOfTranslationUnits(); + auto translationUnitResult = passThrough( + source, + sourceFilePath, + Options, + translationUnitOptions); - // Parse everything from the input files requested - // - // TODO: this may trigger the loading and/or compilation of additional modules. - for( auto& translationUnitOptions : Options.translationUnits ) - { - auto translationUnit = parseTranslationUnit(translationUnitOptions); - mCollectionOfTranslationUnits->translationUnits.Add(translationUnit); + mResult.translationUnits.Add(translationUnitResult); } - if( mResult.GetErrorCount() != 0 ) - return 1; - - // Now perform semantic checks, emit output, etc. - mSession->compiler->Compile( - mResult, mCollectionOfTranslationUnits.Ptr(), Options); - if(mResult.GetErrorCount() != 0) - return 1; - - mReflectionData = mCollectionOfTranslationUnits->layout; - return 0; } - // Act as expected of the API-based compiler - int executeAPIActions() - { - mResult.mSink = &mSink; - - int err = executeCompilerDriverActions(); + // TODO: load the stdlib - mDiagnosticOutput = mSink.outputBuffer.ProduceString(); + mCollectionOfTranslationUnits = new CollectionOfTranslationUnits(); - if(mSink.GetErrorCount() != 0) - return mSink.GetErrorCount(); - - return err; + // Parse everything from the input files requested + // + // TODO: this may trigger the loading and/or compilation of additional modules. + for (auto& translationUnitOptions : Options.translationUnits) + { + auto translationUnit = parseTranslationUnit(translationUnitOptions); + mCollectionOfTranslationUnits->translationUnits.Add(translationUnit); } + if (mResult.GetErrorCount() != 0) + return 1; - int addTranslationUnit(SourceLanguage language, String const& name) - { - int result = Options.translationUnits.Count(); + // Perform semantic checking on the whole collection + checkCollectionOfTranslationUnits(mCollectionOfTranslationUnits); + if (mResult.GetErrorCount() != 0) + return 1; - TranslationUnitOptions translationUnit; - translationUnit.sourceLanguage = SourceLanguage(language); + // Generate output code, in whatever format was requested + generateOutputForCollectionOfTranslationUnits(mCollectionOfTranslationUnits); + if (mResult.GetErrorCount() != 0) + return 1; - Options.translationUnits.Add(translationUnit); + // Extract the reflection layout information so that users + // can easily query it. + mReflectionData = mCollectionOfTranslationUnits->layout; - return result; - } + return 0; + } - void addTranslationUnitSourceString( - int translationUnitIndex, - String const& path, - String const& source) - { - RefPtr<SourceFile> sourceFile = new SourceFile(); - sourceFile->path = path; - sourceFile->content = source; + // Act as expected of the API-based compiler + int executeAPIActions() + { + mResult.mSink = &mSink; - Options.translationUnits[translationUnitIndex].sourceFiles.Add(sourceFile); - } + int err = executeCompilerDriverActions(); - void addTranslationUnitSourceFile( - int translationUnitIndex, - String const& path) - { - String source; - try - { - source = File::ReadAllText(path); - } - catch( ... ) - { - // Emit a diagnostic! - mSink.diagnose( - CodePosition(0,0,0,path), - Diagnostics::cannotOpenFile, - path); - return; - } + mDiagnosticOutput = mSink.outputBuffer.ProduceString(); - addTranslationUnitSourceString( - translationUnitIndex, - path, - source); + if (mSink.GetErrorCount() != 0) + return mSink.GetErrorCount(); - mDependencyFilePaths.Add(path); - } + return err; + } - int addTranslationUnitEntryPoint( - int translationUnitIndex, - String const& name, - Profile profile) - { - EntryPointOption entryPoint; - entryPoint.name = name; - entryPoint.profile = profile; + int addTranslationUnit(SourceLanguage language, String const& name) + { + int result = Options.translationUnits.Count(); - // TODO: realistically want this to be global across all TUs... - int result = Options.translationUnits[translationUnitIndex].entryPoints.Count(); + TranslationUnitOptions translationUnit; + translationUnit.sourceLanguage = SourceLanguage(language); - Options.translationUnits[translationUnitIndex].entryPoints.Add(entryPoint); - return result; - } - }; + Options.translationUnits.Add(translationUnit); - void Session::addBuiltinSource( - RefPtr<Scope> const& scope, - String const& path, - String const& source) + return result; + } + + void addTranslationUnitSourceString( + int translationUnitIndex, + String const& path, + String const& source) { - CompileRequest compileRequest(this); + RefPtr<SourceFile> sourceFile = new SourceFile(); + sourceFile->path = path; + sourceFile->content = source; - auto translationUnitIndex = compileRequest.addTranslationUnit(SourceLanguage::Slang, path); + Options.translationUnits[translationUnitIndex].sourceFiles.Add(sourceFile); + } - compileRequest.addTranslationUnitSourceString( + void addTranslationUnitSourceFile( + int translationUnitIndex, + String const& path) + { + String source; + try + { + source = File::ReadAllText(path); + } + catch (...) + { + // Emit a diagnostic! + mSink.diagnose( + CodePosition(0, 0, 0, path), + Diagnostics::cannotOpenFile, + path); + return; + } + + addTranslationUnitSourceString( translationUnitIndex, path, source); - int err = compileRequest.executeAPIActions(); - if(err) - { - fprintf(stderr, "%s", compileRequest.mDiagnosticOutput.Buffer()); + mDependencyFilePaths.Add(path); + } -#ifdef _WIN32 - OutputDebugStringA(compileRequest.mDiagnosticOutput.Buffer()); -#endif + int addTranslationUnitEntryPoint( + int translationUnitIndex, + String const& name, + Profile profile) + { + EntryPointOption entryPoint; + entryPoint.name = name; + entryPoint.profile = profile; - assert(!"error in stdlib"); - } + // TODO: realistically want this to be global across all TUs... + int result = Options.translationUnits[translationUnitIndex].entryPoints.Count(); + + Options.translationUnits[translationUnitIndex].entryPoints.Add(entryPoint); + return result; + } - // Extract the AST for the code we just parsed - auto syntax = compileRequest.mCollectionOfTranslationUnits->translationUnits[translationUnitIndex].SyntaxNode; + Dictionary<String, RefPtr<ProgramSyntaxNode>> loadedModules; - // HACK(tfoley): mark all declarations in the "stdlib" so - // that we can detect them later (e.g., so we don't emit them) - for (auto m : syntax->Members) + RefPtr<ProgramSyntaxNode> findOrImportModule( + String const& name, + CodePosition const& loc) + { + // Have we already loaded a module matching this name? + // If so, return it. + RefPtr<ProgramSyntaxNode> moduleDecl; + if (loadedModules.TryGetValue(name, moduleDecl)) + return moduleDecl; + + // Derive a file name for the module, by taking the given + // identifier, replacing all occurences of `_` with `-`, + // and then appending `.slang`. + // + // For example, `foo_bar` becomes `foo-bar.slang`. + + StringBuilder sb; + for (auto c : name) { - auto fromStdLibModifier = new FromStdLibModifier(); + if (c == '_') + c = '-'; - fromStdLibModifier->next = m->modifiers.first; - m->modifiers.first = fromStdLibModifier; + sb.Append(c); } + sb.Append(".slang"); - // Add the resulting code to the appropriate scope - if( !scope->containerDecl ) - { - // We are the first chunk of code to be loaded for this scope - scope->containerDecl = syntax.Ptr(); - } - else + String fileName = sb.ProduceString(); + + // Next, try to find the file of the given name, + // using our ordinary include-handling logic. + + IncludeHandlerImpl includeHandler; + includeHandler.request = this; + + String pathIncludedFrom = loc.FileName; + + String foundPath; + String foundSource; + bool found = includeHandler.TryToFindIncludeFile(fileName, pathIncludedFrom, &foundPath, &foundSource); + if (!found) { - // We need to create a new scope to link into the whole thing - auto subScope = new Scope(); - subScope->containerDecl = syntax.Ptr(); - subScope->nextSibling = scope->nextSibling; - scope->nextSibling = subScope; + this->mSink.diagnose(loc, Diagnostics::cannotFindFile, fileName); + + loadedModules[name] = nullptr; + return nullptr; } - // We need to retain this AST so that we can use it in other code - // (Note that the `Scope` type does not retain the AST it points to) - loadedModuleCode.Add(syntax); + // We've found a file that we can load for the given module, so + // now we need to try compiling it, etc. + + // We don't want to use the same options that the user specified + // for loading modules on-demand. In particular, we always want + // semantic checking to be enabled. + CompileOptions moduleOptions; + moduleOptions.SearchDirectories = Options.SearchDirectories; + moduleOptions.profile = Options.profile; + + RefPtr<SourceFile> sourceFile = new SourceFile(); + sourceFile->path = foundPath; + sourceFile->content = foundSource; + + TranslationUnitOptions translationUnitOptions; + translationUnitOptions.sourceFiles.Add(sourceFile); + + CompileUnit translationUnit = parseTranslationUnit(translationUnitOptions, moduleOptions); + + // TODO: handle errors + + checkTranslationUnit(translationUnit, moduleOptions); + + // Skip code generation + + // + + moduleDecl = translationUnit.SyntaxNode; + + loadedModules.Add(name, moduleDecl); + + return moduleDecl; } + +}; + +RefPtr<ProgramSyntaxNode> findOrImportModule( + CompileRequest* request, + String const& name, + CodePosition const& loc) +{ + return request->findOrImportModule(name, loc); } -using namespace SlangLib; +void Session::addBuiltinSource( + RefPtr<Scope> const& scope, + String const& path, + String const& source) +{ + CompileRequest compileRequest(this); + + auto translationUnitIndex = compileRequest.addTranslationUnit(SourceLanguage::Slang, path); + + compileRequest.addTranslationUnitSourceString( + translationUnitIndex, + path, + source); + + int err = compileRequest.executeAPIActions(); + if (err) + { + fprintf(stderr, "%s", compileRequest.mDiagnosticOutput.Buffer()); + +#ifdef _WIN32 + OutputDebugStringA(compileRequest.mDiagnosticOutput.Buffer()); +#endif + + assert(!"error in stdlib"); + } + + // Extract the AST for the code we just parsed + auto syntax = compileRequest.mCollectionOfTranslationUnits->translationUnits[translationUnitIndex].SyntaxNode; + + // HACK(tfoley): mark all declarations in the "stdlib" so + // that we can detect them later (e.g., so we don't emit them) + for (auto m : syntax->Members) + { + auto fromStdLibModifier = new FromStdLibModifier(); + + fromStdLibModifier->next = m->modifiers.first; + m->modifiers.first = fromStdLibModifier; + } + + // Add the resulting code to the appropriate scope + if (!scope->containerDecl) + { + // We are the first chunk of code to be loaded for this scope + scope->containerDecl = syntax.Ptr(); + } + else + { + // We need to create a new scope to link into the whole thing + auto subScope = new Scope(); + subScope->containerDecl = syntax.Ptr(); + subScope->nextSibling = scope->nextSibling; + scope->nextSibling = subScope; + } + + // We need to retain this AST so that we can use it in other code + // (Note that the `Scope` type does not retain the AST it points to) + loadedModuleCode.Add(syntax); +} + +}} // implementation of C interface -#define SESSION(x) reinterpret_cast<SlangLib::Session *>(x) -#define REQ(x) reinterpret_cast<SlangLib::CompileRequest*>(x) +#define SESSION(x) reinterpret_cast<Slang::Compiler::Session *>(x) +#define REQ(x) reinterpret_cast<Slang::Compiler::CompileRequest*>(x) SLANG_API SlangSession* spCreateSession(const char * cacheDir) { - return reinterpret_cast<SlangSession *>(new SlangLib::Session((cacheDir ? true : false), cacheDir)); + return reinterpret_cast<SlangSession *>(new Slang::Compiler::Session((cacheDir ? true : false), cacheDir)); } SLANG_API void spDestroySession( @@ -476,7 +642,7 @@ SLANG_API SlangCompileRequest* spCreateCompileRequest( SlangSession* session) { auto s = SESSION(session); - auto req = new SlangLib::CompileRequest(s); + auto req = new Slang::Compiler::CompileRequest(s); return reinterpret_cast<SlangCompileRequest*>(req); } @@ -536,7 +702,7 @@ SLANG_API void spAddPreprocessorDefine( const char* key, const char* value) { - REQ(request)->Options.PreprocessorDefinitions[key] = value; + REQ(request)->Options.preprocessorDefinitions[key] = value; } SLANG_API char const* spGetDiagnosticOutput( @@ -561,6 +727,18 @@ SLANG_API int spAddTranslationUnit( name ? name : ""); } +SLANG_API void spTranslationUnit_addPreprocessorDefine( + SlangCompileRequest* request, + int translationUnitIndex, + const char* key, + const char* value) +{ + auto req = REQ(request); + + req->Options.translationUnits[translationUnitIndex].preprocessorDefinitions[key] = value; + +} + SLANG_API void spAddTranslationUnitSourceFile( SlangCompileRequest* request, int translationUnitIndex, diff --git a/source/slang/syntax-visitors.h b/source/slang/syntax-visitors.h index 565ac3ace..da1984f05 100644 --- a/source/slang/syntax-visitors.h +++ b/source/slang/syntax-visitors.h @@ -10,11 +10,27 @@ namespace Slang namespace Compiler { class CompileOptions; + struct CompileRequest; class ShaderCompiler; class ShaderLinkInfo; class ShaderSymbol; - SyntaxVisitor * CreateSemanticsVisitor(DiagnosticSink * err, CompileOptions const& options); + SyntaxVisitor* CreateSemanticsVisitor( + DiagnosticSink* err, + CompileOptions const& options, + CompileRequest* request); + + // Look for a module that matches the given name: + // either one we've loaded already, or one we + // can find vai the search paths available to us. + // + // Needed by import declaration checking. + // + // TODO: need a better location to declare this. + RefPtr<ProgramSyntaxNode> findOrImportModule( + CompileRequest* request, + String const& name, + CodePosition const& loc); } } diff --git a/source/slang/syntax.cpp b/source/slang/syntax.cpp index 3050afff6..fffc6c725 100644 --- a/source/slang/syntax.cpp +++ b/source/slang/syntax.cpp @@ -193,11 +193,12 @@ namespace Slang return visitor->VisitParameter(this); } - // UsingFileDecl + // ImportDecl - RefPtr<SyntaxNode> UsingFileDecl::Accept(SyntaxVisitor * visitor) + RefPtr<SyntaxNode> ImportDecl::Accept(SyntaxVisitor * visitor) { - return visitor->VisitUsingFileDecl(this); + visitor->visitImportDecl(this); + return this; } // diff --git a/source/slang/syntax.h b/source/slang/syntax.h index 9e4486d4e..ac2bcac87 100644 --- a/source/slang/syntax.h +++ b/source/slang/syntax.h @@ -72,7 +72,16 @@ namespace Slang IntrinsicOp findIntrinsicOp(char const* name); - class IntrinsicModifier : public Modifier + // Base class for modifiers that mark something as "intrinsic" + // and thus lacking a direct implementation in the language. + class IntrinsicModifierBase : public Modifier + { + }; + + // A modifier that marks something as one of a small set of + // truly intrinsic operations that the compiler knows about + // directly. + class IntrinsicOpModifier : public IntrinsicModifierBase { public: // token that names the intrinsic op @@ -82,6 +91,20 @@ namespace Slang IntrinsicOp op = IntrinsicOp::Unknown; }; + // A modifier that marks something as an intrinsic function, + // for some subset of targets. + class TargetIntrinsicModifier : public IntrinsicModifierBase + { + public: + // Token that names the target that the operation + // is an intrisic for. + Token targetToken; + + // A custom definition for the operation + Token definitionToken; + }; + + class InOutModifier : public OutModifier {}; @@ -2041,22 +2064,12 @@ namespace Slang virtual RefPtr<SyntaxNode> Accept(SyntaxVisitor * visitor) override; }; - class UsingFileDecl : public Decl - { - public: - Token fileName; - - virtual RefPtr<SyntaxNode> Accept(SyntaxVisitor * visitor) override; - }; - + // A "module" of code (essentiately, a single translation unit) + // that provides a scope for some number of declarations. class ProgramSyntaxNode : public ContainerDecl { public: // Access members of specific types - FilteredMemberList<UsingFileDecl> GetUsings() - { - return GetMembersOfType<UsingFileDecl>(); - } FilteredMemberList<FunctionSyntaxNode> GetFunctions() { return GetMembersOfType<FunctionSyntaxNode>(); @@ -2074,15 +2087,26 @@ namespace Slang { return GetMembersOfType<TypeDefDecl>(); } -#if 0 - void Include(ProgramSyntaxNode * other) - { - Members.AddRange(other->Members); - } -#endif + virtual RefPtr<SyntaxNode> Accept(SyntaxVisitor * visitor) override; }; + class ImportDecl : public Decl + { + public: + // The name of the module we are trying to import + Token nameToken; + + // The scope that we want to import into + RefPtr<Scope> scope; + + // The module that actually got imported + RefPtr<ProgramSyntaxNode> importedModuleDecl; + + virtual RefPtr<SyntaxNode> Accept(SyntaxVisitor * visitor) override; + }; + + class IfStatementSyntaxNode : public StatementSyntaxNode { public: @@ -2492,10 +2516,7 @@ namespace Slang return program; } - virtual RefPtr<UsingFileDecl> VisitUsingFileDecl(UsingFileDecl * decl) - { - return decl; - } + virtual void visitImportDecl(ImportDecl * decl) = 0; virtual RefPtr<FunctionSyntaxNode> VisitFunction(FunctionSyntaxNode* func) { |
