summaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2019-11-06Add basic support for entry points in `.slang-lib` files. (#1112)Tim Foley
* Add basic support for entry points in `.slang-lib` files. The basic idea here is that when writing out a `.slang-lib` file based on a compile request, we include new sections in the generated RIFF that represent the entry points that were requested. The entry-point information is serialized in an entirely ad hoc fashion (a future change might clean it up to use the `OffsetContainer` machinery), and contains the name, profile, and mangled symbol name of an entry point. When deserializing this information, we create a list of "extra" entry points that gets attached to the front-end compile requests. These "extra" entry points get turned into `EntryPoint` objects at the same place in the code that entry points specified on the command line or via API would be checked, but the extra entry points bypass the semantic checking and just create "dummy" `EntryPoint` objects. Aside: the ability for a compile request to end up with entry points that weren't originally specified via API or command-line is not new. We already had support for compiling a translation unit with entry points entirely specified via `[shader(...)]` attributes, and this new support tries to function similarly. Because the "dummy" entry points don't retain AST-level information, several parts of the code have been modified to defensively check for `EntryPoint` objects without a matching AST declaration, and skip over them. The main place where this creates a problem is paramete binding, where ignoring the dummy entry point is appropriate since we currently assume linked-in library code has been laid out manually. One small cleanup here is that the `-r` command-line flag and the `spAddLibraryReference` API functio now bottleneck through a common routine to do their work, so that they both gain the new behavior without needing copy-paste programming. In order to keep the existing test case for library linking with entry points working, I had to add a flag to the `render-test` tool so that it can skip specifying entry point names as part of the compile request it creates. In that case it must instead assume that the entry points will be added to the compile request via other means. This logic is a bit magical, and hints that we should be looking for other ways to expose the library linking functionality over time. * fixup: remove alignment assertion
2019-11-06Support for [__extern] attribute (#1111)jsmall-nvidia
* Added RiffReadHelper * Move type to fourCC in Chunk simplifies some code. * Make MemoryArena able to track external blocks. Allow ownership of Data to vary. Changed IR serialization to use moved allocations to avoid copies. As it turns out all of the array writes could use unowned data, but doing so requires the IRData to stay in scope longer than IRSerialData, which it does at the moment - but perhaps needs better naming or a control for the feature. * Write out slang-module container. * WIP on -r option. Loading modules - with -r. * Making the serialized-module run (without using imported module). * Split compiling module from the test. * Separate module compilation with a function working. * Remove serialization test as not used. * Fix warning on gcc. * Updated test to have types across module boundary. * Allow entry point declaration. A test that tries to build with just an entry point declaration and a module. * Try to make link work with multiple modules. * Multi module linking first pass working. * Multi module test working with -module-name option * Added feature to repro manifest of approximation of command line that was used. * Use isDefinition - for determining to add decorations to entry point lowering. * Added support for repo-file-system.h More precise control of CacheFileSystem. Allow RelativeFileSystem to strip paths optionally. Use canonical paths in PathInfo cache. Fix bug in -D options for command line output of StateSerailizeUtil * Add missing slang-options.h * Fix bug in bit slang-state-serialize.cpp with bit removal. * Added documentation around -repro-file-system Added spLoadReproAsFileSystem function. * Fix warning. * spAddLibraryReference * * Add support for slang-lib extension * Container output when using -no-codegen option * Use the m_containerFormat to determine if the module container is constructed. Store the result in a blob. This allows for potential access via the API. Write the blob if a filename is set. Use m_ prefix for container variables. * Added spGetContainerCode. Made spGetCompileRequestCode work. * * Put obfuscateCode on linkage * Remove obfuscation from variable names - as can be achieved by either stripping and/or removing NameHintDecorations at lowering * Remove name hints being added during lowering * Add stripping of SourceLoc location in strip phase * Hashing of linkage import/export names. * Do final strip in emitEntryPoint, removes any remaining SourceLoc. * Support for [__extern] to mark struct/function that are defined elsewhere. * Allow adding extern to any decl. * Use ExternAtrtibute to apply import decoration, rather than use an ir extern decoration. * Added a test for [__extern] * Improved comment around [__extern]
2019-10-31Reference IR modules with entry point (#1101)jsmall-nvidia
* Added RiffReadHelper * Move type to fourCC in Chunk simplifies some code. * Make MemoryArena able to track external blocks. Allow ownership of Data to vary. Changed IR serialization to use moved allocations to avoid copies. As it turns out all of the array writes could use unowned data, but doing so requires the IRData to stay in scope longer than IRSerialData, which it does at the moment - but perhaps needs better naming or a control for the feature. * Write out slang-module container. * WIP on -r option. Loading modules - with -r. * Making the serialized-module run (without using imported module). * Split compiling module from the test. * Separate module compilation with a function working. * Remove serialization test as not used. * Fix warning on gcc. * Updated test to have types across module boundary. * Allow entry point declaration. A test that tries to build with just an entry point declaration and a module. * Try to make link work with multiple modules. * Multi module linking first pass working. * Multi module test working with -module-name option * Use isDefinition - for determining to add decorations to entry point lowering.
2019-10-30WIP: Simple separate IR module linkage working (#1100)jsmall-nvidia
* Added RiffReadHelper * Move type to fourCC in Chunk simplifies some code. * Make MemoryArena able to track external blocks. Allow ownership of Data to vary. Changed IR serialization to use moved allocations to avoid copies. As it turns out all of the array writes could use unowned data, but doing so requires the IRData to stay in scope longer than IRSerialData, which it does at the moment - but perhaps needs better naming or a control for the feature. * Write out slang-module container. * WIP on -r option. Loading modules - with -r. * Making the serialized-module run (without using imported module). * Split compiling module from the test. * Separate module compilation with a function working. * Remove serialization test as not used. * Fix warning on gcc. * Updated test to have types across module boundary.
2019-10-24Strip IR after front-end steps are done (#1092)Tim Foley
* Strip IR after front-end steps are done The main feature of this change is to unconditonally strip out the `IRHighLevelDeclDecoration`s in an IR module once the "mandatory" IR passes in the front end have run. This ensures that later IR passes (e.g., code emission) *cannot* rely on AST-level information to get their job done. Since I was already writing a pass to remove some instructions at the end of the front-end passes, I went ahead and also made the `-obfuscate` flag apply to the front-end IR generation by causing it to strip `IRNameHintDecoration`s while it is doing the other stripping. With this, the main identifying information left in IR modules (other than semantics and entry-point names) is mangled name strings for imported/exported symbols. A few other things got changes along the way: * Removed the `.expected` file for one of the tests, where that file seemingly shouldn't have been checked in at all. * Updated the signature of the DCE pass both so that it doesn't require a back-end compile request (it wasn't using it anyway), and so that it takes some options to decide whether to keep symbols marked `[export(...)]` alive (the front-end wants to keep these, while back-end passes currently need to be able to eliminate them). * Moved the `obfuscateCode` flag from the back-end compile request to the base class shared between front- and back-end requests, and updated the options and repro logic to set both as needed. An obvious improvement in the future would be to have the front- and back-end requests share these settings by referencing a single common object in the end-to-end case, rather than each having their own copy. * Removed logic that was keeping layout instructions alive in DCE, even if they weren't used. This seems to have been a vestige of an intermediate step between AST and IR layout. * fixup: add the new files
2019-10-21`Repro` functionality (#1085)jsmall-nvidia
* WIP on serialize/save state. * Relative string encoding. * Added RelativeContainer unit test. Split out RelativeContainer into core. * Fix bug in RelativeString encoding. * More work around relative container. * Fix checks. * Use RelativeBase for safe access. Use malloc/free/realloc instead of List. * Add natvis support for relative types. * Setting up of state (not includes) writing of repro state. * Capture after spCompile. * Writing SourceFile and file system files. Added -dump-repo * First pass at loading state. * First pass at reading repro. * Small optimization around Safe32Ptr * Refactor how repro data is stored - to make saving off the files more simple, by having all all backed by 'files'. Make file loading always set up PathInfo so we get uniqueIdentifier info. * Generate unique file names. * Added RelativeFileSystem Added saveFile to ISlangFileSystemExt and implemented for interfaces Added mechanism to save of files (and manifest) * Added ability to replace files in repo with directory holding their contents. * Add support for entry points. * Fix problem compiling on linux. * Added SIMPLE_EX option, where everything on command line must be specified. * Fix typo in unit test for relative container. * Fix another typo in unit test for RelativeContainer. * Fix small bugs. * Fix release unused variable issue in slang-state-serialize.cpp * Fix checking for SIMPLE_EX in testing, else broke COMMAND_LINE_SIMPLE. * Fix warnings on 32 bit debug build. * Added import-subdir-search-path-repro.slang test. Although disabled for now as writes to root of slang project. * Remove wrong version of import-subdir-search-path-repro.slang * Added import-subdir-search-path-repro.slang
2019-10-17Initial work on representing layout at IR level (#1079)Tim Foley
* Initial work on representing layout at IR level This change starts the process of making the back-end of the compiler independent of the AST-level layout information (`TypeLayout`, `VarLayout`, etc.) so that it instead only relies on layout information that is embedded into IR modules. This brings us incrementally closer to a world in which the back-end could be run without the AST-level structures even existing (e.g., for an application that just wants to ship IR without any AST information for IP protection, while still supporting some amount of linking and specialization). The main parts of the change are: * There is a bunch of incidental churn related to specifying entry points by index instead of the `EntryPoint` object for certain operations. This ends up being a better choice because we can use the index to look up side-band information about the entry point that might not be stored on the `EntryPoint` object itself. In particular... * We expand the `ComponentType` interface to support looking up the mangled name of an entry point by index. In common cases (no generic/interface specialization) this would be the same as asking the `EntryPoint` for its mangled name, but in cases where we have specialized a generic entry point, the mangled name would include speicalization arguments that are only available on the `SpecializedComponentType` that wraps the entry point. This part of the change isn't ideal and there might be a better solution waiting to be invented. Note that we store mangled entry point names as strings rather than using `DeclRef`s because that ensures that the information could be serialized and deserialized without a dependence on the AST. * The `TargetProgram` type (which represents binding a specific `ComponentType` for a shader program to a specific `TargetRequest` that represents the target platform) is expanded to include an `IRModule` that represents layout information, in addition to the AST-level `ProgramLayout` it already contained. We create both of these objects at the same time (on-demand) to simplify the overall flow (so that any code that triggers creation of the AST-level layout will also ensure that the IR-level layout exists). * A bunch of code in the emit passes that was passing down layout-related objects has been eliminated. It appears that most of those objects weren't actually being used, so this is just a cleanup, but it helps ensure that the back-end steps are "clean" and don't depend on the AST-level information. The one big exception here is that the emit logic needs to know the stage for the entry point being emitted (to deal with one wrinkle in translating DXR to VKRT). * A big change (actually introduced by @jsmall-nvidia in a branch that this change copied and then built from) is to introduce some more explicit IR instructions to represent layout information, notably an `IRTypeLayout` and an `IRVarLayout`. For now these objects still reference their AST equivalents, but the separation gives us an incremental path to move information from the AST-level objects over to the IR ones. This work includes logic in `IRBuilder` to construct the IR-level layout objects from the AST-level ones on-demand, so that the existing code paths that try to attach AST-level layout will continue to work for now. * Because layout information is now embedded in the IR, the `slang-ir-link.cpp` logic loses a lot of cases that used to deal with attaching AST-level layout objects to IR-level instructions during the linking process. Instead, the linker now assumes that one (or more) of the input IR modules will have layout information associated with it, and the linker makes sure to copy layout decorations (and the instructions they reference) from the input IR module(s) to the output using its more ordinary mechanisms. * Inside `slang-lower-to-ir.cpp`, we add logic to construct an IR module in a `TargetProgram` that simply references the global shader parameters, entry points, etc. and attaches IR layout decorations to them. This is akin to the existing pass in the same file that constructs IR to represent specialization information, and both of these passes share infrastructure with the main AST->IR lowering pass. Eventually, it is expected that this pass will encompass more of the logic for copying AST-level layout information over to IR-level equivalents. * One small wrinkle with this change was that the output for an HLSL generation test case changed some of its `#line` directives. The old code was actually more inaccurate than the new, so this change just updated the baseline. It also added some logic in the linker to make sure that when an IR instruction has multiple definitions, we try to pick up a source location from any of them, in case the "main" one somehow didn't get a location. * Another small fix was that the key/value map in `StructTypeLayout` for mapping fields/members to their layouts was keyed on `Decl*` when it really should have been `VarDeclBase*`. This change should in principle be a pure refactoring with no functionality changes, so no new tests were added. It is unfortunately also a change that has a high probability of breaking at least *some* client code, so we may want to be defensive and mark this with a new major version number (well, a new *minor* version number since we are pre-`1.0`) to give us some room for releasing hotfixes to the old version if needed. * fixup: infinite recursion bug detected by clang * fixup: remove commented-out code
2019-10-17Feature/gpu unbound array of array (#1083)jsmall-nvidia
* Simple testing of unbounded array of array on GPU. * Fix problem on CPU targets around NonUniformResourceIndex Use the unbounded-array-of-array-syntax test for CPU and GPU tests.
2019-10-15GetDimension on GLSL for StructuredBuffer (#1081)jsmall-nvidia
* Fix GetDimensions for glsl. * Add test for Load on RWStructuredBuffer as part of GetDimension.
2019-10-11CPU unsized array documentation and another example (#1080)jsmall-nvidia
* WIP: Unsized arrays on CPU. * unbounded-array-of-array working on CPU. * Test that has an unbounded array of array directly (ie without wrapping with ParameterBlock). Test works on CPU. * Remove some left over comments. * Added documention on unsized array usage on CPU targets.
2019-10-11Support for unbounded array of arrays (#1078)jsmall-nvidia
* WIP: Unsized arrays on CPU. * unbounded-array-of-array working on CPU. * Remove some left over comments.
2019-10-09Feature/decor entry point name (#1073)jsmall-nvidia
* Use name hint on EntryPoint naming. * Placed the entry point name on the EntryPointDecoration.
2019-10-08Feature/ir entry point profile (#1068)jsmall-nvidia
* Split out EntryPointParamDecoration. * Add profile to EntryPointDecoration. * WIP for GS handling for GLSL. * WIP for StreamOut GLSL * Fixed GLSL geometry output. * Clean up - remove unneeded/commented out code from the entry point change. * Use Op nums to identify GeometryTypeDecorations (as opposed to contained enum).
2019-10-04IR types for subset of Attributes (#1067)jsmall-nvidia
* IROutputControlPointsDecoration * IROutputTopologyDecoration * IRPartitioningDecoration * IRDomainDecoration * Use IRPatchConstantDecoration alone for hlsl output. * IRMaxVertexCountDecoration * IRInstanceDecoration * Removed _emitHLSLAttributeSingleString and _emitHLSLAttributeSingleInt Removed GLSLBindingAttribute and just use NumThreadsAttribute * Added IRNumThreadsDecoration. * Added IRNumThreadsDecoration * Fix build problem on x86. Improve diagnostic text based on review.
2019-09-23Simple test profiling (#1062)jsmall-nvidia
* First pass support for performance profiling * Test across all elements * Fix bug - sourceContents is not used, should use rawSource. * * Add ability to get prelude from API. * Allow specifying source language for render-test * Made it possible to compile a test input file as C++ * Special handling for reflection * Added C++ impl to performance-profile.slang * Remove some clang warnings. * Output profile timings on appveyor and other TC. * Remove passing around of StdWriters (can use global). Small comment improvements.
2019-09-19Disable dx12 half-structured-buffer.slang test, as produces inconsistent ↵jsmall-nvidia
results. (#1061)
2019-09-18Clean up some behavior of operator% (#1060)Tim Foley
Work on #1059 The `%` operator in the Slang implementation had several issues, and this change tries to address some of them: * Renamed most occurences of "mod" describing this operator to be "rem" for "remainder" to better match its semantics in HLSL * Split the operator into distinct integer and floating-point variants (`IRem` and `FRem`) to simplify having different codegen for the two * Added floating-point variants of `operator%` and `operator%=` to the stdlib. * Added custom C++ codegen for `kIROp_FRem` such that it maps to the standard C/C++ `remainder()` function * Added custom GLSL codegen so that `kIROp_FRem` maps to the GLSL `mod()` function (which isn't correct...) * Added a test case to confirm that D3D11, D3D12, and CPU targets all agree on the definition of floating-point `%` * Fixed `render-test-tool` to allow a negative integer in a `data=...` specification. This didn't end up being used in the final test, but still seems like a good fix. * Added a customized baseline for the Vulkan flavor of that test to confirm that we are *not* compiling correctly to SPIR-V just yet Addressing the correctness of the output for GLSL/SPIR-V will have to come as a later change given that the operation we want is not exposed directly by unextended GLSL.
2019-09-18Improvements to testing and ABI for CPU (#1057)jsmall-nvidia
* WIP: Improving CPU performance/ABI * Optionally output code on CPU for groupThreadID and groupID. * Added ability to set compute dispatch size on command line for render-test. Dispatch compute tests taking into account dispatch size. Added test for semantics are working. * Test using GroupRange. * Fix problem with adding \n for externa diagnostic - to do it if there isn't a \n at the end. Change the ouput order (put result before) so last value is diagnostic string. * Made GroupRange the default exposed CPU ABI entry point style. Removed CPU_EXECUTE test style -as tested via the now cross platform render-test * Split out execution from setup for execution to improve perf. * For better code coverage/testing test all styles of CPU compute entry point. * Improve documentation for ABI changes for CPU code. Add 'expecting' to error message from review. * Fix small typos.
2019-09-17CPU ABI improvements (#1056)jsmall-nvidia
* WIP: Improving CPU performance/ABI * Optionally output code on CPU for groupThreadID and groupID. * Added ability to set compute dispatch size on command line for render-test. Dispatch compute tests taking into account dispatch size. Added test for semantics are working. * Test using GroupRange. * Fix problem with adding \n for externa diagnostic - to do it if there isn't a \n at the end. Change the ouput order (put result before) so last value is diagnostic string.
2019-09-16CPU Performance/Testing improvements (#1055)jsmall-nvidia
* First pass of render-test refactor. * Make window construction a function that can choose an implementation. * Remove OpenGL as currently has windows dependency. * Disable Vulkan as Renderer impl has dependency on windows. * Pass Window in as parameter of 'update'. * Add win-window.cpp as was missing. * Fix warning on windows about signs during comparison. * * Added mechanism to add random arrays as buffer inputs and select type * Improved RenderGenerator to generate more types, and to be more careful around int32 ranges. * Added support for security checks (for Visual Studio C++) * Disable Execption handling being on by default when compiling kernels * Added a 'Group' version of the entry point that will evaluate all threads in a group in a single call. In test code use this method if available. * Added -compile-arg to be able to pass arguments to the compile within render-test * Add documention for the _Group execution feature. * Fix some typos in cpu-target.md
2019-09-12Fix problems with x86 CPU tests (#1051)jsmall-nvidia
* Try to make x86 builds on x86 platforms (not the default for the os). * Use c style include for stdint.h cos not found on x86 linux. * Simplified x86 issue for linux. * Fix typo. * Remove the need for the shared-library category. * Disable CPU tests on linux x86. * Fix typo. * Named test requirement methods so overloading not confusing (around flags, and SlangPassThroughType which are both 'int')
2019-09-12ParameterBlock support. (#1049)jsmall-nvidia
* Updated docs to reflect ParameterBlock support * Fixed CPU binding to handle ParameterBlocks * Updated parameter-block.slang to be able to work as a CPU test
2019-09-03CPU uniform entry point params (#1041)jsmall-nvidia
* * Made entry point parameters a separate entry point * Made CPUMemoryBinding work with entry point parameters/initialize constant buffers * Added isCPUOnly to bindings, because entry point parameters do not layout like constant buffer * entry-point-uniform.slang works on CPU * EntryPointParams -> UniformEntryPointParams Updated CPU documentation. * Update cpu-target.md to removed completed issues. * Only allocate CPU buffers if the size is > 0. Small update to cpu-target doc.
2019-08-26WIP: CPU sample working with Texture2D (#1033)jsmall-nvidia
* WIP: Memory binding. * WIP for binding. * Fix handling of writing to constant buffer. * Fix bug in handling indices.
2019-08-22WIP: CPU compute coverage (#1030)jsmall-nvidia
* Add support for '=' when defining a name in test. * Add support for double intrinsics. * Add support for asdouble Add findOrAddInst - used instead of findOrEmitHoistableInst, for nominal instructions. Support cloning of string literals. C++ working on more compute tests. * Constant buffer support in reflection. Fixed debugging into source for generated C++. buffer-layout.slang works. * Added cpu test result. * Remove some commented out code. Comment on next fixes. * Improvements to reflection CPU code. * C++ working with ByteAddressBuffer. * Enabled more compute tests for CPU. * Enabled more compute tests on CPU. Added support for [] style access to a vector. * Enabled more CPU compute tests. * Handling of buffer-type-splitting.slang Named buffers can be paths to resources * Fix some warnings, remove some dead code. * Fix problem with verification of number of operands for asuint/asint as they can have 1 or 3 operands. asdouble takes 2. * Fix handling in MemoryArena around aligned allocations. That _allocateAlignedFromNewBlock assumed the block allocated has the aligment that was requested and so did not correct the start address.
2019-08-20User defined downstream compiler prelude (#1028)jsmall-nvidia
* Added setDownstreamCompilerPrelude Renamed setPassThroughPath to setDownstreamCompilerPath. Fixed tests. Added prelude directory & code to TestToolUtil to setup default preludes for testing/command line apis. * Fix merge problem * Remove hacks to make prelude work by adding a search path as no longer needed with 'user prelude'. * Split up prelude into scalar intrinsics, and types. Use slang.h for main header. slang-cpp-prelude.h can now just include what it needs (relative to prelude directory) and define the few remaining things/work arounds. * Fix typo.
2019-08-19Support shifts and a few other ops in front-end constant folding (#1027)Tim Foley
The set of supported operations in front-end constant folding was very limited: `+`, `-`, `*`, `/`, and `%`. This meant that enum declarations like: ``` enum MyBits { A = 1 << 0, B = 1 << 1, C = A | C, } ``` would fail to compile, with a claim that the expressions like `1 << 0` aren't compile-time constants. This change adds `<<`, `>>`, `&`, `|`, and `^` to the list of integer operations we will cosntant-fold in the front-end. It also changes one of the declarations in the existing test case for `enum`s to use the added functionality. Note that this change does *not* address the more deep-seated problems with our approach to constant-folding in the front-end. It does not change the constant folding to rely on IR machinery, or to allow for more general `constexpr` functions, and it does not address the fact that constant-folding is currently applied without paying attention to the type (and thus precision) of the original expression.
2019-08-19Testing using a 'naked' ISlangFileSystemExt option. (#1026)jsmall-nvidia
2019-08-19WIP: Compute test running on CPU (#1023)jsmall-nvidia
* * Simplify some of test code around CPPCompiler * Test using 'callable' with pass-through * Small cpu doc improvements * Improvements to Clang output parsing. * Remove temporary file (base filename) . * Improve handling of external errors - handle severity. * On error dumping out to 'actual' file for runCPPCompilerCompile. * Small fixes. Set the source language type correctly for pass thru. * Remove warning for test for clang backend c * Preliminary work around making render-test compute potentiall work with CPU. Made ShaderCompiler -> a stateless ShaderCompilerUtil. Means we don't require a Renderer interface to do shader compilation. * Refactor such that CPU test can take place in without Window or Renderer. * Hack to look for prelude in source file directory. Fix bug returning the SharedLibrary for HostCallable. * Compute test running on CPU. * Need the prelude currently in same directly as test. * Hack to remove warning - that then produces an error on appveyor build. Disable running render CPU test on non-windows. * Improve handling of disabling CPU tests on linux. * Added bit-cast.slang working on CPU.
2019-08-16Fix a typo in core.meta.slang which was causing an assert when (#1024)Robert Stepinski
compiling shaders that used texture2DMS Load() operations
2019-08-15A more convoluted #pragma once file identity test, using relative paths. (#1021)jsmall-nvidia
* A more convoluted #pragma once file identity test, using relative paths. * Fix bug with passing - to slang as a command line option causes a crash. Ability to set file-system to use on command line. #pragma once tests try with 'normal' and 'read-file' only versions * OSFileSystem -> OSFileSystemExt LoadFileOSFileSystem -> OSFileSystem Implemented OSFileSystem like OSFileSystemExt as as singleton. Fixes to comments.
2019-08-14Small improvements around C/C++ testing (#1017)jsmall-nvidia
* * Simplify some of test code around CPPCompiler * Test using 'callable' with pass-through * Small cpu doc improvements * Improvements to Clang output parsing. * Remove temporary file (base filename) . * Improve handling of external errors - handle severity. * On error dumping out to 'actual' file for runCPPCompilerCompile. * Small fixes. Set the source language type correctly for pass thru. * Remove warning for test for clang backend c
2019-08-12Callable CPU code support (#1014)jsmall-nvidia
* First pass support for compiling to a loaded shared library. * Improve documentation for cpu target. * Removed the SLANG_COMPILE_FLAG_LOAD_SHARED_LIBRARY flag. Use the SLANG_HOST_CALLABLE code target Document mechanism. * Fix typo in cpp-resource.slang In test code if the target is 'callable' we don't need to compile (indeed there is no source file). * Small refactor using CommandLineCPPCompiler as base class to implement VisualStudioCPPCompiler and GCCCPPCompiler. * Improvements around CPPCompiler. Mechanism to know products produced. Cleaning up products after execution. * Fix multiple definition of 'SourceType'
2019-08-09Fix TeamCity CI issues (#1013)jsmall-nvidia
* Added CPU_REFLECTION test option - that has two versions of the reflection output depending on ptr size. * Added 'shared-library' test category. This category is disabled on CI targets that have issues.
2019-08-08WIP: Preliminary Slang -> C++ code generation (#1009)jsmall-nvidia
* Expanded prelude for some other resource types. Disable C++ output for ParameterGroup. * WIP: Layout for CPU. * Fixes to CPU layout. * WIP: The uniform is output, but the variable definition is not. * WIP: Entry point parameters to global scope in C++. Handling of resource types (in so far as outputting) * Some discussion of ABI and different input types. * WIP: More C++ support around resource types. * WIP: Split up variables into different structures on emit. * WIP: Emitting C++ with wrapping up of 'Context' * WIP: C++ code has access to semantic values. Wrap in struct so can use method calls to pass shared state. Disable legalizeResourceTypes and legalizeExistentialTypeLayout * Fix structured buffer layout for CPU. * Remove testing/handling of global uniforms on CPU path. Typo fix. Changed CPU tests to use new CPU calling convention. * Check globals are working. Initalize context to zero globals. * Order the global parameters for C++ ouput by their layout. Note - that layout isn't quite working correctly because the StructuredBuffer<int> the int seems to be consuming uniform space. * Work around for reflection not having all data needed for layout ordering for C++ code. * Output constant buffers as pointers. * Entry point parameters accessed through pointer to struct. * WIP: Layout for CPU is reasonable for test case. * Only output 'f' after float literal if type marks as a float. * Cast construction works on C++. * Made IntrinsicOp::ConvertConstruct to make intent clearer. * C++ handling construction from scalar. Handle access of a scalar with .x. Check default initialization. * Comment about need for split of kIROp_construct. Release build works. * Added support from constructVectorFromScalar to C/C++ target. * Handling of in/out in C/C++. * First pass documentation CPU support. * Improvements to C++/C slang code generation documentation. * Small doc change to include need for mechansim to specify cpp compiler path. * Better handling of swizzling - allow swizzling a scalar into a vector.
2019-08-06Add support for the HLSL "cast from zero" idiom (#1008)Tim Foley
If the user writes code like this: MyStruct s = (MyStruct) 0; then we will interpret it as if they had written: MyStruct s = {}; That is, the "cast from zero" idiom will be taken as a legacy syntax for default construction (using an empty initializer list). This will be semantically equivalent to zero-initialization for all existing HLSL code (where `struct` fields can't have default initialization expressions defined), and is the easiest option for us to support in Slang (since we already support default-initialization using empty initializer lists). The implementation of this feature is narrowly scoped: * It only targets explicit cast expressions like `(MyStruct) 0` and not "constructor" syntax like `MyStruct(0)` * It only applies when there is a single argument that is exactly an integer literal with a zero value (not a reference to a `static const int` that happens to be zero). This change adds a test case to make sure that the feature works as expected. Because it relies on our existing initializer-list handling, the "cast from zero" idiom should work for any user-defined type where an initializer list would work.
2019-07-29Fix issue with outputting "static" in GLSL (#1006)Tim Foley
This appears to be a regression introduced in #1001, and missed because none of our existing tests covered `static const` arrays on the GLSL/SPIR-V targets. The basic problem is that we cannot output a `static const` definition in GLSL because `static` is a reserved word and not a keyword. Instead for GLSL we just want a `const` array. This change makes the emission of `static` for global-scope constants key on the target language for code generation, and only emit it for HLSL, C, and C++. This change also adds a test case specifically for running Slang input that has a `static const` array on the Vulkan target.
2019-07-29Add an attribute to disable the overlapping-bindings warning (#1005)Tim Foley
Currently if the user gives two global shader parameters conflicting bindings, they get a warning diagnostic: ```hlsl Texture2D a : register(t0); Texture2D b : register(t0); // WARNING: overlapping bindings ``` This change adds a way to locally disable that warning using an attribute: ```hlsl [allow("overlapping-bindings")] Texture2D a : register(t0); [allow("overlapping-bindings")] Texture2D b : register(t0); // OK ``` Note that as a policy decision, the implementation requires `[allow("overlapping-bindings")]` on both declarations in order to disable the warning, under the assumption that the behavior should be strictly opt-in, and not silently affect a programmer who adds a new shader parameter with no knowledge or expectation of possible overlap. The `[allow(...)]` attribute is intended to be a fairly generally mechanism for disabling optional diagnostics within certain scopes (e.g., for the body of a function definition), but as implemented in this change it is quite restrictive: * Only the single name `"overlapping-bindings"` will be recognized, and this name cannot be used with, e.g., a `-W` flag on the command line to enable/disable the same diagnostic, or turn it into an error. Adding more cases would be easy enough, but wiring it up to command-line flags could be trickier. * Only the code that checks for parameter binding overlap is currently checking for `[allow(...)]` attributes, so it is not "wired up" to enable/disable any others. Doing this systematically would ideally involve something in `diagnose()`, but there could be complications to a systematic approach (finding the AST node(s) to use when searching for `[allow(...)]`. On gotcha here is that versions of Slang without this feature will error out on the `[allow(...)]` attribute since they don't understand it, and if we add future diagnostics that it covers then old compiler versions will (as written) error out on a diagnostic they haven't heard of rather than just assume the `[allow(...)]` attribute doesn't apply to them. These kinds of issues can and should be addressed in future changes.
2019-07-17Change how global-scope constants are handled (#1001)Tim Foley
Before this change, global and function-scope `static const` declarations were represented as instructions of type `IRGlobalConstant`, which was represented similarly to an `IRGlobalVar`: with a "body" block of instructions that compute/return the initial value. This representation inhibited optimizations (because a reference to a global constant would not in general be replaced with a reference to its value), and also caused problems for resource type legalization because the logic for type legalization did not (and still does not) handle initializers on globals (so global *variables* that contain resource types are still unsupported). The change here is simple at the high level: we get rid of `IRGlobalConstant` and instead handle global-scope constants as "ordinary" instructions at the global scope. E.g., if we have a declaration like: static const int a[] = { ... } that will be represented in the IR as a `makeArray` instruction at the global scope, referencing other global-scope instructions that represent the values in the array. This simple choice addresses both of the main limitations. A `static const` variable of integer/float/whatever type is now represented as just a reference to the given IR value and thus enables all the same optimizations. When a `static const` variable uses a type with resources, the existing legalization logic (which can handle most of the "ordinary" instructions already) applies. Another secondary benefit of this approach is that the hacky `IREmitMode` enumeration is no longer needed to help us special-case source code emit for `static const` variables. Beyond just removing `IRGlobalConstant`, and updating the lowering logic to use the initializer direclty, the main change here is to the emit logic to make it properly handle "ordinary" instructions that might appear at global scope. One open issue with this change, that could be addressed in a follow-up change, is that "extern" global constants that need to be imported from another module (but which might not have a known value when the current module is compiled) aren't supported - we don't have a way to put a linkage decoration on them. A future change might re-introduce global constants as a distinct IR instruction type that just references the value as an operand (if it is available). We would then need to replace references to an IR constant with references to its value right after linking.
2019-07-17Slang -> C++ -> SharedLibrary -> Test (#999)jsmall-nvidia
* WIP: Adding support for C/C++ compilation to slang API. * Removed BackEndType in test harness -> use SlangPassThrough to identify backends Only require stage for targets that require it. Detection of all different backends. * Windows/Unix create temporary filename. * WIP: Output CPU binaries. * Added a pass-through c/c++ test. * Compile C++/C and store in temporary file. * Read the binary back into memory. * Set debug info and optimization flags for C/C++. Make the CPPCompiler debug/optimization levels match slangs. * Handling of include paths and math precision. * Dumping c++/c source and exe/shared library. * Put hex dump into own util. * End to end pass through c compilation test. * WIP: Simple execute test working on Linux/Unix. * Fix typo on linux. * WIP: To compile slang to cpp shared library. Report backend compiler errors. * Compiles slang -> cpp and loads as shared library. * Fix problem on c-cross-compile test because prelude is now included with <> quotes. * Run slang generated cpp code - using hard coded data. * Added cpp-execute-simple, and test output. * Fix warning that broke win32 build. * Fix compilation problem on osx.
2019-07-09WIP: slang to C++ code generation (#997)jsmall-nvidia
* WIP: Emitting Cpp * Added HLSLType instead of using IRInst - because they don't seem to be deduped. * Removed need for lexer to take a String. Added mechansim to lookup intrinsic functions on C++. * A c/c++ cross compilation test. * WIP Cpp output using cloning and slang types. * More work to generate mul funcs. * WIP: Outputting some simple C++. * Expose findOrEmitHoistableInst to IRBuilder to aid cloning, * Simplification for checking for BasicTypes. Test infrastructure compiles output C++ code. * Dot and mat/vec multiplication output. * First pass at swizzling. * First support for binary ops. * Builtin binary and unary functions. * Any and all. * WIP adding support for other functions. Added code to generate function signature. * Add scalar functions to slang-cpp-prelude.h * Support for most built in operations. * Tested first ternary. * Checking the emitting of corner cases functions - normalize, length, any, all, normalize, reflect. * Check asfloat etc work. * Fmod support. * WIP Array handling in C++. * First stage in being able to handl arbitrary type output for CLikeSourceEmitter * Removed Handler/Emitter split - so can implement more easily complex type naming. * Array passing by value first pass. * Rename Array -> FixedArray * Outputs structs in C++. * Emit the thread config. * Dimension -> TypeDimension * SpecializedOperation -> SpecializedIntrinsic Operation -> IntrinsicOp Use shared impl of isNominalOp Commented use of m_uniqueModule etc. * Add code to test slang->cpp when compiled doesn't have errors. Does so by building shared library and exporting the entry point. * Fix linux clang/gcc compile error about override not being specified. * Make sure c-cross-compile is run on linux targets/smoke. * Remove c-cross-compile.slang from smoke. * Fix running tests/cross-compile/c-cross-compile.slang on Ubuntu 16.04 * Only add -std=c++11 for C++ source.
2019-07-03Fix declaration of scalar sincos() function. (#996)Tim Foley
The function was accidentally defined with a generic `int` parameter copy-pasted from the vector definition, but that made the scalar version impossible to call with inferred generic arguments, because there wasn't a way to infer `N` when it isn't used in the parameter list. Includes a simple test case to confirm that the front-end no longer chokes on calls to scalar `sincos()`.
2019-06-21Parsing CPP Compiler output (#994)jsmall-nvidia
* Added extractLine line parsing to StringUtil. Use for matching lines instead of calcLines. calcLines uses extractLine to extract lines. Fixed problems found in output of some tests- due to how a how final line is handled. Now a final line has a \r or \n\r combination, but nothing else after it, it is considered the last line (not the line after it). * Use StringUtil::extractLine in slang-generate. * Improved comment on extractLine * Remove test code from StringUtil::extractLine * Made StringUtil::extractLine act as if line terminators are 'separators'. Added unit-test-string.cpp - to check behavior. * Adding LineParser - not entirely necessary, but slightly easier to use. * Hack to output start of tests. * WIP parsing CPPCompiler output. * Make extractLine return a bool. * First attempt at Visual Studio output parsing. * Add handling for checking error returning from CPPCompiler. * First pass parsing output of Gcc/Clang. * Split out VisualStudioCompilerUtil and GCCCompilerUtil. Simplified parsing of versions. * Simplify CPPCompiler::Output interface. * Fix problem with cpp-compiler on linux targets. * Add shared library link error. * Improving GCC/Clang parsing output. * Make cpp compiler parsing function able to return a SlangResult. * Handling for 'info' on clang * Add expected result for c-compile-shared-library-error.c * * Add flags such that link errors on shared libraries are supported. * Added StringUtil::join * Turn off the link shared library unfound symbol option on MacOS because it causes an error (and it's not needed on that target). * Add natvis inclusion back to visual studio projects. * Display message to try and determine crash problem on travisbuild. * Fix bug in handling continuations for clang. Disabled output of exception text. * WIP: See what clang is outputting that is parsing incorrectly on travis. * More handling for travis clang parsing issue. * Restore natvis to core.vcxproj * Fix visual studio project such that it still as natvis.
2019-06-19Start exposing a new COM-lite API (#987)Tim Foley
* Start exposing a new COM-lite API This change is mostly about exposing a new API to the Slang compiler that allows more fine-grained control over the compilation flow. The basic concepts in the new API are: * An `IGlobalSession` is the granularity at which we load/parse the Slang stdlib, and therefore gives applications a way to amortize startup cost for the library across multiple compiles. This is a concept that might be able to go away in a future version of Slang. * An `ISession` owns all the code that gets loaded/compiled/generated. Any `import`ed modules are shared across everything in a session (we don't re-parse/-check the code when we see another `import` for the same module). Any generic- or interface-based code in the session can be specialized using types from the same session (but not necessarily across sessions). * An `IModule` is the unit of code loading and scoping. It doesn't expose any API in this change, but would be the right scope for looking up types or entry points by name. * An `IProgram` is a "linked" combination of modules and entry points from which code can be generated and reflection information queried. This change re-uses the existing reflection API types, rather than introduce a new API that duplicates that functionality. That will probably change in a future revision. There are two major pieces of functionality added here that aren't related to the new API: * We now have an API concept of "entry point groups" which are one or more entry points that are intended to be used together so that they need to have non-overlapping parameters. For now this is being used to handle "hit groups" and local root signatures for ray tracing, but I'm not sure this is a concept we will keep in the long run. * We have a very special-case (client-application-specific) flag that ascribes special meaning to the `shared` keyword, so that it can be attached to global parameters to indicate that they are actually to be part of the local root signature rather than the global one for DXR. None of the API design (including naming) here is finalized; the only reason to check in the changes at this point to avoid having a long-running branch that leads to merge pain. Clients should *not* try to depend on the new API just yet, since it is still a work in progress. * fixup: clang warning * fixup: try to detect clang C++11 support * fixup * fixup * fixup * fixup * fixup: review feedback
2019-06-14Runtime Shared Library compilation and testing (#985)jsmall-nvidia
* Removed the need for VisualStudio specific CPPCompiler Improved the version parsing for gcc/clang Removed need for slang-unix-cpp-compiler-util.cpp/.h Remove binary before compiling in the compile c tests * Moved VisualStudio calcArgs into CPPCompilerUtil - as code is not windows specific. * Set up compile time version for gcc and clang * Fix compilation on OSX - use remove instead of unlink for file deletion. * On OSX - clang uses different string format. * Removed /bin/sh invoking as not required for OSX. * First pass working testing with shared libraries.
2019-06-13Runtime C++ compiling and testing on Unix-like targets (#981)jsmall-nvidia
* Work in progress to be able to invoke VS from within code. * First pass at windows version of refactor of OSProcessSpawner * Closer to getting VS path lookup working. * Make OSString assignable/ctor able * Work out program files directory directly, so don't have to expand %%. * WIP: Improve handling of process spawning. * Add support for splitting input by line. * * Correctly locates visual studio install * Added functionality to invoke vs via cmd * Add option to execute the command line. * Handle in ProcessUtil for windows -> WinHandle. * Rename files slang-win-visual-studio-util.cpp/.h and slang-process-util.h * First pass at unix/linux version of ProcessUtil. * Fix reading Visual Studio path from the registry. * Get compiling on linux with. * Fix vcvarsall.bat name * Use ProcessUtil to execute external code. * Remove OSProcessSpawner. * Remove includes for "os.h" where no longer needed. * Fix tabbing issue in premake5.lua Remove test code from slang-test-main.cpp * Fix premake4.lua tabbing issue. * Small fixes to slang-process-util.h Init ExecuteResult on Win execute. * Improve comments. * Fix bug in StringUtil::calcLines - with oddly terminated source input being able to read past end. Make slang-generate use StringUtil over it's own impl. * Fix off by one bug in working out Visual Studio version. * Fix bug in calculating Visual Studio Version * Fix compilation on linux with string parameter being passed to messageFormat. * Remove erroneous use of kOSError codes - use Result. * First effort to generate standard compiler options. * Initial efforts in compiling source code in test framework for VisualStudio. * Testing compiling c code on VisualStudio on Windows. * Fix warning on linux. * Fix clang on linux warning (and therefore failing) returning a StringBuilder as String. * Disable return-std-move on clang. * CommandLine arguments are now tagged if they are escaped or not. That it is the clients responsibility to escape command lines that cannot be automatically escaped. * Add checks on unix/linux that command line args are all unescaped. * WIP getting runtime GCC to work. * First pass compiler working on unix-like targets. * Enable c-compile.c test on 'smoke'.
2019-06-12Visual Studio compilation working in test harness (#979)jsmall-nvidia
* Work in progress to be able to invoke VS from within code. * First pass at windows version of refactor of OSProcessSpawner * Closer to getting VS path lookup working. * Make OSString assignable/ctor able * Work out program files directory directly, so don't have to expand %%. * WIP: Improve handling of process spawning. * Add support for splitting input by line. * * Correctly locates visual studio install * Added functionality to invoke vs via cmd * Add option to execute the command line. * Handle in ProcessUtil for windows -> WinHandle. * Rename files slang-win-visual-studio-util.cpp/.h and slang-process-util.h * First pass at unix/linux version of ProcessUtil. * Fix reading Visual Studio path from the registry. * Get compiling on linux with. * Fix vcvarsall.bat name * Use ProcessUtil to execute external code. * Remove OSProcessSpawner. * Remove includes for "os.h" where no longer needed. * Fix tabbing issue in premake5.lua Remove test code from slang-test-main.cpp * Fix premake4.lua tabbing issue. * Small fixes to slang-process-util.h Init ExecuteResult on Win execute. * Improve comments. * Fix bug in StringUtil::calcLines - with oddly terminated source input being able to read past end. Make slang-generate use StringUtil over it's own impl. * Fix off by one bug in working out Visual Studio version. * Fix bug in calculating Visual Studio Version * Fix compilation on linux with string parameter being passed to messageFormat. * Remove erroneous use of kOSError codes - use Result. * First effort to generate standard compiler options. * Initial efforts in compiling source code in test framework for VisualStudio. * Testing compiling c code on VisualStudio on Windows. * Fix warning on linux. * Fix clang on linux warning (and therefore failing) returning a StringBuilder as String. * Disable return-std-move on clang.
2019-05-31WIP: Support for other source target language (#971)jsmall-nvidia
* WIP: Setting up C/Cpp source compilation targets. * WIP: Emitting C/CPP. * WIP: Split out SourceSink, and use it for source output on emit. * SourceSink -> SourceStream * * Made SourceStream use m_ prefixing of members. * Make all methods use lower camel * Removed methods from SourceStream interface that are not used externally (use _ prefixing) * Improvements to documentation * EmitContext is now effectively empty, so just use SharedEmitContext as EmitContext. * SharedEmitContext -> EmitContext * Methods to LowerCamel in emit.cpp * Split out EmitContext and ExtensionUsageTracker into separate files. * Split out EmitVisitor into slang-c-like-source-emitter files. * EmitVisitor -> CLikeSourceEmitter * Tidy up around CLikeSourceEmitter - simplify header. * Small tidy up - removing repeated comments that are in header. * Remove EmitContext paramter threading. * Small tidy up. Use prefixed macros for slang-c-like-source-emitter.h * Small tidy up in slang-c-like-source-emitter.cpp * First pass at splitting out UnmangleContext. * MangledNameParser -> MangledLexer. * WIP making EmitOp (EOp) enum available outside of cpp * Generating EmitOpInfo from macro. * Split out emit precedence handling. Don't use kOp_ style anymore, just use an array indexed by EmitOp. * Disable C simple test for now. * Keep g++/clang happy with token pasting. * Fix win32 narrowing warning.
2019-05-22Translate .Load() to imageLoad() for Vulkan (#967)Tim Foley
* Translate .Load() to imageLoad() for Vulkan We were already emitting calls to `imageLoad()` and `imageStore` when a `RWTexture*` was used with `operator[]`: ```hlsl RWTexture2D<float> myTex; ... float value = myTex[xy]; // becomes an imageLoad myTex[xy] = value; // becomes an imageStore ``` However, we were *not* correctly handling the translation of an explicit `.Load()` operation: ```hlsl float value = myTex.Load(xy); ``` The `.Load()` operation was being translated to a GLSL `texelFetch` as it would be a for a `Texture2D`, and not to an `imageLoad()` as would make sense for a `RWTexture2D` (which becomes a GLSL `image2D`). This fix is confined to the stdlib, and is mostly a matter of emitting either `texelFetch` or `imageLoad` as the GLSL function name depending on the "access" of the resource type. It is messy code, but straightforward. One extra detail was that there had been logic to emit a `, 0` argument in the `texelFetch` calls in the non-read-only case, because `texelFetch` usualy requires an explicit mip-level argument and `.Load()` on a `RWTexture*` doesn't recieve an LOD parameter. This is a non-issue now that we are calling `imageLoad()` instead, because `imageLoad` doesn't need/want the extra argument. * fixup: change test baseline based on recent GLSL output changes * fixup: review feedback
2019-05-22Hotfix/improve glsl semantic conversion review (#968)jsmall-nvidia
* Small changes based on review * Remove the explicit 'nominal' tests * Made isValueEqual and isEqual on on IRConstant take a pointer * Small improvements to comments, and clarity of using 'nominal' * Simplify comparison by just using isTypeOperandEqual as basis for isTypeEqual * Use cross compile to test half-texture.slang on glsl * Don't need half-texture.slang.expected * Fix handling of nominal comparison based on review, ensuring that for nominal insts, they can only be compared by pointer.