Making it easier to work with shaders https://git.yummers.dev/slang.git/atom?h=master 2019-05-31T21:20:37+00:00 2019-05-31T21:20:37+00:00 jsmall-nvidia jsmall@nvidia.com 2019-05-31T21:20:37+00:00 urn:sha1:6cbc3929a54d37bd23cb5efa8e3320ba02f78b2f * Prefixing source files in source/slang with slang- * Prefix source in source/slang with slang- prefix. * Rename core source files with slang- prefix. * Update project files. * Fix problems from automatic merge. 2019-04-29T21:03:46+00:00 jsmall-nvidia jsmall@nvidia.com 2019-04-29T21:03:46+00:00 urn:sha1:4880789e3003441732cca4471091563f36531635 * List made members m_ Tweaked types to closer match conventions. * Use asserts for checking conditions on List. Other small improvements. * List<T>.Count() -> getSize() * List<T> Add -> add First -> getFirst Last -> getLast RemoveLast -> removeLast ReleaseBuffer -> detachBuffer GetArrayView -> getArrayView * List<T>:: AddRange -> addRange Capacity -> getCapacity Insert -> insert InsertRange -> insertRange AddRange -> addRange RemoveRange -> removeRange RemoveAt -> removeAt Remove -> remove Reverse -> reverse FastRemove -> fastRemove FastRemoveAt -> fastRemoveAt Clear -> clear * List<T> FreeBuffer -> _deallocateBuffer Free -> clearAndDeallocate SwapWith -> swapWith * List<T> SetSize -> setSize Reserve -> reserve GrowToSize growToSize * UnsafeShrinkToSize -> unsafeShrinkToSize Compress -> compress FindLast -> findLastIndex FindLast -> findLastIndex Simplify Contains * List<T> Removed m_allocator (wasn't used) Swap -> swapElements Sort -> sort Contains -> contains ForEach -> forEach QuickSort -> quickSort InsertionSort -> insertionSort BinarySearch -> binarySearch Max -> calcMax Min -> calcMin * Initializer::Initialize -> initialize List<T>:: Allocate -> _allocate Init -> _init IndexOf -> indexOf * * Put #include <assert.h> in common.h, and remove unneeded inclusions * Small refactor of ArrayView - remove stride as not used * getSize -> getCount setSize -> setCount unsafeShrinkToSize->unsafeShrinkToCount growToSize -> growToCount m_size -> m_count * Some tidy up around Allocator. * Use Index type on List. * Refactor of IntSet. First tentative look at using Index. * Made Index an Int Did preliminary fixes. Made String use Index. * Partial refactor of String. * String::Buffer -> getBuffer ToWString -> toWString * Small improvements to String. String:: Buffer() -> getBuffer() Equals() -> equals * Try to use Index where appropriate. * Fix warnings on windows x86 builds. 2019-03-12T19:24:03+00:00 Tim Foley tfoleyNV@users.noreply.github.com 2019-03-12T19:24:03+00:00 urn:sha1:3cfccfd4991df01deaf132f11b4eaa6848a32c4e The short version for command-line users is: * Use `-g` to get debug info in the output, where supported * Use `-O0` to disable optimizations, in case that improves debugability * Use `-O2` for optimized/release builds where you can spend the extra compile time The command-line options are matched with new API functions `spSetDebugInfoLevel()` and `spSetOptimizationLevel()` that set the equivalent information. Right now these settings only affect how we invoke fxc and dxc. In the longer run I expect we will want to use them to control other things: * Once we are emitting our own SPIR-V, the `-g` option should control what source-level name information we include in it. * Whether or not `-g` is used could be used to decide whether we preserve the "name hints" in the IR, which in turn decide whether we output GLSL/HLSL source that uses names based on the original program. * We will eventually need/want to include some amount of optimization passes on the Slang IR, and the `-O` options should control which of those passes are enabled on a particular invocation. In this change I decided to expose the options at the level of the entire compile request for API users, and to store the actual information on the Linkage. We might want to revisit this decision and instead allow for the level of optimization to be chosen per-target as part of back-end state. Similarly, we might want to have more fine-grained control over the level of debug output per-target (although we'd still need a front-end setting to determine what debug info is generated into the Slang IR). 2019-02-15T17:08:19+00:00 Tim Foley tfoleyNV@users.noreply.github.com 2019-02-15T17:08:19+00:00 urn:sha1:a3fd4e2bc40cfc77db953b14744c30e7a18e7c1d * Split front- and back-ends This change is a major refactor of several of the types that provide the behind-the-scenes implementation of the public C API. The goal of this refactor is primarily to allow for future API services that let the user operate both the front- and back-ends of the compiler in a more complex fashion. For example, as user should be able to compile a bunch of source code into modules, look up types, functions, etc. in those modules, specialize generic types/functions to the types they've looked up, and then finally request target code to be gernerated for specialized entry points. The back-end code generation they trigger should re-use the front-end compilation work (parsing, semantic checking, IR generation) that was already performed. The most visible change is that `CompileRequest` has been split up into several smaller types that take responsibility for parts of what it did: * The `Linkage` type owns the storage for `import`ed modules, and well as the `TargetRequest`s that represent code-generation targets. The intention is that an application could use a single `Linkage` for the duration of its runtime (so long as it was okay with the memory usage), so that each `import`ed module only gets loaded once. For now, this type needs to manage the search paths, file system, and source manager, because of its responsibility for loading files. * A `FrontEndCompileRequest` owns the stuff related to parsing, semantic checking, and initial IR generation. This most notably includes the `TranslationUnitRequest`s and the `FrontEndEntryPointRequest`s (which used to be just `EntryPointRequest`s). It's main job is to produce AST and IR modules for each translation unit, and to find and validate the entry points. The front-end request does *not* interact with generic arguments for global or entry-point generic parameters. * The main output of both `import` operations and front-end translation units is the `Module` type, which is just a simple container for both the AST module (to service the reflection/layout APIs, and also for semantic checking of code that `import`s the module) and the IR module (for linking and code generation). This type captures the commonalities between the old `LoadedModule` (which is now just an alias for `Module`) and `TranslationUnitRequest` (which now owns a `Module`). * The secondary output of front-end compilation is a `Program`, which comprises a list of referenced `Module`s and validated `EntryPoint`s that will be used together. Layout and code generation both need a `Program` to tell them what modules and entry points will be used together (we don't want to just code-gen everythin that has ever been loaded into the linakge). The `Program`s created by the front-end do not include generic arguments, so they may provide incomplete layout information and/or be unsuitable for code generation. * A `BackEndCompileRequest` owns stuff related to turning a `Program` into output kernels for the targets of a `Linkage`. Most of the data it owns beyond the `Program` to be compiled is minor, so this is a good candidate for demotion from a heap-allocated object to just a `struct` of options that gets passed around. * The `CompileRequestBase` type is an attempt to wrap up the common functionality of both front-end and back-end compile requests. Most of it is just exposing the availability of a linkage and `DiagnosticSink`, so this type is a good candidate for subsequent removal. The main interesting thing it has is the flags related to dumping and validation of IR, so there is probably a good refactoring still to be made around deciding how options should be handled going forward. * Behind the scenes, the `Program` type is set up to handle some level of on-line compilation and layout work. The `Program` knows the `Linkage` it belongs to, and allows for a `TargetProgram` to be looked up based on a specific `TargetRequest`. A `TargetProgram` then allows layout information and compiled kernel code to be asked for on-demand, in order to support eventual "live" compilation scenarios. * The `EndToEndCompileRequest` type is a composition/coordination type that replaces the old `CompileRequest` in a way that uses the services of the various other types. It owns a few pieces of state that only make sense in the context of an end-to-end compile (e.g., there is really no way to "pass through" code when the front- and back-ends are run separately) or a command-line compile (everything to do with specifying output paths for files is really just for the benefit of `slangc`, and might even be moved there over time). * One important detail is that the `EndToEndCompilRequest` owns all of the string-based generic arguments for both global and entry-point generic parameters. The logic in `check.cpp` for dealing with those arguments has been heavily refactored to separate out the parsings steps that are specific to end-to-end compilation with string-based type arguments, and the semantic checking steps that result in a specialized `Program` (which can be exposed through new APIs that aren't tied to end-to-end compilation). It is perhaps not surprising that this change had a lot of consequences, so I'll briefly run over some of the main categories of changes required: * I changed the way that global generic arguments are passed via API (use `spSetGlobalGenericArgs` instead of the generic arguments for `spAddEntryPointEx`, which are not just for entry-point generics), which has been a change that we've needed for a long time. This is technically a breaking API change, although we should have very few client applications that care about it. * A bunch of places that used to take "big" objects like `CompileRequest` now just take the sub-pieces they care about (e.g., a function might have only needed a `Linkage` and a `DiagnosticSink`). This makes many subroutines or "context" struct types more generally useful, at the cost of taking more parameters. * In a few cases the conceptually clean separation of the layers breaks down (often for edge-case or compatibility features), and so we may pass along additional objects that are allowed to be null, but are used when present. A big example of this is how the back-end code generation routines accept an `EndToEndCompileRequest` that is optional, and only used to check whether "pass through" compilation is needed. We should probably look into cleaning this kind of logic up over time so that we don't need to violate the apparent separation of phases of compilation. * In cases where separation of layers was being broken for the sake of GLSL features, I went ahead and ripped them out, since all of that should be dead code anyway. * In many cases I increased the encapsulation of data in the core types to help track down use sites and make sure they are following invariants better. * In cases where code was doing, e.g., `context->shared->compileRequest->session->getThing()` I have tried to introduce convenience routines so that the usage site is just `context->getThing()` to improve encapsulation and allow changes to be made more easily going forward. * The `noteInternalErrorLoc` functionality was moved off of the compile request and into `DiagnosticSink`, since that is the one type you can rely on having around when you want to note an internal error. We may consider going forward if (and how) it should reset the counter used for noting locations on internal errors. * A few APIs now take `DiagnosticSink*` arguments where they didn't before, and as a result some public APIs need to create `DiagnosticSink`s to pass in, before going ahead and ignoring the messages. In the future there should be variations of these APIs that accept an `ISlangBlob**` parameter for the output. * fixup: missing include for compilers with accurate template checking (non-VS) * fixup: review feedback 2019-01-17T02:49:38+00:00 jsmall-nvidia jsmall@nvidia.com 2019-01-17T02:49:38+00:00 urn:sha1:0db6e249293b85338acb77f8858c823422949c3f * * Allow dxc compilation to take place if dxil is not found. * Output a warning that output will not be signed. * Remove .dll from dxil in warning so more applicable cross platform. 2019-01-16T18:31:42+00:00 jsmall-nvidia jsmall@nvidia.com 2019-01-16T18:31:42+00:00 urn:sha1:86e11e0e111fab60b9517056ac049bfac6e3bd25 * Added support for converting SlangResult to string in PlatformUtil. * * Added reportExternalCompilerError * Made external compilers use this * Made DiagnosticSink accept UnownedStringSlice * Made emitXXX compiler functions return SlangError * Use smart pointers to handle life of Com interfaces * * Make SlangResult compatible with HRESULT for some common cases. * Make PlatformUtil::appendResult return SlangResult * Compile check SLANG_RESULT. * Add tests for checking diagnostics from external compilers. * * Make external compiler tests only run on windows for now. * Added 'windows' and 'unix' categories * Added categories based on what backends are available. Will make more tests run on linux and handle case where dxcompiler is not available on appveyor. * * Added spSessionCheckPassThroughSupport * Use to determine whats available for categories for tests * Add support for outputting source filename/s when using pass through. 2018-11-13T22:02:27+00:00 jsmall-nvidia jsmall@nvidia.com 2018-11-13T22:02:27+00:00 urn:sha1:cfb1f61d478c92026718cedca4bfec8c2afc1b00 * Make the disassembly methods returns SlangResult and String as last output param so as to make error case clear. 2018-11-06T19:28:25+00:00 jsmall-nvidia jsmall@nvidia.com 2018-11-06T19:28:25+00:00 urn:sha1:1185bd464092f372430cbfaa15a7be4dcaa90752 * * Added ISlangSharedLibraryLoader and ISlangSharedLibrary * Implemented default implementations * Added slang API function to get/set the ISlangSharedLibraryLoader on the session * Put function caching onto the Session - so that if the loader is chaged, its easy to reset the shared libraries, and functions * Run premake. * Fix problem with setting null, would cause an unnecessary function/shared lib flush. * * Unload SharedLibrary when DefaultSharedLibrary is deleted. * Make SharedLibrary handle unload safely if already unloaded. * Refactor SharedLibrary, such that it becomes a utility class - simplifying it's semantics. * Simplified ISlangSharedLibrary such that doesn't have unload and isLoaded so easier to implement. Use updated SharedLibrary impl. * Disable aarch64 on windows * Premake windows files without aarch64 build. * Moved slang-shared-library to core (so can be used in code outside of main slang) Fixed problem in premake5 where on windows projects were incorrectly constructed * Allowed RefObject to base class of com types Added ConfigurableSharedLibraryLoader Added -dxc-path -fxc-path -glslang-path Fix problem with dxc-path not honoring it's path when loading dxil * Added documentation for command line control of dll loading paths. * Remove some tabbing issues. * Change name of include guard. 2018-11-01T22:08:54+00:00 Tim Foley tfoleyNV@users.noreply.github.com 2018-11-01T22:08:54+00:00 urn:sha1:453331951b0df2a612f9bc0d68eab2ad786ec5bf This change adds an API function and command line options for controlling the default floating-point behavior for a target, with options for "fast" and "precise" computation. The "precise" option gets mapped to the "IEEE strictness" mode in `fxc` and `dxc` (there is currently no equivalent option for glslang that I could find). 2018-11-01T20:23:45+00:00 Tim Foley tfoleyNV@users.noreply.github.com 2018-11-01T20:23:45+00:00 urn:sha1:82f57ae3561e9f9e8ece8ab6c12c2b318580bc38 I had hoped to just disable certain warnings (false positives that trigger based on how Slang outputs HLSL code), but dxc doesn't support fine-grained control over warnings from the command line (we might investigate `#pragma`-based options later). There are enough user complaints about how downstream compiler errors come mixed with tons of distracting warnings that disabling them will lead to a better user experience for now. This change also fixes a long-standing bug where apparently dxc does *not* guaranteee that the diagnostic blob it returns is nul-terminated (despite this convention being established by fxc), so that we need to nul-terminate it ourselves before emitting any diagnostic messages produced by dxc.