Generalize heterogenous code emit (#1968)

* Bring heterogeneous-hello-world back up to date.

* Reintroduced heterogeneous-hello-world into the premake

* No longer uses compiled bytecode for entry point, instead a loadModule
call is hardocoded with the slang file name.

* Entry point is, similarly, hardcoded for now.

* Added a bypass to slang-legalize-types for an unneeded GPUForeach check

* Run premake and change to relative path

* Removed experimental and added README

* Add prebuild command to premake for heterogeneous example

* Pass in entry point as parameter (also remove shader bytecode)

* Pass in module name as parameter

* Squashed commit of the following:

commit 5b13b57fe600724344c556fe4309a5d6bb3d39ab
Author: Kai Yao <kyao@nvidia.com>
Date:   Thu Oct 7 23:38:50 2021 -0700

    Return diagnostics data when encountering module load error by exception (#1966)

commit 112e1515c30fa972ff56f91514b70946153c718c
Author: jsmall-nvidia <jsmall@nvidia.com>
Date:   Thu Oct 7 16:12:29 2021 -0400

    Disable test crashing CI (#1965)

    * #include an absolute path didn't work - because paths were taken to always be relative.

    * Disable test that appears to be crashing.

commit da32069a0c1c8c723d7ef45100049a8f0dd5d9c4
Author: Kai Yao <kyao@nvidia.com>
Date:   Mon Oct 4 13:58:51 2021 -0700

    Modified barrier API to accept multiple resources per call (#1959)

    Co-authored-by: Yong He <yonghe@outlook.com>

commit 97bb82ebcdf8f1391b9d93b5a8d7b1dfc4e88e52
Author: jsmall-nvidia <jsmall@nvidia.com>
Date:   Mon Oct 4 14:15:51 2021 -0400

    Removing exceptions from core/compiler-core (#1953)

    * #include an absolute path didn't work - because paths were taken to always be relative.

    * Refactor Stream. Working on all tests.

    * Split out CharEncode.

    * Make method names lower camel.
    m_prefix in Writer/Reader

    * Tidy up around CharEncode interface.

    * Small improvements around encode/decode.

    * Better use of types.

    * Remove readLine from TextReader.

    * Remove exceptions from Stream/Text handling.

    * Fix some typos.

    * Fix tabbing.

    * Fix missing override.

    * Remove remaining exception throw/catch via using signal mechanism.

    * Remove exceptions that are not used anymore.

    * Document the Stream interface.

    * Remove index for decoding 'get byte' function.

    * Fix CharReader -> ByteReader.

commit b3dfe383c6d31ff3dbd76dcfb32de8d536382f3e
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Mon Oct 4 09:46:33 2021 -0700

    Get native handles for TextureResource and BufferResource (#1960)

    * Added getNativeHandle() to TextureResource and BufferResource; Implemented getNativeHandle() in Vulkan and D3D12; Added new unit test files for the aforementioned implementation

    * Added missing getNativeHandle() implementations to renderer-shared.cpp and CUDA

    * Finished new getNativeHandle() unit tests for ITextureResource and IBufferResource; Modified ICommandQueue and ICommandBuffer unit tests to call QueryInterface to convert to IUnknown then back and compare resulting pointers for equality

    * Unit tests updated and pass locally

    * Cast m_buffer.m_buffer and m_image to uint64_t

commit 35bca4cc432613af3926da3bed217a6baa9cbd26
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Fri Oct 1 13:08:25 2021 -0700

    Add getNativeHandle() to ICommandQueue and ICommandBuffer (#1952)

    * Added support for getting command buffer and command queue handles to ICommandBuffer and ICommandQueue; D3D12Device, VkDevice, and DebugDevice modifieid to implement this new functionality; immediate-renderer-base.cpp also modified to implement the new functions

    * Removed excess boilerplate

    * Changed readRef() to get() in D3D12 getNativeHandle() implementation for ICommandBuffer and ICommandQueue

    * Added unit tests for new getNativeHandle() implementations, unfinished

    * Queue test added; Minor cleanup changes

    * getBufferHandleTestImpl() now closes the command buffer before returning

    * Added getNativeHandle() implementations to CUDADevice

    * Added comment clarifying that the Vulkan check is checking for a null handle, which is defined to be 0

commit 6c6200f547c7387598743b23bb3c8f0d375d9494
Author: Kai Yao <kyao@nvidia.com>
Date:   Thu Sep 30 20:25:34 2021 -0700

    VK Resource Barrier (#1955)

    * Resource barrier API and VK implementation

    * Stub implementations

    * Handle VK Acceleration Structure flag

    * Add a couple more cases to pipeline barrier stages

commit 627fc976bac5c2381dbace9c7925cb6a68b8de12
Author: Yong He <yonghe@outlook.com>
Date:   Thu Sep 30 19:48:47 2021 -0700

    Fix aarch64 build on github (#1957)

    Co-authored-by: Yong He <yhe@nvidia.com>

commit 122d701513e116856bd59c999221ce36a373d7db
Author: Yong He <yonghe@outlook.com>
Date:   Thu Sep 30 17:51:56 2021 -0700

    Fix GitHub release (#1956)

    * Fix aarch64 release build config.

    * Fix for WinAarch64 build.

    * Update premake for embed-std-lib build on aarch64.

    * `platform` fix for aarach64 build.

    * Try revert back to use absolute output path for slang-stdlib-generated.h

    * Fix

    * fix

    Co-authored-by: Yong He <yhe@nvidia.com>

commit aa8f7b899b7b562b3d3c6e25c3da41569505e70c
Author: Chad Engler <englercj@live.com>
Date:   Wed Sep 29 13:02:47 2021 -0700

    Fix ARM64 detection for MSVC (#1951)

commit 6736b0c1c5fa3e89bc561eb7965a1a0d17af3466
Author: Yong He <yonghe@outlook.com>
Date:   Wed Sep 29 11:29:46 2021 -0700

    Add ISession::loadModuleFromSource. (#1950)

    Co-authored-by: Yong He <yhe@nvidia.com>

commit d8e452412e14a6a8ba137f2adcae13b398e5cecb
Author: Yong He <yonghe@outlook.com>
Date:   Tue Sep 28 15:03:03 2021 -0700

    Fix AbortCompilationException leaking through loadModule API. (#1949)

    * Fix AbortCompilationException leaking through loadModule API.

    * Update.

    * Fix.

    Co-authored-by: Yong He <yhe@nvidia.com>

commit cdf1b2c007fefdca128584d2a9f63dec3d350e16
Author: Yong He <yonghe@outlook.com>
Date:   Tue Sep 28 11:54:24 2021 -0700

    Improvements to the unit test framework. (#1948)

commit af788b62e18bbd55cd748ad60400a74cf1bc93ee
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Fri Sep 24 16:53:41 2021 -0700

    Add existing device handle support unit test (#1946)

commit bec8e6aec85b6e3f875c58bdd59eb15613978358
Author: Yong He <yonghe@outlook.com>
Date:   Fri Sep 24 11:33:44 2021 -0700

    Move existing unit tests to a standalone dll. (#1945)

commit f2a3c933bc11a498c622fa18694c84beca8ca031
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Thu Sep 23 12:19:49 2021 -0700

    Add method to retrieve native handles (#1944)

    * Added a getNativeHandle() method that retrieves the natively created handles; Modified RendererBase, VKDevice, D3D12Device, and DebugDevice to implement this new method

    * Moved ExistingDeviceHandles out of Desc directly inside IDevice and renamed to NativeHandles; Modified calls accessing the struct accordingly in RendererBase, DebugDevice, VKDevice, and D3D12Device

    * Minor cleanup changes (renames, etc.)

commit b9b398d038b524f15a86ff27cd6888d54e8754e0
Author: Yong He <yonghe@outlook.com>
Date:   Wed Sep 22 10:06:59 2021 -0700

    Add gfx unit testing framework. (#1943)

    * Add gfx unit testing framework.

    * Fix compilation error.

    * Reset gfxDebugCallback after render_test.

    * Pass enabledApi flags through.

    * Fix for code review suggestions.

    Co-authored-by: Yong He <yhe@nvidia.com>

commit 6e9cee69b3588ddae09b08b9f580f59ad899983f
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Tue Sep 21 18:46:32 2021 -0700

    Support for existing device/instance handles in Vulkan (#1942)

commit b1f04c8544c650de3947955ca68f679535d249aa
Author: lucy96chen <47800040+lucy96chen@users.noreply.github.com>
Date:   Wed Sep 15 20:22:45 2021 -0700

    Allow D3D12Device to use an existing device handle (#1940)

    * Added a new field for an existing device handle to IDevice::Desc; Modified D3D12Device::initialize to set the device stored in desc if it already exists instead of creating a new one

    * Turned existingDeviceHandle into a struct containing an array of two elements; Updated D3D12Device::initialize to match changes to existingDeviceHandle; Updated comments

    * Fixed style error for ExistingDeviceHandles struct

commit 2f7b9f5ae8be21c6c1d75ae9caefbc7b3f8986a9
Author: Pablo Delgado <private@pablode.com>
Date:   Thu Sep 16 01:17:57 2021 +0200

    Fix incorrect WIN32 macros and missing Windows.h inclusion (#1939)

    * Replace WIN32 preprocessor macros with _WIN32

    * Add missing Windows.h include for InterlockedIncrement

commit 11d43642008905ac69a3832eb8a9b2ae7b785f86
Author: Yong He <yonghe@outlook.com>
Date:   Tue Sep 14 11:36:44 2021 -0700

    Avoid upcasting to f32 in 16bit float-uint bit cast. (#1938)

    Co-authored-by: Yong He <yhe@nvidia.com>

commit 502aa3812a82cf0d091cff0c67804e4ee448ac78
Author: David Siher <32305650+dsiher@users.noreply.github.com>
Date:   Tue Sep 14 12:59:55 2021 -0400

    Bring heterogeneous-hello-world back up to date. (#1935)

    * Bring heterogeneous-hello-world back up to date.

    * Reintroduced heterogeneous-hello-world into the premake

    * No longer uses compiled bytecode for entry point, instead a loadModule
    call is hardocoded with the slang file name.

    * Entry point is, similarly, hardcoded for now.

    * Added a bypass to slang-legalize-types for an unneeded GPUForeach check

    * Run premake and change to relative path

    * Removed experimental and added README

    Co-authored-by: Yong He <yonghe@outlook.com>

* Revert "Squashed commit of the following:"

This reverts commit 4f665858d65f7c332c616ef6db9fdafa1c5e0b9f.

* Run premake

* Remove prebuild command (only works on Windows?)

* Rerun premake

* Fix heterogeneous prebuild command

* Remove linux specific prebuild command

* Fix prebuild command (again)

* Change target from dxbc to hlsl to see if that fixes linux issues

* Use Path::getFileNameWithoutExt

* Change string-literal.slang.expected to have extra filename in decoration

Co-authored-by: Yong He <yonghe@outlook.com>

3 files changed, 8 insertions, 43 deletions

diff --git a/examples/heterogeneous-hello-world/main.cpp b/examples/heterogeneous-hello-world/main.cpp
index 9e0bb8b0f..05d1e4787 100644
--- a/examples/heterogeneous-hello-world/main.cpp
+++ b/examples/heterogeneous-hello-world/main.cpp
@@ -62,7 +62,7 @@ gfx::IDevice* createDevice()
 
 // Loads the shader code defined in `shader.slang` for use by the `gfx` layer.
 //
-gfx::IShaderProgram* loadShaderProgram(gfx::IDevice *device)
+gfx::IShaderProgram* loadShaderProgram(gfx::IDevice *device, String entryPoint, String moduleName)
 {
     // We need to obtain a compilation session (`slang::ISession`) that will provide
     // a scope to all the compilation and loading of code we do.
@@ -83,14 +83,14 @@ gfx::IShaderProgram* loadShaderProgram(gfx::IDevice *device)
     // already been loaded previously, that would be used directly.
     //
     ComPtr<slang::IBlob> diagnosticsBlob;
-    slang::IModule *module = slangSession->loadModule("shader", diagnosticsBlob.writeRef());
+    slang::IModule *module = slangSession->loadModule(moduleName.getBuffer(), diagnosticsBlob.writeRef());
     diagnoseIfNeeded(diagnosticsBlob);
     if (!module)
         return NULL;
 
-    // Look up entry point (hardcoded for now)
+    // Look up entry point
     //
-    char const *computeEntryPointName = "computeMain";
+    char const *computeEntryPointName = entryPoint.getBuffer();
     ComPtr<slang::IEntryPoint> computeEntryPoint;
     SLANG_RETURN_NULL_ON_FAIL(
         module->findEntryPointByName(computeEntryPointName, computeEntryPoint.writeRef()));
@@ -270,9 +270,9 @@ gfx_BufferResource_0* createStructuredBuffer_0(gfx_Device_0* _0, FixedArray<floa
     return (gfx_BufferResource_0*)createStructuredBuffer((gfx::IDevice*)_0, (float*)&_1);
 }
 
-gfx_ShaderProgram_0* loadShaderProgram_0(gfx_Device_0* _0)
+gfx_ShaderProgram_0* loadShaderProgram_0(gfx_Device_0* _0, char* _1, char* _2)
 {
-    return (gfx_ShaderProgram_0*)loadShaderProgram((gfx::IDevice*)_0);
+    return (gfx_ShaderProgram_0*)loadShaderProgram((gfx::IDevice*)_0, _1, _2);
 }
 
 gfx_ResourceView_0* createBufferView_0(gfx_Device_0* _0, gfx_BufferResource_0* _1)
diff --git a/examples/heterogeneous-hello-world/shader.cpp b/examples/heterogeneous-hello-world/shader.cpp
index 0c0c24ebc..3f08af05f 100644
--- a/examples/heterogeneous-hello-world/shader.cpp
+++ b/examples/heterogeneous-hello-world/shader.cpp
@@ -28,45 +28,10 @@ Vector<uint32_t, 3> make_VecU3(uint32_t a, uint32_t b, uint32_t c)
     return Vector<uint32_t, 3>{ a, b, c};
 }
 
-size_t __computeMainSize = 668;
-unsigned char __computeMain[] = {68, 88, 66, 67, 87, 111, 81, 164, 2, 29, 72, 42, 151, 28, 13, 217, 55, 37, 7, 95, 1, 
-0, 0, 0, 156, 2, 0, 0, 5, 0, 0, 0, 52, 0, 0, 0, 8, 1, 0, 0, 24, 
-1, 0, 0, 40, 1, 0, 0, 32, 2, 0, 0, 82, 68, 69, 70, 204, 0, 0, 0, 1, 
-0, 0, 0, 88, 0, 0, 0, 1, 0, 0, 0, 28, 0, 0, 0, 0, 4, 83, 67, 0, 
-9, 16, 0, 164, 0, 0, 0, 60, 0, 0, 0, 6, 0, 0, 0, 6, 0, 0, 0, 1, 
-0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 101, 
-110, 116, 114, 121, 80, 111, 105, 110, 116, 80, 97, 114, 97, 109, 115, 95, 105, 111, 66, 117, 
-102, 102, 101, 114, 95, 48, 0, 60, 0, 0, 0, 1, 0, 0, 0, 112, 0, 0, 0, 4, 
-0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 136, 0, 0, 0, 0, 0, 0, 0, 4, 
-0, 0, 0, 2, 0, 0, 0, 148, 0, 0, 0, 0, 0, 0, 0, 36, 69, 108, 101, 109, 
-101, 110, 116, 0, 171, 171, 171, 0, 0, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 
-0, 0, 0, 77, 105, 99, 114, 111, 115, 111, 102, 116, 32, 40, 82, 41, 32, 72, 76, 83, 
-76, 32, 83, 104, 97, 100, 101, 114, 32, 67, 111, 109, 112, 105, 108, 101, 114, 32, 49, 48, 
-46, 49, 0, 73, 83, 71, 78, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 79, 
-83, 71, 78, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 83, 72, 69, 88, 240, 
-0, 0, 0, 64, 0, 5, 0, 60, 0, 0, 0, 106, 8, 0, 1, 158, 0, 0, 4, 0, 
-224, 17, 0, 0, 0, 0, 0, 4, 0, 0, 0, 95, 0, 0, 2, 18, 0, 2, 0, 104, 
-0, 0, 2, 1, 0, 0, 0, 155, 0, 0, 4, 4, 0, 0, 0, 1, 0, 0, 0, 1, 
-0, 0, 0, 167, 0, 0, 8, 18, 0, 16, 0, 0, 0, 0, 0, 10, 0, 2, 0, 1, 
-64, 0, 0, 0, 0, 0, 0, 6, 224, 17, 0, 0, 0, 0, 0, 49, 0, 0, 7, 34, 
-0, 16, 0, 0, 0, 0, 0, 10, 0, 16, 0, 0, 0, 0, 0, 1, 64, 0, 0, 0, 
-0, 0, 63, 0, 0, 0, 7, 66, 0, 16, 0, 0, 0, 0, 0, 10, 0, 16, 0, 0, 
-0, 0, 0, 10, 0, 16, 0, 0, 0, 0, 0, 75, 0, 0, 5, 18, 0, 16, 0, 0, 
-0, 0, 0, 10, 0, 16, 0, 0, 0, 0, 0, 55, 0, 0, 9, 18, 0, 16, 0, 0, 
-0, 0, 0, 26, 0, 16, 0, 0, 0, 0, 0, 42, 0, 16, 0, 0, 0, 0, 0, 10, 
-0, 16, 0, 0, 0, 0, 0, 168, 0, 0, 8, 18, 224, 17, 0, 0, 0, 0, 0, 10, 
-0, 2, 0, 1, 64, 0, 0, 0, 0, 0, 0, 10, 0, 16, 0, 0, 0, 0, 0, 62, 
-0, 0, 1, 83, 84, 65, 84, 116, 0, 0, 0, 7, 0, 0, 0, 1, 0, 0, 0, 0, 
-0, 0, 0, 1, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
-0, 0, 0, 0, 0, 0, 0, };
 void computeMain_wrapper(gfx_Device_0* device, Vector<uint32_t, 3> gridDims, 
 	RWStructuredBuffer<float> buffer)
 {
-	gfx_ShaderProgram_0* shaderProgram = loadShaderProgram_0(device);
+	gfx_ShaderProgram_0* shaderProgram = loadShaderProgram_0(device, "computeMain", "shader");
 	gfx_TransientResourceHeap_0* transientHeap = buildTransientHeap_0(device);
 	gfx_PipelineState_0* pipelineState = buildPipelineState_0(device, shaderProgram);
 	gfx_ResourceView_0* bufferView = createBufferView_0(device, unconvertBuffer_0(buffer));
diff --git a/examples/heterogeneous-hello-world/shader.slang b/examples/heterogeneous-hello-world/shader.slang
index b66640e3d..d67fd5582 100644
--- a/examples/heterogeneous-hello-world/shader.slang
+++ b/examples/heterogeneous-hello-world/shader.slang
@@ -29,7 +29,7 @@ namespace gfx {
 // Forward declarations of cpp functions
 //
 Ptr<gfx::Device> createDevice();
-Ptr<gfx::ShaderProgram> loadShaderProgram(Ptr<gfx::Device> device);
+Ptr<gfx::ShaderProgram> loadShaderProgram(Ptr<gfx::Device> device, String entryPoint, String module);
 Ptr<gfx::BufferResource> createStructuredBuffer(
     Ptr<gfx::Device> device,
     float[4] initialData);