Support for [__extern] attribute (#1111)

* Added RiffReadHelper

* Move type to fourCC in Chunk simplifies some code.

* Make MemoryArena able to track external blocks.
Allow ownership of Data to vary.
Changed IR serialization to use moved allocations to avoid copies.

As it turns out all of the array writes could use unowned data, but doing so requires the IRData to stay in scope longer than IRSerialData, which it does at the moment - but perhaps needs better naming or a control for the feature.

* Write out slang-module container.

* WIP on -r option.
Loading modules - with -r.

* Making the serialized-module run (without using imported module).

* Split compiling module from the test.

* Separate module compilation with a function working.

* Remove serialization test as not used.

* Fix warning on gcc.

* Updated test to have types across module boundary.

* Allow entry point declaration.
A test that tries to build with just an entry point declaration and a module.

* Try to make link work with multiple modules.

* Multi module linking first pass working.

* Multi module test working with -module-name option

* Added feature to repro manifest of approximation of command line that was used.

* Use isDefinition - for determining to add decorations to entry point lowering.

* Added support for repo-file-system.h
More precise control of CacheFileSystem.
Allow RelativeFileSystem to strip paths optionally.
Use canonical paths in PathInfo cache.
Fix bug in -D options for command line output of StateSerailizeUtil

* Add missing slang-options.h

* Fix bug in bit slang-state-serialize.cpp with bit removal.

* Added documentation around -repro-file-system
Added spLoadReproAsFileSystem function.

* Fix warning.

* spAddLibraryReference

* * Add support for slang-lib extension
* Container output when using -no-codegen option

* Use the m_containerFormat to determine if the module container is constructed.
Store the result in a blob. This allows for potential access via the API.
Write the blob if a filename is set.
Use m_ prefix for container variables.

* Added spGetContainerCode.
Made spGetCompileRequestCode work.

* * Put obfuscateCode on linkage
* Remove obfuscation from variable names - as can be achieved by either stripping and/or removing NameHintDecorations at lowering
* Remove name hints being added during lowering
* Add stripping of SourceLoc location in strip phase

* Hashing of linkage import/export names.

* Do final strip in emitEntryPoint, removes any remaining SourceLoc.

* Support for [__extern] to mark struct/function that are defined elsewhere.

* Allow adding extern to any decl.

* Use ExternAtrtibute to apply import decoration, rather than use an ir extern decoration.

* Added a test for [__extern]

* Improved comment around [__extern]

4 files changed, 67 insertions, 0 deletions

diff --git a/tests/serialization/extern/extern-test.slang b/tests/serialization/extern/extern-test.slang
new file mode 100644
index 000000000..0079a3527
--- /dev/null
+++ b/tests/serialization/extern/extern-test.slang
@@ -0,0 +1,26 @@
+// serialized-module-entry-point-test.slang
+
+//TEST:COMPILE: -module-name module -no-codegen tests/serialization/extern/module-a.slang -o tests/serialization/extern/module-a.slang-lib
+//TEST:COMPILE: -module-name module -no-codegen tests/serialization/extern/module-b.slang -o tests/serialization/extern/module-b.slang-lib
+//TEST:COMPARE_COMPUTE_EX: -xslang -module-name -xslang module -slang -compute -xslang -r -xslang tests/serialization/extern/module-a.slang-lib -xslang -r -xslang tests/serialization/extern/module-b.slang-lib 
+
+//TEST_INPUT:ubuffer(data=[0 0 0 0 ], stride=4):dxbinding(0),glbinding(0),out,name outputBuffer
+RWStructuredBuffer<int> outputBuffer;
+
+// Declare the type exists
+[__extern] struct Thing {};
+// A mechanism to make a Thing without knowing the specific fields.
+[__extern] Thing makeThing(int a, int b);
+
+[__extern] int doSomething(Thing a, Thing b);
+
+[numthreads(4, 1, 1)]
+void computeMain(uint3 dispatchThreadID : SV_DispatchThreadID)
+{
+    int index = int(dispatchThreadID.x);
+
+    Thing a = makeThing(index + 1, index + 2);
+    Thing b = makeThing(-index , index * 2);
+    
+    outputBuffer[index] = doSomething(a, b);
+}
\ No newline at end of file
diff --git a/tests/serialization/extern/extern-test.slang.expected.txt b/tests/serialization/extern/extern-test.slang.expected.txt
new file mode 100644
index 000000000..e326b6740
--- /dev/null
+++ b/tests/serialization/extern/extern-test.slang.expected.txt
@@ -0,0 +1,4 @@
+5
+B
+11
+17
diff --git a/tests/serialization/extern/module-a.slang b/tests/serialization/extern/module-a.slang
new file mode 100644
index 000000000..81816a213
--- /dev/null
+++ b/tests/serialization/extern/module-a.slang
@@ -0,0 +1,23 @@
+//TEST_IGNORE_FILE:
+
+// module-a.slang
+
+struct Thing
+{
+    int a; 
+    int b;
+};
+
+Thing makeThing(int a, int b)
+{
+    Thing thing;
+    thing.a = a;
+    thing.b = b;
+    return thing;
+}
+
+int foo(Thing thing) 
+{ 
+    return thing.a + thing.b * 2; 
+}
+
diff --git a/tests/serialization/extern/module-b.slang b/tests/serialization/extern/module-b.slang
new file mode 100644
index 000000000..20371f156
--- /dev/null
+++ b/tests/serialization/extern/module-b.slang
@@ -0,0 +1,14 @@
+//TEST_IGNORE_FILE:
+
+// module-b.slang
+
+// This looks like a definition (and it is) but with [__extern] it's definition will be replaced at link time with a defintion
+[__extern] struct Thing {};
+[__extern] int foo(Thing thing);
+
+int doSomething(Thing a, Thing b)
+{
+    return foo(a) + foo(b);
+}
+
+