WIP: CMake (#3326)

* More robust input and output selection in generator tools

* Add cmake build system

* Get slang-test running with cmake

* Bump lz4 and miniz dependencies

* Make cmake build more declarative

* Correct preprocessor logic in slang.h

* Add cuda test to compute/simple

* Remove empty cmake files

* output placement for cmake, and commenting

* Correct include paths in spirv-embed-generator

* Format cmake with gersemi

* Make cmake build clerer

* Neaten header generation

Also work around https://gitlab.kitware.com/cmake/cmake/-/issues/18399
by introducing correct_generated_properties to set the GENERATED flag in
the correct scope

* remove unused files

* use 3.20 to set GENERATOR property properly

* spelling

* more flexible linker arg setting

* replace slang-static with obj collection

* Set rpath and linker path correctly

* neaten generated file generation

* tests working with cmake build

* fix premake5 build

* comment and neaten cmake

* remove unnecessary dependency

* Build aftermath example only when aftermath is enabled

* Add slang-llvm and other dependencies

* Put modules alongside binaries

* Find slang-glslang correctly

* Better option handling

* comments

* add llvm build test

* Better option handling

* cmake wobble

* use UNICODE and _UNICODE

* remove other workflows

* use ccache

* neaten

* limit parallel for llvm build

* use ninja for build

* Windows and Darwin slang-llvm builds

* cache key

* verbose llvm build

* cl on windows

* sccache and cl.exe

* use cl.exe

* Correct package detection

* less verbosity

* Simplify miniz inclusion

* fix build with sccache

* Neaten llvm building

* neaten

* Neaten slang-llvm fetching

* more surgical workarounds

* Add ci action

* Get version from git

* better variable naming

* add missing include

* clean up after premake in cmake

* more docs on cmake build

* ci wobble

* add imgui target

* more selective source

* do not download swiftshader

* Some missing dependencies

* only build llvm on dispatch

* Disable /Zi in CI where sccache is present

* simplify

* set PIC for miniz

* set policies before project

* reengage workaround

* more runs on ci

* Add cmake presets

* Add cpack

* move iterator debug level to preset

* Correct lib flag

* simplify action

* Neaten cmake init

* Add todo

* Add simple test wrapper

* Add tests to workflow presets

* rename packing preset

* Correctly set definitions

* docs

* correct preset names

* Make slang-test depend on test-server/test-process

* neaten

* use workflow in actions

* install docs

* Correct module install dir

* debug dist workflow

* Install headers

* neaten header globbing

* Neaten dependency handling

* make lib and bin variables

* Do not set compiler for vs builds, unnecessary

* docs

* allow setting explicit source for target

* maintain archive subdir

* cmake docs

* install headers

* place targets into folders

* cmake docs

* nest external projects in folder

* remove name clash

* Neater external packages

* meta targets in folder structure

* cleaner slang-glslang dll

* Add missing static directive to slang-no-embedded-stdlib

* more robust module copying

* make slang-test the startup project

* folder tweak

* Make FETCH_BINARY the default on all platforms

* Set DEBUG_DIR

* add natvis files to source

* skip spirv tests

* remove test step from debug dist

* Add build to .gitignore

* redo warnings to be more like premake

* Update imgui

* clean more premake files

* Disable PCH for glslang, gcc throws a warning

* Add /MP for msvc builds

* warning wobble

* Add script to build llvm

* Add slang-llvm and generators components

* Build slang-llvm in ci

* comments

* fetch llvm with git

* better abi approximation for cache

* better sccache key

* formatting

* Correct logic around disabling problematic debug info for ccache

* exclude gcc and clang from windows ci

* Make dist workflows use system llvm

* naming

* restore normal dist builds

* formatting

* run tests in ci

* Correct slang-llvm url setting

* Rely on the system to find the test tool library

* actions matrix wiggle

* cope with OSX ancient bash

* Correct compilers on windows

* more ci debugging

* Correct rpath handling on OSX

* neaten

* correct path to slang-llvm

* Correct rpath separator on osx

* Find slang-llvm correctly

* smoke tests only on osx

* ci wobble

* Give MacOS module a dylib suffix

* get swiftshader correctly

* cope with bsd cp

* remove debug output

* full tests on osx

* ci wobble

* Add some vk tests to expected failures

* simplify ci

* ci wobble

* exclude dx12 tests from github ci

* remove cmake code for building llvm

* warnings

* warnings as errors for cl

* spirv-tools in path

* add aarch64 ci build

* Add SLANG_GENERATORS_PATH option for prebuilt generators

* neaten

* Correct generator target name

* remove yaml anchors because github actions does not support them

* Demote CMake in docs

Also add info on cross compiling

* Restore premake CI

* use minimal ci for cmake

* Write miniz_export for premake build

and .gitignore it

* Mention build config tool options in docs

* Remove redefined macro for miniz

* regenerate vs project

2 files changed, 253 insertions, 0 deletions

diff --git a/.github/actions/common-setup/action.yml b/.github/actions/common-setup/action.yml
new file mode 100644
index 000000000..60c81b007
--- /dev/null
+++ b/.github/actions/common-setup/action.yml
@@ -0,0 +1,112 @@
+name: Common setup
+
+description: Performs setup common to all our actions
+
+inputs:
+  os:
+    required: true
+  runs-on:
+    required: true
+  compiler:
+    required: true
+  platform:
+    required: true
+  config:
+    required: true
+runs:
+  using: composite
+  steps:
+    - name: Set up MSVC dev tools on Windows
+      uses: ilammy/msvc-dev-cmd@v1
+    - name: Set up sccache
+      uses: hendrikmuhs/ccache-action@v1.2
+      with:
+        key: ${{github.job}}-${{inputs.os}}-${{inputs.compiler}}-${{inputs.platform}}-${{inputs.config}}
+        variant: sccache
+    - shell: bash
+      run: |
+        # Set up system dependencies
+
+        # Install Ninja
+        case "${{inputs.os}}" in
+          linux*) sudo apt-get install -y ninja-build;;
+          windows*) choco install ninja;;
+          macos*) brew install ninja;;
+        esac
+
+        # Set compiler
+        CC=${{inputs.compiler}}
+        CXX=${{inputs.compiler}}
+        # infer C++ compiler
+        CXX=${CXX/gcc/g++}
+        CXX=${CXX/clang/clang++}
+        # Correct version on older ubuntu
+        if [[ "${{inputs.runs-on}}" == ubuntu-20.04 ]]; then
+          CC=${CC/gcc/gcc-10}
+          CXX=${CXX/g++/g++-10}
+        fi
+        # Export
+        echo "CC=$CC" >> "$GITHUB_ENV"
+        echo "CXX=$CXX" >> "$GITHUB_ENV"
+
+        # Set CMake to use sccache
+        echo "CMAKE_CXX_COMPILER_LAUNCHER=sccache" >> "$GITHUB_ENV"
+        echo "CMAKE_C_COMPILER_LAUNCHER=sccache" >> "$GITHUB_ENV"
+
+        # Some useful variables
+        config=${{inputs.config}}
+        Config=$(echo "${{inputs.config}}" | sed 's/debug/Debug/;s/release/Release/')
+        bin_dir=$(pwd)/build/$Config/bin
+        lib_dir=$(pwd)/build/$Config/lib
+        echo "config=$config" >> "$GITHUB_ENV"
+        echo "Config=$Config" >> "$GITHUB_ENV"
+        echo "bin_dir=$bin_dir" >> "$GITHUB_ENV"
+        echo "lib_dir=$lib_dir" >> "$GITHUB_ENV"
+
+    # Try to restore a LLVM install, and build it otherwise
+    - uses: actions/cache/restore@v3
+      id: cache-llvm
+      with:
+        path: ${{ github.workspace }}/build/llvm-project-install
+        # Use os*compiler*platform in lieu of an ABI key here, which is what we really want
+        key: ${{ inputs.os }}-${{ inputs.compiler }}-${{ inputs.platform }}-${{ hashFiles('external/build-llvm.sh') }}
+    - name: Build LLVM
+      if: steps.cache-llvm.outputs.cache-hit != 'true'
+      shell: bash
+      run: ./external/build-llvm.sh --install-prefix "${{ github.workspace }}/build/llvm-project-install"
+    - uses: actions/cache/save@v3
+      if: steps.cache-llvm.outputs.cache-hit != 'true'
+      with:
+        path: ${{ github.workspace }}/build/llvm-project-install
+        key: ${{ steps.cache-llvm.outputs.cache-primary-key }}
+    - name: Set environment variable for CMake
+      shell: bash
+      run: |
+        echo "LLVM_DIR=${{ github.workspace }}/build/llvm-project-install" >> "$GITHUB_ENV"
+        echo "Clang_DIR=${{ github.workspace }}/build/llvm-project-install" >> "$GITHUB_ENV"
+
+    # Install swiftshader
+    - uses: robinraju/release-downloader@v1.8
+      continue-on-error: true
+      with:
+        latest: true
+        repository: "shader-slang/swiftshader"
+        out-file-path: "swiftshader"
+        extract: true
+        fileName: "vk_swiftshader_${{inputs.os}}_${{inputs.platform}}.zip"
+
+    - name: Install SwiftShader
+      shell: bash
+      run: |
+        case "${{inputs.os}}" in
+          windows*) echo "${{github.workspace}}/swiftshader/" >> "$GITHUB_PATH";;
+          *) echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}${{github.workspace}}/swiftshader" >> "$GITHUB_ENV";;
+        esac
+
+    # Put spirv-tools in path
+    - shell: bash
+      run: |
+        case "${{inputs.os}}" in
+          windows*) echo "${{github.workspace}}/external/slang-binaries/spirv-tools/windows-${{inputs.platform}}/bin" >> "$GITHUB_PATH";;
+          linux*) echo "${{github.workspace}}/external/slang-binaries/spirv-tools/$(uname -m)-linux/bin" >> "$GITHUB_PATH";;
+        esac
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 000000000..4c5a863a3
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,141 @@
+name: CI
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        # Until we switch to CMake fully, just run something minimal
+        include:
+          - os: linux
+            compiler: gcc
+            platform: x64
+            config: debug
+            warnings-as-errors: false
+            test-category: smoke
+            full-gpu-tests: false
+            runs-on: ubuntu-20.04
+        # os: [linux, macos, windows]
+        # compiler: ['gcc', 'clang', 'cl']
+        # config: ['debug', 'release']
+        # platform: ['x64']
+        # full-gpu-tests: [false]
+        # exclude:
+        #   # Exclude invalid or undesired os/compiler
+        #   - { os: linux, compiler: cl }
+        #   - { os: macos, compiler: cl }
+        #   - { os: macos, compiler: gcc }
+        #   - { os: windows, compiler: gcc }
+        #   - { os: windows, compiler: clang }
+        #   # or os/config combinations
+        #   - { os: macos, config: debug }
+        # include:
+        #   - { os: linux, runs-on: ubuntu-20.04 }
+        #   - { os: windows, runs-on: windows-latest }
+        #   - { os: macos, runs-on: macos-latest }
+        #   # Set a test category depending on the config, smoke by default,
+        #   # quick or full conditionally otherwise
+        #   - test-category: smoke
+        #   - { os: windows, test-category: quick }
+        #   - { config: release, test-category: full }
+        #   # Only CL has werror
+        #   - warnings-as-errors: false
+        #   - { compiler: cl, warnings-as-errors: true }
+        #   # Self-hosted aarch64 build
+        #   - os: linux
+        #     compiler: gcc
+        #     platform: aarch64
+        #     config: release
+        #     warnings-as-errors: false
+        #     test-category: smoke
+        #     full-gpu-tests: false
+        #     runs-on: [self-hosted, Linux, ARM64]
+        #   # Self-hosted full gpu build
+        #   - os: windows
+        #     compiler: cl
+        #     platform: x64
+        #     config: release
+        #     warnings-as-errors: false
+        #     test-category: full
+        #     full-gpu-tests: true
+        #     runs-on: [Windows, self-hosted]
+      fail-fast: false
+    runs-on: ${{ matrix.runs-on }}
+
+    defaults:
+      run:
+        shell: bash
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          submodules: 'true'
+          fetch-depth: '0'
+      - name: Setup
+        uses: ./.github/actions/common-setup
+        with:
+          os: ${{matrix.os}}
+          runs-on: ${{matrix.runs-on}}
+          compiler: ${{matrix.compiler}}
+          platform: ${{matrix.platform}}
+          config: ${{matrix.config}}
+      - name: Build Slang
+        run: |
+          if [[ "${{ matrix.os }}" =~ "windows" && "${{ matrix.config }}" != "release" ]]; then
+            # Doing a debug build will try to link against a release built llvm, this
+            # is a problem on Windows, so make slang-llvm in release build and use
+            # that as though it's a fetched binary via these presets.
+            cmake --workflow --preset slang-llvm
+            # Configure, pointing to our just-generated slang-llvm archive
+            cmake --preset default --fresh \
+              -DSLANG_SLANG_LLVM_FLAVOR=FETCH_BINARY \
+              -DSLANG_SLANG_LLVM_BINARY_URL=$(pwd)/build/dist-release/slang-slang-llvm.zip \
+              -DCMAKE_COMPILE_WARNING_AS_ERROR=${{matrix.warnings-as-errors}}
+            cmake --workflow --preset "${{matrix.config}}"
+          else
+            # Otherwise, use the system llvm we have just build or got from the
+            # cache in the setup phase
+            cmake --preset default --fresh \
+              -DSLANG_SLANG_LLVM_FLAVOR=USE_SYSTEM_LLVM \
+              -DCMAKE_COMPILE_WARNING_AS_ERROR=${{matrix.warnings-as-errors}}
+            cmake --workflow --preset "${{matrix.config}}"
+          fi
+      - name: Test Slang
+        run: |
+          export SLANG_RUN_SPIRV_VALIDATION=1
+          failed=0
+          if [[ "${{matrix.full-gpu-tests}}" == "true" ]]; then
+            $bin_dir/slang-test \
+              -use-test-server \
+              -server-count 8 \
+              -category ${{ matrix.test-category }} \
+              -api all-cpu ||
+              failed=1
+            $bin_dir/slang-test \
+              -use-test-server \
+              -server-count 8 \
+              -emit-spirv-directly \
+              -api vk \
+              -category ${{ matrix.test-category }} \
+              -expected-failure-list tests/expected-failure.txt ||
+              failed=1
+          else
+            $bin_dir/slang-test \
+              -use-test-server \
+              -api all-dx12 \
+              -category ${{ matrix.test-category }} \
+              -expected-failure-list tests/expected-failure-github.txt ||
+              failed=1
+          fi
+          exit $failed
+      - uses: actions/upload-artifact@v3
+        if: ${{ ! matrix.full-gpu-tests }}
+        with:
+          name: slang-build-${{matrix.os}}-${{matrix.platform}}-${{matrix.compiler}}-${{matrix.config}}
+          # The install directory used in the packaging step
+          path: build/dist-${{matrix.config}}/**/slang/*