From 4321929879c1ed5b87ff95a99ca7da91e28d18fd Mon Sep 17 00:00:00 2001 From: Ellie Hermaszewska Date: Fri, 8 Dec 2023 19:24:34 +0800 Subject: WIP: CMake (#3326) * More robust input and output selection in generator tools * Add cmake build system * Get slang-test running with cmake * Bump lz4 and miniz dependencies * Make cmake build more declarative * Correct preprocessor logic in slang.h * Add cuda test to compute/simple * Remove empty cmake files * output placement for cmake, and commenting * Correct include paths in spirv-embed-generator * Format cmake with gersemi * Make cmake build clerer * Neaten header generation Also work around https://gitlab.kitware.com/cmake/cmake/-/issues/18399 by introducing correct_generated_properties to set the GENERATED flag in the correct scope * remove unused files * use 3.20 to set GENERATOR property properly * spelling * more flexible linker arg setting * replace slang-static with obj collection * Set rpath and linker path correctly * neaten generated file generation * tests working with cmake build * fix premake5 build * comment and neaten cmake * remove unnecessary dependency * Build aftermath example only when aftermath is enabled * Add slang-llvm and other dependencies * Put modules alongside binaries * Find slang-glslang correctly * Better option handling * comments * add llvm build test * Better option handling * cmake wobble * use UNICODE and _UNICODE * remove other workflows * use ccache * neaten * limit parallel for llvm build * use ninja for build * Windows and Darwin slang-llvm builds * cache key * verbose llvm build * cl on windows * sccache and cl.exe * use cl.exe * Correct package detection * less verbosity * Simplify miniz inclusion * fix build with sccache * Neaten llvm building * neaten * Neaten slang-llvm fetching * more surgical workarounds * Add ci action * Get version from git * better variable naming * add missing include * clean up after premake in cmake * more docs on cmake build * ci wobble * add imgui target * more selective source * do not download swiftshader * Some missing dependencies * only build llvm on dispatch * Disable /Zi in CI where sccache is present * simplify * set PIC for miniz * set policies before project * reengage workaround * more runs on ci * Add cmake presets * Add cpack * move iterator debug level to preset * Correct lib flag * simplify action * Neaten cmake init * Add todo * Add simple test wrapper * Add tests to workflow presets * rename packing preset * Correctly set definitions * docs * correct preset names * Make slang-test depend on test-server/test-process * neaten * use workflow in actions * install docs * Correct module install dir * debug dist workflow * Install headers * neaten header globbing * Neaten dependency handling * make lib and bin variables * Do not set compiler for vs builds, unnecessary * docs * allow setting explicit source for target * maintain archive subdir * cmake docs * install headers * place targets into folders * cmake docs * nest external projects in folder * remove name clash * Neater external packages * meta targets in folder structure * cleaner slang-glslang dll * Add missing static directive to slang-no-embedded-stdlib * more robust module copying * make slang-test the startup project * folder tweak * Make FETCH_BINARY the default on all platforms * Set DEBUG_DIR * add natvis files to source * skip spirv tests * remove test step from debug dist * Add build to .gitignore * redo warnings to be more like premake * Update imgui * clean more premake files * Disable PCH for glslang, gcc throws a warning * Add /MP for msvc builds * warning wobble * Add script to build llvm * Add slang-llvm and generators components * Build slang-llvm in ci * comments * fetch llvm with git * better abi approximation for cache * better sccache key * formatting * Correct logic around disabling problematic debug info for ccache * exclude gcc and clang from windows ci * Make dist workflows use system llvm * naming * restore normal dist builds * formatting * run tests in ci * Correct slang-llvm url setting * Rely on the system to find the test tool library * actions matrix wiggle * cope with OSX ancient bash * Correct compilers on windows * more ci debugging * Correct rpath handling on OSX * neaten * correct path to slang-llvm * Correct rpath separator on osx * Find slang-llvm correctly * smoke tests only on osx * ci wobble * Give MacOS module a dylib suffix * get swiftshader correctly * cope with bsd cp * remove debug output * full tests on osx * ci wobble * Add some vk tests to expected failures * simplify ci * ci wobble * exclude dx12 tests from github ci * remove cmake code for building llvm * warnings * warnings as errors for cl * spirv-tools in path * add aarch64 ci build * Add SLANG_GENERATORS_PATH option for prebuilt generators * neaten * Correct generator target name * remove yaml anchors because github actions does not support them * Demote CMake in docs Also add info on cross compiling * Restore premake CI * use minimal ci for cmake * Write miniz_export for premake build and .gitignore it * Mention build config tool options in docs * Remove redefined macro for miniz * regenerate vs project --- .github/workflows/ci.yml | 141 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 .github/workflows/ci.yml (limited to '.github/workflows') diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 000000000..4c5a863a3 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,141 @@ +name: CI + +on: + push: + branches: [master] + pull_request: + branches: [master] + +jobs: + build: + strategy: + matrix: + # Until we switch to CMake fully, just run something minimal + include: + - os: linux + compiler: gcc + platform: x64 + config: debug + warnings-as-errors: false + test-category: smoke + full-gpu-tests: false + runs-on: ubuntu-20.04 + # os: [linux, macos, windows] + # compiler: ['gcc', 'clang', 'cl'] + # config: ['debug', 'release'] + # platform: ['x64'] + # full-gpu-tests: [false] + # exclude: + # # Exclude invalid or undesired os/compiler + # - { os: linux, compiler: cl } + # - { os: macos, compiler: cl } + # - { os: macos, compiler: gcc } + # - { os: windows, compiler: gcc } + # - { os: windows, compiler: clang } + # # or os/config combinations + # - { os: macos, config: debug } + # include: + # - { os: linux, runs-on: ubuntu-20.04 } + # - { os: windows, runs-on: windows-latest } + # - { os: macos, runs-on: macos-latest } + # # Set a test category depending on the config, smoke by default, + # # quick or full conditionally otherwise + # - test-category: smoke + # - { os: windows, test-category: quick } + # - { config: release, test-category: full } + # # Only CL has werror + # - warnings-as-errors: false + # - { compiler: cl, warnings-as-errors: true } + # # Self-hosted aarch64 build + # - os: linux + # compiler: gcc + # platform: aarch64 + # config: release + # warnings-as-errors: false + # test-category: smoke + # full-gpu-tests: false + # runs-on: [self-hosted, Linux, ARM64] + # # Self-hosted full gpu build + # - os: windows + # compiler: cl + # platform: x64 + # config: release + # warnings-as-errors: false + # test-category: full + # full-gpu-tests: true + # runs-on: [Windows, self-hosted] + fail-fast: false + runs-on: ${{ matrix.runs-on }} + + defaults: + run: + shell: bash + + steps: + - uses: actions/checkout@v3 + with: + submodules: 'true' + fetch-depth: '0' + - name: Setup + uses: ./.github/actions/common-setup + with: + os: ${{matrix.os}} + runs-on: ${{matrix.runs-on}} + compiler: ${{matrix.compiler}} + platform: ${{matrix.platform}} + config: ${{matrix.config}} + - name: Build Slang + run: | + if [[ "${{ matrix.os }}" =~ "windows" && "${{ matrix.config }}" != "release" ]]; then + # Doing a debug build will try to link against a release built llvm, this + # is a problem on Windows, so make slang-llvm in release build and use + # that as though it's a fetched binary via these presets. + cmake --workflow --preset slang-llvm + # Configure, pointing to our just-generated slang-llvm archive + cmake --preset default --fresh \ + -DSLANG_SLANG_LLVM_FLAVOR=FETCH_BINARY \ + -DSLANG_SLANG_LLVM_BINARY_URL=$(pwd)/build/dist-release/slang-slang-llvm.zip \ + -DCMAKE_COMPILE_WARNING_AS_ERROR=${{matrix.warnings-as-errors}} + cmake --workflow --preset "${{matrix.config}}" + else + # Otherwise, use the system llvm we have just build or got from the + # cache in the setup phase + cmake --preset default --fresh \ + -DSLANG_SLANG_LLVM_FLAVOR=USE_SYSTEM_LLVM \ + -DCMAKE_COMPILE_WARNING_AS_ERROR=${{matrix.warnings-as-errors}} + cmake --workflow --preset "${{matrix.config}}" + fi + - name: Test Slang + run: | + export SLANG_RUN_SPIRV_VALIDATION=1 + failed=0 + if [[ "${{matrix.full-gpu-tests}}" == "true" ]]; then + $bin_dir/slang-test \ + -use-test-server \ + -server-count 8 \ + -category ${{ matrix.test-category }} \ + -api all-cpu || + failed=1 + $bin_dir/slang-test \ + -use-test-server \ + -server-count 8 \ + -emit-spirv-directly \ + -api vk \ + -category ${{ matrix.test-category }} \ + -expected-failure-list tests/expected-failure.txt || + failed=1 + else + $bin_dir/slang-test \ + -use-test-server \ + -api all-dx12 \ + -category ${{ matrix.test-category }} \ + -expected-failure-list tests/expected-failure-github.txt || + failed=1 + fi + exit $failed + - uses: actions/upload-artifact@v3 + if: ${{ ! matrix.full-gpu-tests }} + with: + name: slang-build-${{matrix.os}}-${{matrix.platform}}-${{matrix.compiler}}-${{matrix.config}} + # The install directory used in the packaging step + path: build/dist-${{matrix.config}}/**/slang/* -- cgit v1.2.3