From 4321929879c1ed5b87ff95a99ca7da91e28d18fd Mon Sep 17 00:00:00 2001 From: Ellie Hermaszewska Date: Fri, 8 Dec 2023 19:24:34 +0800 Subject: WIP: CMake (#3326) * More robust input and output selection in generator tools * Add cmake build system * Get slang-test running with cmake * Bump lz4 and miniz dependencies * Make cmake build more declarative * Correct preprocessor logic in slang.h * Add cuda test to compute/simple * Remove empty cmake files * output placement for cmake, and commenting * Correct include paths in spirv-embed-generator * Format cmake with gersemi * Make cmake build clerer * Neaten header generation Also work around https://gitlab.kitware.com/cmake/cmake/-/issues/18399 by introducing correct_generated_properties to set the GENERATED flag in the correct scope * remove unused files * use 3.20 to set GENERATOR property properly * spelling * more flexible linker arg setting * replace slang-static with obj collection * Set rpath and linker path correctly * neaten generated file generation * tests working with cmake build * fix premake5 build * comment and neaten cmake * remove unnecessary dependency * Build aftermath example only when aftermath is enabled * Add slang-llvm and other dependencies * Put modules alongside binaries * Find slang-glslang correctly * Better option handling * comments * add llvm build test * Better option handling * cmake wobble * use UNICODE and _UNICODE * remove other workflows * use ccache * neaten * limit parallel for llvm build * use ninja for build * Windows and Darwin slang-llvm builds * cache key * verbose llvm build * cl on windows * sccache and cl.exe * use cl.exe * Correct package detection * less verbosity * Simplify miniz inclusion * fix build with sccache * Neaten llvm building * neaten * Neaten slang-llvm fetching * more surgical workarounds * Add ci action * Get version from git * better variable naming * add missing include * clean up after premake in cmake * more docs on cmake build * ci wobble * add imgui target * more selective source * do not download swiftshader * Some missing dependencies * only build llvm on dispatch * Disable /Zi in CI where sccache is present * simplify * set PIC for miniz * set policies before project * reengage workaround * more runs on ci * Add cmake presets * Add cpack * move iterator debug level to preset * Correct lib flag * simplify action * Neaten cmake init * Add todo * Add simple test wrapper * Add tests to workflow presets * rename packing preset * Correctly set definitions * docs * correct preset names * Make slang-test depend on test-server/test-process * neaten * use workflow in actions * install docs * Correct module install dir * debug dist workflow * Install headers * neaten header globbing * Neaten dependency handling * make lib and bin variables * Do not set compiler for vs builds, unnecessary * docs * allow setting explicit source for target * maintain archive subdir * cmake docs * install headers * place targets into folders * cmake docs * nest external projects in folder * remove name clash * Neater external packages * meta targets in folder structure * cleaner slang-glslang dll * Add missing static directive to slang-no-embedded-stdlib * more robust module copying * make slang-test the startup project * folder tweak * Make FETCH_BINARY the default on all platforms * Set DEBUG_DIR * add natvis files to source * skip spirv tests * remove test step from debug dist * Add build to .gitignore * redo warnings to be more like premake * Update imgui * clean more premake files * Disable PCH for glslang, gcc throws a warning * Add /MP for msvc builds * warning wobble * Add script to build llvm * Add slang-llvm and generators components * Build slang-llvm in ci * comments * fetch llvm with git * better abi approximation for cache * better sccache key * formatting * Correct logic around disabling problematic debug info for ccache * exclude gcc and clang from windows ci * Make dist workflows use system llvm * naming * restore normal dist builds * formatting * run tests in ci * Correct slang-llvm url setting * Rely on the system to find the test tool library * actions matrix wiggle * cope with OSX ancient bash * Correct compilers on windows * more ci debugging * Correct rpath handling on OSX * neaten * correct path to slang-llvm * Correct rpath separator on osx * Find slang-llvm correctly * smoke tests only on osx * ci wobble * Give MacOS module a dylib suffix * get swiftshader correctly * cope with bsd cp * remove debug output * full tests on osx * ci wobble * Add some vk tests to expected failures * simplify ci * ci wobble * exclude dx12 tests from github ci * remove cmake code for building llvm * warnings * warnings as errors for cl * spirv-tools in path * add aarch64 ci build * Add SLANG_GENERATORS_PATH option for prebuilt generators * neaten * Correct generator target name * remove yaml anchors because github actions does not support them * Demote CMake in docs Also add info on cross compiling * Restore premake CI * use minimal ci for cmake * Write miniz_export for premake build and .gitignore it * Mention build config tool options in docs * Remove redefined macro for miniz * regenerate vs project --- external/CMakeLists.txt | 78 ++++++++++++++++++++++++++++ external/build-llvm.sh | 134 ++++++++++++++++++++++++++++++++++++++++++++++++ external/imgui | 2 +- external/lz4 | 2 +- external/miniz | 2 +- external/spirv-tools | 2 +- 6 files changed, 216 insertions(+), 4 deletions(-) create mode 100644 external/CMakeLists.txt create mode 100755 external/build-llvm.sh (limited to 'external') diff --git a/external/CMakeLists.txt b/external/CMakeLists.txt new file mode 100644 index 000000000..e61db9921 --- /dev/null +++ b/external/CMakeLists.txt @@ -0,0 +1,78 @@ +# Don't let all these packages spam up with status messages, filter to only the +# important stuff. +# Don't change things however if the user has explicitly set +# CMAKE_MESSAGE_LOG_LEVEL +set(set_cmake_log_level FALSE) +if(NOT CMAKE_MESSAGE_LOG_LEVEL) + set(set_cmake_log_level TRUE) + set(CMAKE_MESSAGE_LOG_LEVEL NOTICE) +endif() + +# Miniz +add_subdirectory(miniz EXCLUDE_FROM_ALL) +set_property(TARGET miniz PROPERTY POSITION_INDEPENDENT_CODE ON) +# Work around https://github.com/richgel999/miniz/pull/292 +get_target_property(miniz_c_launcher miniz C_COMPILER_LAUNCHER) +if(MSVC AND miniz_c_launcher MATCHES "ccache") + set_property(TARGET miniz PROPERTY C_COMPILER_LAUNCHER) + set_property(TARGET miniz PROPERTY MSVC_DEBUG_INFORMATION_FORMAT "") +endif() + +# LZ4 +set(LZ4_BUNDLED_MODE ON) +add_subdirectory(lz4/build/cmake EXCLUDE_FROM_ALL) +if(MSVC) + target_compile_options( + lz4_static + PRIVATE /wd5045 /wd4820 /wd4711 /wd6385 /wd6262 + ) +endif() + +# Vulkan headers +add_subdirectory(vulkan EXCLUDE_FROM_ALL) + +# SPIRV-Headers +add_subdirectory(spirv-headers EXCLUDE_FROM_ALL) + +# SPIRV-Tools +set(SPIRV_TOOLS_BUILD_STATIC ON) +set(SPIRV_WERROR OFF) +set(SPIRV_HEADER_DIR "${CMAKE_CURRENT_LIST_DIR}/spirv-headers/") +set(SPIRV_SKIP_TESTS ON) +add_subdirectory(spirv-tools EXCLUDE_FROM_ALL) + +# glslang +set(SKIP_GLSLANG_INSTALL ON) +set(ENABLE_OPT ON) +set(ENABLE_PCH OFF) +add_subdirectory(glslang EXCLUDE_FROM_ALL) + +# imgui +add_library(imgui INTERFACE) +target_include_directories(imgui INTERFACE "${CMAKE_CURRENT_LIST_DIR}/imgui") + +# Tidy things up: + +# Restore log level if we set it +if(set_cmake_log_level) + unset(CMAKE_MESSAGE_LOG_LEVEL) +endif() + +# for this directory and all subdirectories, prepend +# `external/` to the IDE FOLDER property to every target +function(make_external dir) + get_property(external_targets DIRECTORY ${dir} PROPERTY BUILDSYSTEM_TARGETS) + foreach(external_target ${external_targets}) + get_property(folder TARGET ${external_target} PROPERTY FOLDER) + set_property( + TARGET ${external_target} + PROPERTY FOLDER "external/${folder}" + ) + endforeach() + + get_property(subdirs DIRECTORY ${dir} PROPERTY SUBDIRECTORIES) + foreach(subdir ${subdirs}) + make_external(${subdir}) + endforeach() +endfunction() +make_external(.) diff --git a/external/build-llvm.sh b/external/build-llvm.sh new file mode 100755 index 000000000..a01158e78 --- /dev/null +++ b/external/build-llvm.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash + +set -e + +help() { + me=$(basename "$0") + cat <&2 +} + +fail() { + msg "$1" + exit 1 +} + +for prog in "cmake" "ninja" "git"; do + if ! command -v "$prog" &>/dev/null; then + msg "This script needs $prog, but it isn't in \$PATH" + missing_bin=1 + fi +done +if [ "$missing_bin" ]; then + exit 1 +fi + +# +# Temp dir with cleanup on exit +# +temp_dir=$(mktemp -d) +cleanup() { + local exit_status=$? + rm -rf "$temp_dir" + exit $exit_status +} +trap cleanup EXIT SIGHUP SIGINT SIGTERM + +# +# Options and parsing +# +repo=https://github.com/llvm/llvm-project +branch=llvmorg-13.0.1 +source_dir=$temp_dir +install_prefix= +config=Release +extra_arguments=() + +while [[ "$#" -gt 0 ]]; do + case $1 in + -h | --help) help; exit ;; + --repo) repo=$2; shift;; + --branch) branch=$2; shift;; + --source-dir) source_dir=$2; shift;; + --config) config=$2; shift;; + --install-prefix) install_prefix=$2; shift;; + --) shift; extra_arguments+=("$@"); break;; + *) + msg "Unknown parameter passed: $1" + help >&2 + exit 1 + ;; + esac + shift +done + +[ -n "$repo" ] || fail "please set --repo" +[ -n "$branch" ] || fail "please set --branch" +[ -n "$source_dir" ] || fail "please set --source-dir" +[ -n "$config" ] || fail "please set --config" +[ -n "$install_prefix" ] || fail "please set --install-prefix" + +msg "##########################################################" +msg "# Fetching LLVM from $repo at $branch" +msg "##########################################################" +git -c advice.detachedHead=false clone "$repo" --branch "$branch" "$source_dir" --depth 1 + +msg "##########################################################" +msg "# Configuring LLVM in $source_dir" +msg "##########################################################" +cmake_arguments_for_slang=( + # Don't build unnecessary things + -DLLVM_BUILD_LLVM_C_DYLIB=0 + -DLLVM_INCLUDE_BENCHMARKS=0 + -DLLVM_INCLUDE_DOCS=0 + -DLLVM_INCLUDE_EXAMPLES=0 + -DLLVM_INCLUDE_TESTS=0 + -DLLVM_ENABLE_TERMINFO=0 + -DCLANG_BUILD_TOOLS=0 + -DCLANG_ENABLE_STATIC_ANALYZER=0 + -DCLANG_ENABLE_ARCMT=0 + -DCLANG_INCLUDE_DOCS=0 + -DCLANG_INCLUDE_TESTS=0 + # Requirements for Slang + -DLLVM_ENABLE_PROJECTS=clang + "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64" + -DLLVM_BUILD_TOOLS=1 +) +build_dir=$source_dir/build +mkdir -p "$build_dir" +cmake \ + -S "$source_dir/llvm" -B "$build_dir" \ + -G "Ninja Multi-Config" \ + "${cmake_arguments_for_slang[@]}" \ + "${extra_arguments[@]}" + +msg "##########################################################" +msg "# Building LLVM in $build_dir" +msg "##########################################################" +cmake --build "$build_dir" -j --config "$config" + +msg "##########################################################" +msg "# Installing LLVM to $install_prefix" +msg "##########################################################" +cmake --install "$build_dir" --prefix "$install_prefix" --config "$config" + +msg "##########################################################" +msg "LLVM installed in $install_prefix" +msg "Please add $install_prefix to CMAKE_PREFIX_PATH" +msg "##########################################################" diff --git a/external/imgui b/external/imgui index fd2a90ee6..3c15dffc9 160000 --- a/external/imgui +++ b/external/imgui @@ -1 +1 @@ -Subproject commit fd2a90ee60c1b766af5a5ae167f38d601414141f +Subproject commit 3c15dffc944419eb4bb17984548468270ca90486 diff --git a/external/lz4 b/external/lz4 index d44371841..039ab4696 160000 --- a/external/lz4 +++ b/external/lz4 @@ -1 +1 @@ -Subproject commit d44371841a2f1728a3f36839fd4b7e872d0927d3 +Subproject commit 039ab4696526378ec0d111058b48305964284bf2 diff --git a/external/miniz b/external/miniz index a4264837a..18795fa61 160000 --- a/external/miniz +++ b/external/miniz @@ -1 +1 @@ -Subproject commit a4264837ae37384b1d7a205a6732db322f0f3769 +Subproject commit 18795fa61e590521381ba9e1fa4a4ab362b095f6 diff --git a/external/spirv-tools b/external/spirv-tools index 8994351e5..866e60def 160000 --- a/external/spirv-tools +++ b/external/spirv-tools @@ -1 +1 @@ -Subproject commit 8994351e58e45e1f067733966628383c50341de9 +Subproject commit 866e60defc05326c5cf0ad711ec453dd25e6edec -- cgit v1.2.3