nissy-core

The "engine" of nissy, including the H48 optimal solver.
git clone https://git.tronto.net/nissy-core
Download | Log | Files | Refs | README | LICENSE

commit 2a38070d44ed01753c51067bbd7f007e4b74bd6b
parent de9b93c702e771d4b8ca8c6dae6f33e25f383741
Author: Sebastiano Tronto <sebastiano@tronto.net>
Date:   Tue, 12 Aug 2025 15:49:36 +0200

Renamed UNIX build script to build.sh, added info to README.md related to windows build

Diffstat:
MREADME.md | 54+++++++++++++++++++++++++++++++++---------------------
Mbenchmarks/benchmarks.md | 2+-
Dbuild | 475-------------------------------------------------------------------------------
Abuild.sh | 475+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mpython/examples/solve.py | 2+-
Mutils/container-run.sh | 6+++---
Mweb/http/nissyapp.mjs | 4++--
7 files changed, 515 insertions(+), 503 deletions(-)

diff --git a/README.md b/README.md @@ -13,43 +13,55 @@ of pruning tables that were developed independently. ## Building -This project currently supports only POSIX systems (Linux, MacOS, BSD...). -If you want to build on Windows you can do so via -[WSL](https://learn.microsoft.com/en-us/windows/wsl/install). Note that -the resulting executable will not be a native Windows executable. -Windows as a build target will be available in the future. +This project is mainly developed on UNIX systems (Linux, MacOS, BSD...), +but it is also possible to build it on Windows, with some limitations. + +### UNIX (Linux, MaxOS, BSD...) To build nissy simply run ``` -$ ./build +$ ./build.sh ``` -For a list of options and targets for the build system run `./build help`. +For a list of options and targets for the build system run `./build.sh help`. Some compiler settings can be overridden using environment variables, as explained in the comments at the beginning of the build script. For example, the command: ``` $ export NISSY_BUILD_THREADS=3 -$ CC=gcc ./build +$ CC=gcc ./build.sh ``` is going to configure `nissy` to use at most 3 threads, and build it with -`gcc`. Se the comments in `./build` for more details. +`gcc`. Se the comments in `./build.sh` for more details. + +### Windows + +It is possible to build this project on Windows using `build.bat`. This script +is going to build: + +* The core nissy library +* The shell `run.exe` (see below) +* The Python module (see below) + +All other options are unavailable. Moreover, Windows build will not +enable certain optimizations, such as multithreading and advanced CPU +instructions. Work is ongoing to improve Windows support. ## Running tests This project includes a suite of "unit" test. You can run them with: ``` -$ ./build test +$ ./build.sh test ``` For running the tests for the WebAssembly build with nodejs you can use: ``` -$ ./build webtest +$ ./build.sh webtest ``` To run only a subset of the tests, you can pass as argument a regular @@ -84,7 +96,7 @@ are not run in debug mode by default. To run a tool you can use: ``` -$ ./build tool TOOLNAME PARAMETERS... +$ ./build.sh tool TOOLNAME PARAMETERS... ``` Where `TOOLNAME` is the name of one of the tools, or a regular expression @@ -94,7 +106,7 @@ parameters. For example: ``` -$ :./build tool gendata h48h2k2 +$ :./build.sh tool gendata h48h2k2 ``` Will run a tool that generates the data table for the H48 solver with @@ -104,7 +116,7 @@ Each tool run is automatically timed, so these tools can be used as benchmark. The output as well as the time of the run are saved to a file in the tools/results folder. -To build and run a tool in debug mode, use `./build -d tool`. +To build and run a tool in debug mode, use `./build.sh -d tool`. ### The `solvetest` tools @@ -113,7 +125,7 @@ solvers produce the correct solutions. They can be run individually as all other tools, or all together with ``` -$ ./build solvetest # Use -d for debug mode (very slow for some solvers) +$ ./build.sh solvetest # Use -d for debug mode (very slow for some solvers) ``` If one of the solvetests fails, subsequent tests are going to be skipped. @@ -127,7 +139,7 @@ as the commands require quite verbose options. To build the shell run: ``` -$ ./build shell +$ ./build.sh shell ``` This will create an executable called `run`. Then you can for example @@ -192,7 +204,7 @@ interface. You can build them and run them with the build tool, for example: ``` -./build cpp cpp/examples/solve_h48h3k2.cpp +./build.sh cpp cpp/examples/solve_h48h3k2.cpp ``` NOTE: If you prefer to use a C-style API, you'll have to write @@ -207,10 +219,10 @@ this module follows the C API quite closely, except its functions sometimes return strings instead of writing to `char *` buffers. To build the Python module you need the Python development headers -installed. You can check this from the output of `./build config`: +installed. You can check this from the output of `./build.sh config`: ``` -$ ./build config +$ ./build.sh config ... Python bindings: version 3.13 ``` @@ -218,7 +230,7 @@ Python bindings: version 3.13 Then to build the module: ``` -$ ./build python +$ ./build.sh python ``` And to import it @@ -255,7 +267,7 @@ Bindings for JavaScript via a WebAssembly build (using The JavaScript module can be built with ``` -$ ./build web +$ ./build.sh web ``` An example web app running nissy can be found in the `web/http` folder. diff --git a/benchmarks/benchmarks.md b/benchmarks/benchmarks.md @@ -234,7 +234,7 @@ Time per cube adjusted for tables size (in seconds \* GiB, lower is better). example: ``` -./build tool solve_file h48h7k2 ./benchmarks/scrambles/scrambles-16.txt +./build.sh tool solve_file h48h7k2 ./benchmarks/scrambles/scrambles-16.txt ``` To find all solutions, add something like `99999 0` at the end of the diff --git a/build b/build @@ -1,475 +0,0 @@ -#!/bin/sh - -# Build system for nissy, run './build help' for info on how to use this. - -# The variables below can be used to personalize the build system. For example, -# to compile with clang instead of the default cc one can use: -# -# CC=clang ./build -# -# Each variable also has a counterpart that starts with NISSY_BUILD_*. These -# other variables can be set, for example, in your shell configuration file -# to change the default build options. They are still overwritten by the values -# of the non-NISSY_BUILD_* variables, if set. -# For example, on a system that supports the addressed and undefined behavior -# sanitizers, one can set -# -# export NISSY_BUILD_SANITIZE="address,undefined" -# -# so that these sanitizers will be enabled when building nissy in debug mode. -# If later the same user wants to build a debug version without sanitizers, -# they may use -# -# SANITIZE="" ./build debug -# -# And the empty string value will take precedence. - -# Specify the C compiler to use. -CC=${CC-${NISSY_BUILD_CC:-"cc"}} - -# Specify the C++ compiler for the C++ examples. -CXX=${CXX-${NISSY_BUILD_CXX:-"c++"}} - -# Specify the compiler to use when building for WASM. -EMCC=${EMCC-${NISSY_BUILD_EMCC:-"emcc"}} - -# Specify the nodejs command for running tests for WASM. -NODE=${NODE-${NISSY_BUILD_NODE:-"node"}} - -# The maximum number of threads to use for multi-threaded operations. -# This is also used as default value in case an operation allows -# specifying how many threads to use. -# The number n must be between 1 and 128. -# The default value is 16. -THREADS=${THREADS-${NISSY_BUILD_THREADS}} - -detectthreads() { - # TODO: detect based on system - # Is 'getconf NPROCESSORD_ONLN' portable? Is it threads or cores? - echo "16" -} - -[ -z "$THREADS" ] && THREADS="$(detectthreads)" - -# You can use this variable to build for a different architecture, for example -# if you want to cross-compile or to use the portable version. -# The value must be one of "AVX2", "NEON", "PORTABLE". If the value is not -# specified, the script will automatically detect the best architecture for the -# system. The architecture "PORTABLE" will work on any system. -ARCH=${ARCH-${NISSY_BUILD_ARCH}} - -greparch() { - $CC -march=native -dM -E - </dev/null 2>/dev/null | grep "$1" -} - -detectarch() { - [ -n "$(greparch __AVX2__)" ] && detected="AVX2" - [ -n "$(greparch __ARM_NEON)" ] && detected="NEON" - [ -z "$detected" ] && detected="PORTABLE" - echo "$detected" -} - -[ -z "$ARCH" ] && ARCH="$(detectarch)" - -# SANITIZE="option1,option2,..." adds the options "-fsanitize=option1", -# "-fsanitize=option2", ... to the C compiler command in debug mode. -# By default none is used because these options are not available on all -# systems. It is advisable to set the NISSY_BUILD_SANITIZE options in your -# shell's configuration file if your system does support them. -SANITIZE=${SANITIZE-${NISSY_BUILD_SANITIZE:-""}} - -# Optimization flags. -OPTIMIZE=${OPTIMIZE-${NISSY_BUILD_OPTIMIZE:-"-O3"}} - -validate_command() { - command="$1" - if ! (command -v "$command" >/dev/null 2>&1) ; then - echo "Error: command '$command' not found" - exit 1 - fi -} - -validate_threads() { - threads="$1" - if [ "$threads" -le 0 ] || [ "$threads" -gt 128 ]; then - echo "Error: number of threads must be between 1 and 128" - exit 1 - fi -} - -validate_arch() { - arch="$1" - case "$arch" in - AVX2|NEON|PORTABLE) - ;; - *) - echo "Error: architecture '$arch' not supported" - echo "Supported architectures: AVX2, NEON, PORTABLE" - exit 1 - ;; - esac -} - -parsesanitize() { - # Use the user-specified comma-separated sanitizers - for opt in $(echo "$1" | tr ',' '\n'); do - printf -- '-fsanitize=%s ' "$opt" - done -} - -maybe_pthread() { - if [ "$THREADS" -gt 1 ]; then - echo "-pthread" - else - echo "" - fi -} - -# Build flags -CFLAGS="-std=c11 -fPIC -D_POSIX_C_SOURCE=199309L $(maybe_pthread)" -PYCFLAGS="-std=c11 -fPIC $(maybe_pthread)" -[ "$ARCH" = "AVX2" ] && CFLAGS="$CFLAGS -mavx2" -WFLAGS="-pedantic -Wall -Wextra -Wno-unused-parameter -Wno-unused-function" -OFLAGS="$OPTIMIZE" -DFLAGS="-DDEBUG -g3 $(parsesanitize "$SANITIZE")" -MFLAGS="-DTHREADS=$THREADS -D$ARCH" -CPPFLAGS="-std=c++20 $(maybe_pthread)" - -# TODO: -# MEMORY64 is supported on Firefox (from version 134) and Chrome (from 133), -# but not on Safari (nor on e.g. Firefox 128 ESR, current default on Debian). -# See also https://webassembly.org/features -# When it becomes widely available, we can support it by adding -sMEMORY64 -# to the WASMCFLAGS, WASMCPPFLAGS and WASMLINKFLAGS variables below, and -# -sMAXIMUM_MEMORY=10737418240 to WASMLINKFLAGS. This way we can enable -# solvers larger than h48h6k2 in the web version. - -# TODO: -# The options below have to be adjusted when native WASM_SIMD is implemented. - -# Build flags for emscripten (WASM target) -WASMCFLAGS="-std=c11 -fPIC -D_POSIX_C_SOURCE=199309L $(maybe_pthread) - -mfpu=neon -mrelaxed-simd" -WASMCPPFLAGS="-std=c++20 $(maybe_pthread)" -WASMDBGFLAGS="-sASSERTIONS" -WASMMFLAGS="-DTHREADS=$THREADS -DNEON" -WASMLINKFLAGS="--no-entry -sEXPORT_NAME='Nissy' -sMODULARIZE - -sEXPORTED_RUNTIME_METHODS=addFunction,UTF8ToString - -sALLOW_TABLE_GROWTH - -sALLOW_MEMORY_GROWTH -sSTACK_SIZE=5MB -sPTHREAD_POOL_SIZE=$THREADS - -sFETCH -sASYNCIFY -sLINKABLE -sEXPORT_ALL" - -if (command -v "python3-config" >/dev/null 2>&1) ; then - PYTHON3_INCLUDES="$(python3-config --includes)" - PYTHON3="version $(echo "$PYTHON3_INCLUDES" | sed 's/.*3\./3./')" -else - PYTHON3_INCLUDES="" - PYTHON3="Not found, Python shell won't be available" -fi - -build_help() { - echo "Build system for nissy. Usage:" - echo "" - echo "$0 [-d] TARGET # build the given TARGET" - echo "$0 [-d] # same as '$0 nissy [-d]'" - echo "$0 test [PATTERN] # run unit tests (matching PATTERN)" - echo "$0 webtest [PATTERN] # same as test, but for WASM build" - echo "$0 [-d] tool PATTERN # run the tool matching PATTERN" - echo "" - echo "targets:" - echo "" - echo "nissy Build the nissy.o object file" - echo "lib Build the static library libnissy.a" - echo "sharedlib Build the shared library libnissy.so" - echo "python Build the Python module for nissy" - echo "shell Build a basic nissy shell (./run)" - echo "web Build the WebAssembly / JavaScript module for nissy" - echo "cpp FILES Build and run the given FILES including cpp/nissy.h" - echo "solvetest Build nissy and run the tests for solvers (tools)" - echo "" - echo "help Show this help message" - echo "config Show build configuration and exit" - echo "clean Remove all build files" - echo "" - echo "The -d option activates debug mode (slower, used for testing)." - echo "Tests are automatically built in debug mode even without -d." - echo "For more on build configurations, see the comments in ./build" -} - -build_config() { - echo "Compiler: $CC" - echo "Architecture: $ARCH" - echo "Max threads: $THREADS" - echo "Optimization options: $OFLAGS" - echo "Debug flags: $DFLAGS" - echo "" - - echo "Optional features:" - - printf '%s' "WASM compiler: " - if (validate_command "$EMCC" >/dev/null); then - echo "$EMCC" - else - echo "Not found, web target won't be available" - fi - - printf '%s' "nodejs executable: " - if (validate_command "$NODE" >/dev/null); then - echo "$NODE" - else - echo "Not found, unit tests won't be available for web target" - fi - - echo "Python bindings: $PYTHON3" -} - -run() { - echo "$@" - $@ -} - -odflags() { - if [ "$debug" = "yes" ]; then - echo "$DFLAGS" - else - echo "$OFLAGS" - fi -} - -build_clean() { - run rm -rf -- *.o *.so *.a run runtest runtool runcpp \ - web/nissy_web_module.* web/http/nissy_web_module.* -} - -build_nissy() { - validate_command "$CC" - validate_threads "$THREADS" - validate_arch "$ARCH" - - run $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -c -o nissy.o src/nissy.c -} - -build_lib() { - build_nissy - run ar rcs libnissy.a nissy.o -} - -build_sharedlib() { - validate_command "$CC" - validate_threads "$THREADS" - validate_arch "$ARCH" - - run $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -shared -c -o nissy.o \ - src/nissy.c -} - -build_shell() { - build_nissy || exit 1 - run $CC $CFLAGS $WFLAGS $(odflags) -o run nissy.o shell/shell.c -} - -build_python() { - if [ -z "$PYTHON3_INCLUDES" ]; then - echo "Python3 development headers could not be located" - echo "Cannot build python module" - exit 1 - fi - if [ "$debug" = "yes" ]; then - echo "Cannot build Python library in debug mode" - exit 1 - fi - build_nissy || exit 1 - run $CC $PYCFLAGS $WFLAGS $PYTHON3_INCLUDES $OFLAGS -shared \ - -o python/nissy.so nissy.o python/nissy_module.c -} - -build_cpp() { - validate_command "$CXX" - if [ -z "$@" ]; then - echo "Please provide one or more valid C++ source files" - echo "usage: ./build cpp FILES" - fi - - build_nissy || exit 1 - run $CXX $(odflags) -std=c++20 -o runcpp cpp/nissy.cpp nissy.o $@ \ - || exit 1 - run ./runcpp -} - -build_web() { - validate_command "$EMCC" - validate_threads "$THREADS" - validate_arch "$ARCH" - - obj="nissy_web_module" - - if [ "$debug" = "yes" ]; then - ODFLAGS="$DFLAGS -sASSERTIONS" - else - ODFLAGS="$OFLAGS" - fi - - run $EMCC $WASMCFLAGS $WFLAGS $WASMMFLAGS $ODFLAGS -c \ - -o nissy.o src/nissy.c || exit 1 - run $EMCC -lembind -lidbfs.js \ - $WASMCPPFLAGS $ODFLAGS $WASMLINKFLAGS -o web/"$obj".mjs \ - cpp/nissy.cpp web/storage.cpp web/adapter.cpp nissy.o || exit 1 - cp web/"$obj".mjs web/http/ - cp web/"$obj".wasm web/http/ -} - -dotest() { - testout="test/last.out" - testerr="test/last.err" - - # Verify that $t is a directory and it starts with three digits - if [ ! -d "$t" ] || ! (basename "$t" | grep -Eq '^[0-9]{3}'); then - return 0 - fi - - # Verify that the test is selected by the given pattern - if [ -n "$pattern" ] && ! (echo "$t" | grep -q "$pattern"); then - return 0 - fi - - $testbuild "$t"/*.c nissy.o || exit 1 - for cin in "$t"/*.in; do - c="$(echo "$cin" | sed 's/\.in//')" - cout="$c.out" - printf '%s: ' "$c" - $testrun < "$cin" > "$testout" 2> "$testerr" - if diff "$cout" "$testout"; then - printf "OK\n" - else - printf "Test failed! stderr:\n" - cat "$testerr" - exit 1 - fi - done -} - -build_test_generic() { - if [ -n "$1" ]; then - pattern="$1" - shift - fi - debug="yes" - build_$obj || exit 1 - for t in test/*; do - dotest || exit 1 - done - echo "All tests passed!" -} - -build_test() { - obj="nissy" - testobj="runtest" - testbuild="$CC $CFLAGS $WFLAGS $DFLAGS $MFLAGS -o $testobj" - testrun="./$testobj" - build_test_generic $@ - rm -f runtest -} - -build_webtest() { - obj="web" - testobj="runtest.js" - testbuild="$EMCC $WASMCFLAGS $WFLAGS $WASMMFLAGS $DFLAGS \ - -sSTACK_SIZE=5MB -o $testobj" - testrun="$NODE $testobj" - build_test_generic $@ - rm -f runtest.js runtest.wasm -} - -run_single_tool() { - results="tools/results" - last="$results/last.out" - date="$(date +'%Y-%m-%d-%H-%M-%S')" - file="$results/$toolname-$date.txt" - failed="tools/failed" - - $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -o runtool "$t"/*.c nissy.o \ - || exit 1 - - ( - date +'%Y-%m-%d %H:%M' - echo "" - echo "=========== Running tool ===========" - echo "tool name: $toolname" - echo "" - echo "======== nissy build command ========" - echo "$CC $CFLAGS $WFLAGS $MFLAGS $(odflags)" - echo "" - echo "======== tool build command ========" - echo "$CC $CFLAGS $WFLAGS $MFLAGS $(odflags)" - echo "" - echo "============ tool output ============" - ./runtool $@ || touch "$failed" - ) | tee "$file" "$last" - - rm -f runtool - if [ -f "$failed" ]; then - rm "$failed" - exit 1 - fi -} - -build_tool() { - pattern="$1" - - if [ -z "$pattern" ]; then - echo "Please provide a valid PATTERN to select a tool" - echo "usage: ./build tool PATTERN" - exit 1 - fi - shift - - # Select a single tool matched by the given pattern - for t in tools/*; do - if [ -d "$t" ] && (echo "$t" | grep -q "$pattern"); then - toolname="$(basename "$t" .c)" - break - fi - done - if [ -z "$toolname" ]; then - echo "pattern '$pattern' does not match any tool" - exit 1 - fi - - build_nissy || exit 1 - run_single_tool $@ -} - -build_solvetest() { - build_nissy || exit 1 - for t in tools/*; do - if [ -d "$t" ] && (echo "$t" | grep -q "solvetest"); then - toolname="$(basename "$t" .c)" - run_single_tool || exit 1 - fi - done -} - -if [ "$1" = "-d" ]; then - debug="yes" - shift -fi - -target="$1" -if [ -z "$target" ]; then - target="nissy" -else - shift -fi - -case "$target" in -help|config|clean|\ -nissy|lib|sharedlib|shell|python|cpp|web|test|webtest|tool|solvetest) - mkdir -p tables tools/results - (build_"$target" $@) || exit 1 - exit 0 - ;; -*) - echo "Target '$target' unavailable, run '$0 help' for info" - exit 1 - ;; -esac diff --git a/build.sh b/build.sh @@ -0,0 +1,475 @@ +#!/bin/sh + +# Build system for nissy, run './build.sh help' for info on how to use this. + +# The variables below can be used to personalize the build system. For example, +# to compile with clang instead of the default cc one can use: +# +# CC=clang ./build.sh +# +# Each variable also has a counterpart that starts with NISSY_BUILD_*. These +# other variables can be set, for example, in your shell configuration file +# to change the default build options. They are still overwritten by the values +# of the non-NISSY_BUILD_* variables, if set. +# For example, on a system that supports the addressed and undefined behavior +# sanitizers, one can set +# +# export NISSY_BUILD_SANITIZE="address,undefined" +# +# so that these sanitizers will be enabled when building nissy in debug mode. +# If later the same user wants to build a debug version without sanitizers, +# they may use +# +# SANITIZE="" ./build.sh debug +# +# And the empty string value will take precedence. + +# Specify the C compiler to use. +CC=${CC-${NISSY_BUILD_CC:-"cc"}} + +# Specify the C++ compiler for the C++ examples. +CXX=${CXX-${NISSY_BUILD_CXX:-"c++"}} + +# Specify the compiler to use when building for WASM. +EMCC=${EMCC-${NISSY_BUILD_EMCC:-"emcc"}} + +# Specify the nodejs command for running tests for WASM. +NODE=${NODE-${NISSY_BUILD_NODE:-"node"}} + +# The maximum number of threads to use for multi-threaded operations. +# This is also used as default value in case an operation allows +# specifying how many threads to use. +# The number n must be between 1 and 128. +# The default value is 16. +THREADS=${THREADS-${NISSY_BUILD_THREADS}} + +detectthreads() { + # TODO: detect based on system + # Is 'getconf NPROCESSORD_ONLN' portable? Is it threads or cores? + echo "16" +} + +[ -z "$THREADS" ] && THREADS="$(detectthreads)" + +# You can use this variable to build for a different architecture, for example +# if you want to cross-compile or to use the portable version. +# The value must be one of "AVX2", "NEON", "PORTABLE". If the value is not +# specified, the script will automatically detect the best architecture for the +# system. The architecture "PORTABLE" will work on any system. +ARCH=${ARCH-${NISSY_BUILD_ARCH}} + +greparch() { + $CC -march=native -dM -E - </dev/null 2>/dev/null | grep "$1" +} + +detectarch() { + [ -n "$(greparch __AVX2__)" ] && detected="AVX2" + [ -n "$(greparch __ARM_NEON)" ] && detected="NEON" + [ -z "$detected" ] && detected="PORTABLE" + echo "$detected" +} + +[ -z "$ARCH" ] && ARCH="$(detectarch)" + +# SANITIZE="option1,option2,..." adds the options "-fsanitize=option1", +# "-fsanitize=option2", ... to the C compiler command in debug mode. +# By default none is used because these options are not available on all +# systems. It is advisable to set the NISSY_BUILD_SANITIZE options in your +# shell's configuration file if your system does support them. +SANITIZE=${SANITIZE-${NISSY_BUILD_SANITIZE:-""}} + +# Optimization flags. +OPTIMIZE=${OPTIMIZE-${NISSY_BUILD_OPTIMIZE:-"-O3"}} + +validate_command() { + command="$1" + if ! (command -v "$command" >/dev/null 2>&1) ; then + echo "Error: command '$command' not found" + exit 1 + fi +} + +validate_threads() { + threads="$1" + if [ "$threads" -le 0 ] || [ "$threads" -gt 128 ]; then + echo "Error: number of threads must be between 1 and 128" + exit 1 + fi +} + +validate_arch() { + arch="$1" + case "$arch" in + AVX2|NEON|PORTABLE) + ;; + *) + echo "Error: architecture '$arch' not supported" + echo "Supported architectures: AVX2, NEON, PORTABLE" + exit 1 + ;; + esac +} + +parsesanitize() { + # Use the user-specified comma-separated sanitizers + for opt in $(echo "$1" | tr ',' '\n'); do + printf -- '-fsanitize=%s ' "$opt" + done +} + +maybe_pthread() { + if [ "$THREADS" -gt 1 ]; then + echo "-pthread" + else + echo "" + fi +} + +# Build flags +CFLAGS="-std=c11 -fPIC -D_POSIX_C_SOURCE=199309L $(maybe_pthread)" +PYCFLAGS="-std=c11 -fPIC $(maybe_pthread)" +[ "$ARCH" = "AVX2" ] && CFLAGS="$CFLAGS -mavx2" +WFLAGS="-pedantic -Wall -Wextra -Wno-unused-parameter -Wno-unused-function" +OFLAGS="$OPTIMIZE" +DFLAGS="-DDEBUG -g3 $(parsesanitize "$SANITIZE")" +MFLAGS="-DTHREADS=$THREADS -D$ARCH" +CPPFLAGS="-std=c++20 $(maybe_pthread)" + +# TODO: +# MEMORY64 is supported on Firefox (from version 134) and Chrome (from 133), +# but not on Safari (nor on e.g. Firefox 128 ESR, current default on Debian). +# See also https://webassembly.org/features +# When it becomes widely available, we can support it by adding -sMEMORY64 +# to the WASMCFLAGS, WASMCPPFLAGS and WASMLINKFLAGS variables below, and +# -sMAXIMUM_MEMORY=10737418240 to WASMLINKFLAGS. This way we can enable +# solvers larger than h48h6k2 in the web version. + +# TODO: +# The options below have to be adjusted when native WASM_SIMD is implemented. + +# Build flags for emscripten (WASM target) +WASMCFLAGS="-std=c11 -fPIC -D_POSIX_C_SOURCE=199309L $(maybe_pthread) + -mfpu=neon -mrelaxed-simd" +WASMCPPFLAGS="-std=c++20 $(maybe_pthread)" +WASMDBGFLAGS="-sASSERTIONS" +WASMMFLAGS="-DTHREADS=$THREADS -DNEON" +WASMLINKFLAGS="--no-entry -sEXPORT_NAME='Nissy' -sMODULARIZE + -sEXPORTED_RUNTIME_METHODS=addFunction,UTF8ToString + -sALLOW_TABLE_GROWTH + -sALLOW_MEMORY_GROWTH -sSTACK_SIZE=5MB -sPTHREAD_POOL_SIZE=$THREADS + -sFETCH -sASYNCIFY -sLINKABLE -sEXPORT_ALL" + +if (command -v "python3-config" >/dev/null 2>&1) ; then + PYTHON3_INCLUDES="$(python3-config --includes)" + PYTHON3="version $(echo "$PYTHON3_INCLUDES" | sed 's/.*3\./3./')" +else + PYTHON3_INCLUDES="" + PYTHON3="Not found, Python shell won't be available" +fi + +build_help() { + echo "Build system for nissy. Usage:" + echo "" + echo "$0 [-d] TARGET # build the given TARGET" + echo "$0 [-d] # same as '$0 nissy [-d]'" + echo "$0 test [PATTERN] # run unit tests (matching PATTERN)" + echo "$0 webtest [PATTERN] # same as test, but for WASM build" + echo "$0 [-d] tool PATTERN # run the tool matching PATTERN" + echo "" + echo "targets:" + echo "" + echo "nissy Build the nissy.o object file" + echo "lib Build the static library libnissy.a" + echo "sharedlib Build the shared library libnissy.so" + echo "python Build the Python module for nissy" + echo "shell Build a basic nissy shell (./run)" + echo "web Build the WebAssembly / JavaScript module for nissy" + echo "cpp FILES Build and run the given FILES including cpp/nissy.h" + echo "solvetest Build nissy and run the tests for solvers (tools)" + echo "" + echo "help Show this help message" + echo "config Show build configuration and exit" + echo "clean Remove all build files" + echo "" + echo "The -d option activates debug mode (slower, used for testing)." + echo "Tests are automatically built in debug mode even without -d." + echo "For more on build configurations, see the comments in ./build.sh" +} + +build_config() { + echo "Compiler: $CC" + echo "Architecture: $ARCH" + echo "Max threads: $THREADS" + echo "Optimization options: $OFLAGS" + echo "Debug flags: $DFLAGS" + echo "" + + echo "Optional features:" + + printf '%s' "WASM compiler: " + if (validate_command "$EMCC" >/dev/null); then + echo "$EMCC" + else + echo "Not found, web target won't be available" + fi + + printf '%s' "nodejs executable: " + if (validate_command "$NODE" >/dev/null); then + echo "$NODE" + else + echo "Not found, unit tests won't be available for web target" + fi + + echo "Python bindings: $PYTHON3" +} + +run() { + echo "$@" + $@ +} + +odflags() { + if [ "$debug" = "yes" ]; then + echo "$DFLAGS" + else + echo "$OFLAGS" + fi +} + +build_clean() { + run rm -rf -- *.o *.so *.a run runtest runtool runcpp \ + web/nissy_web_module.* web/http/nissy_web_module.* +} + +build_nissy() { + validate_command "$CC" + validate_threads "$THREADS" + validate_arch "$ARCH" + + run $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -c -o nissy.o src/nissy.c +} + +build_lib() { + build_nissy + run ar rcs libnissy.a nissy.o +} + +build_sharedlib() { + validate_command "$CC" + validate_threads "$THREADS" + validate_arch "$ARCH" + + run $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -shared -c -o nissy.o \ + src/nissy.c +} + +build_shell() { + build_nissy || exit 1 + run $CC $CFLAGS $WFLAGS $(odflags) -o run nissy.o shell/shell.c +} + +build_python() { + if [ -z "$PYTHON3_INCLUDES" ]; then + echo "Python3 development headers could not be located" + echo "Cannot build python module" + exit 1 + fi + if [ "$debug" = "yes" ]; then + echo "Cannot build Python library in debug mode" + exit 1 + fi + build_nissy || exit 1 + run $CC $PYCFLAGS $WFLAGS $PYTHON3_INCLUDES $OFLAGS -shared \ + -o python/nissy.so nissy.o python/nissy_module.c +} + +build_cpp() { + validate_command "$CXX" + if [ -z "$@" ]; then + echo "Please provide one or more valid C++ source files" + echo "usage: ./build.sh cpp FILES" + fi + + build_nissy || exit 1 + run $CXX $(odflags) -std=c++20 -o runcpp cpp/nissy.cpp nissy.o $@ \ + || exit 1 + run ./runcpp +} + +build_web() { + validate_command "$EMCC" + validate_threads "$THREADS" + validate_arch "$ARCH" + + obj="nissy_web_module" + + if [ "$debug" = "yes" ]; then + ODFLAGS="$DFLAGS -sASSERTIONS" + else + ODFLAGS="$OFLAGS" + fi + + run $EMCC $WASMCFLAGS $WFLAGS $WASMMFLAGS $ODFLAGS -c \ + -o nissy.o src/nissy.c || exit 1 + run $EMCC -lembind -lidbfs.js \ + $WASMCPPFLAGS $ODFLAGS $WASMLINKFLAGS -o web/"$obj".mjs \ + cpp/nissy.cpp web/storage.cpp web/adapter.cpp nissy.o || exit 1 + cp web/"$obj".mjs web/http/ + cp web/"$obj".wasm web/http/ +} + +dotest() { + testout="test/last.out" + testerr="test/last.err" + + # Verify that $t is a directory and it starts with three digits + if [ ! -d "$t" ] || ! (basename "$t" | grep -Eq '^[0-9]{3}'); then + return 0 + fi + + # Verify that the test is selected by the given pattern + if [ -n "$pattern" ] && ! (echo "$t" | grep -q "$pattern"); then + return 0 + fi + + $testbuild "$t"/*.c nissy.o || exit 1 + for cin in "$t"/*.in; do + c="$(echo "$cin" | sed 's/\.in//')" + cout="$c.out" + printf '%s: ' "$c" + $testrun < "$cin" > "$testout" 2> "$testerr" + if diff "$cout" "$testout"; then + printf "OK\n" + else + printf "Test failed! stderr:\n" + cat "$testerr" + exit 1 + fi + done +} + +build_test_generic() { + if [ -n "$1" ]; then + pattern="$1" + shift + fi + debug="yes" + build_$obj || exit 1 + for t in test/*; do + dotest || exit 1 + done + echo "All tests passed!" +} + +build_test() { + obj="nissy" + testobj="runtest" + testbuild="$CC $CFLAGS $WFLAGS $DFLAGS $MFLAGS -o $testobj" + testrun="./$testobj" + build_test_generic $@ + rm -f runtest +} + +build_webtest() { + obj="web" + testobj="runtest.js" + testbuild="$EMCC $WASMCFLAGS $WFLAGS $WASMMFLAGS $DFLAGS \ + -sSTACK_SIZE=5MB -o $testobj" + testrun="$NODE $testobj" + build_test_generic $@ + rm -f runtest.js runtest.wasm +} + +run_single_tool() { + results="tools/results" + last="$results/last.out" + date="$(date +'%Y-%m-%d-%H-%M-%S')" + file="$results/$toolname-$date.txt" + failed="tools/failed" + + $CC $CFLAGS $WFLAGS $MFLAGS $(odflags) -o runtool "$t"/*.c nissy.o \ + || exit 1 + + ( + date +'%Y-%m-%d %H:%M' + echo "" + echo "=========== Running tool ===========" + echo "tool name: $toolname" + echo "" + echo "======== nissy build command ========" + echo "$CC $CFLAGS $WFLAGS $MFLAGS $(odflags)" + echo "" + echo "======== tool build command ========" + echo "$CC $CFLAGS $WFLAGS $MFLAGS $(odflags)" + echo "" + echo "============ tool output ============" + ./runtool $@ || touch "$failed" + ) | tee "$file" "$last" + + rm -f runtool + if [ -f "$failed" ]; then + rm "$failed" + exit 1 + fi +} + +build_tool() { + pattern="$1" + + if [ -z "$pattern" ]; then + echo "Please provide a valid PATTERN to select a tool" + echo "usage: ./build.sh tool PATTERN" + exit 1 + fi + shift + + # Select a single tool matched by the given pattern + for t in tools/*; do + if [ -d "$t" ] && (echo "$t" | grep -q "$pattern"); then + toolname="$(basename "$t" .c)" + break + fi + done + if [ -z "$toolname" ]; then + echo "pattern '$pattern' does not match any tool" + exit 1 + fi + + build_nissy || exit 1 + run_single_tool $@ +} + +build_solvetest() { + build_nissy || exit 1 + for t in tools/*; do + if [ -d "$t" ] && (echo "$t" | grep -q "solvetest"); then + toolname="$(basename "$t" .c)" + run_single_tool || exit 1 + fi + done +} + +if [ "$1" = "-d" ]; then + debug="yes" + shift +fi + +target="$1" +if [ -z "$target" ]; then + target="nissy" +else + shift +fi + +case "$target" in +help|config|clean|\ +nissy|lib|sharedlib|shell|python|cpp|web|test|webtest|tool|solvetest) + mkdir -p tables tools/results + (build_"$target" $@) || exit 1 + exit 0 + ;; +*) + echo "Target '$target' unavailable, run '$0 help' for info" + exit 1 + ;; +esac diff --git a/python/examples/solve.py b/python/examples/solve.py @@ -1,6 +1,6 @@ # Small example of nissy Python module usage -# Run "./build python", then run this from either the top-level directory +# Run "./build.sh python", then run this from either the top-level directory # of the nissy-core repo or from the python subdirectory. # Append the directories to the python path so we can load the module diff --git a/utils/container-run.sh b/utils/container-run.sh @@ -4,14 +4,14 @@ # It can be used for testing on platforms different from the host. For example, # the command # -# ./container-run.sh arm ./build test +# ./container-run.sh arm ./build.sh test # # builds nissy and runs the unit tests in an ARM container. # # The containers are based on Alpine Linux and the contain the necessary tools # to build the main library and the C++ and Python examples. They DO NOT # contain the emscripten compiler, so they cannot be used to build the web -# version (e.g. ./build web or ./buidl webtest). +# version (e.g. ./build.sh web or ./build.sh webtest). # The images are given a tag starting with 'localhost/nissy/'. # # See below for a list of options. @@ -28,7 +28,7 @@ usage() { echo "arm (equivalent to: arm64)" echo "" echo "Examples:" - echo "$0 ram ./build test # Run unit tests in arm container" + echo "$0 ram ./build.sh test # Run unit tests in arm container" exit 1 } diff --git a/web/http/nissyapp.mjs b/web/http/nissyapp.mjs @@ -1,6 +1,6 @@ -// Run ./build web from the main folder before running this example. +// Run ./build.sh web from the main folder before running this example. // The necessary modules (including worker.mjs) will be built and / or moved -// to this folder when running ./build web. +// to this folder when running ./build.sh web. var solveButton = document.getElementById("solveButton"); var pauseResumeButton = document.getElementById("pauseResumeButton");