Build System

XWOS build system

As the saying goes: “A craftsman must first sharpen his tools.” Every great idea needs an appropriate development environment to put into practice.

Overview

The XWOS build system is written based on make, with the following features:

  • Supports pure command-line compilation, suitable for continuous integration environments;
  • Cross-platform: supports Linux, Windows, macOS;
  • Supports Eclipse-based IDEs;
  • To facilitate integration of third-party software, XWOS references Android’s Android.mk build system design to create xwmo.mk, called XuanWu Module (XWMO).

Build Flow

flowchart TD
    make("make") --> cfg
    cfg["Generate Config"] --> arch
    arch["Compile arch.a"] --> cpu
    cpu["Compile cpu.a"] --> soc
    soc["Compile soc.a"] --> brd
    brd["Compile brd.a"] --> xwos
    xwos["Compile XWOS Kernel"] --> xwmd
    xwmd["Compile Middleware Modules"] --> xwcd
    xwcd["Compile Device Modules"] --> bm
    bm["Compile Board Modules"] --> xwem
    xwem["Compile Third-party Modules"] --> xwam
    xwam["Compile Application Modules"] --> oem
    oem["Compile OEM Modules"] --> elf
    elf["Link ELF File"] --> bin
    bin["Generate bin File"]

Start

The XWOS build begins by executing the make command in the board description layer directory, represented by $(XWOS_BRD_DIR). When executing make, the following parameters can be passed:

  • WKSPC=output-dir: configure the output file path, which can be relative to the board description layer directory or an absolute path;
  • XWOS=topdir: configure the XWOS root path, which can be relative to the board description layer directory or an absolute path.

Generate Configuration

The build system calls the script XWOS/xwbs/util/el/mkcfg.el to process all configuration files under $(XWOS_BRD_DIR)/cfg, converting them into three files:

  • $(XWOS_WKSPC_DIR)/XWOS.cfg, containing makefile environment variables, later included by the makefile.
    • Contains key path information, through which different SOCs locate their respective build configurations:
      • $(XWOS_PATH): root directory of XWOS source code
      • $(XWOS_ARCH_DIR): architecture-related source code path;
      • $(XWOS_CPU_DIR): CPU-related source code path;
      • $(XWOS_SOC_DIR): SOC-related source code path;
      • $(XWOS_BRD_DIR): board-related source code path, also where make was initially executed;
      • $(XWOS_BM_DIR): board-related XuanWu module path;
      • $(XWOS_OEM_DIR): OEM-related XuanWu module path, typically private code;
      • $(XWOS_OBJ_DIR): $(XWOS_WKSPC_DIR)/obj, the build output directory;
  • $(XWOS_BRD_DIR)/cfg/autogen.h: auto-generated header file, included by the top-level header xwos/standard.h
  • $(XWOS_WKSPC_DIR)/XWOS.cfg.rs: Rust language configuration
  • $(XWOS_WKSPC_DIR)/env.rc: shell environment variable script, which can be sourced via the source command to activate Auxiliary Functions.

Compile arch.a

The build system compiles the source code of the Arch Description Layer according to the build rules $(XWOS_ARCH_DIR)/arch.mk. The header file search starts from $(XWOS_PATH). After compilation, the static library $(XWOS_OBJ_DIR)/$(XWOS_ARCH_DIR)/arch.a is output.

Compile cpu.a

The build system compiles the source code of the CPU Description Layer according to the build rules $(XWOS_CPU_DIR)/arch.mk. The header file search starts from $(XWOS_PATH). After compilation, the static library $(XWOS_OBJ_DIR)/$(XWOS_CPU_DIR)/cpu.a is output.

Compile soc.a

The build system compiles the source code of the SOC Description Layer according to the build rules $(XWOS_SOC_DIR)/soc.mk. The header file search starts from $(XWOS_PATH). After compilation, the static library $(XWOS_OBJ_DIR)/$(XWOS_SOC_DIR)/soc.a is output.

Compile brd.a

The build system compiles the source code of the Board Description Layer according to the build rules $(XWOS_BRD_DIR)/brd.mk. The header file search starts from $(XWOS_PATH). After compilation, the static library $(XWOS_OBJ_DIR)/$(XWOS_BRD_DIR)/brd.a is output.

Compile XWOS Kernel

The build system compiles the XWOS kernel source code according to the build rules xwos/xwos.mk. The header file search starts from $(XWOS_PATH). After compilation, the static library $(XWOS_OBJ_DIR)/$(XWOS_XWOS_DIR)/xwos.a is output.

Compile Middleware Modules

  • The build system recursively scans the xwmd/ folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR) directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro XWMDCFG_ in $(XWOS_BRD_DIR)/cfg/xwmd.h:

Compile Device Modules

  • The build system recursively scans the xwcd/ folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR) directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro XWCDCFG_ in $(XWOS_BRD_DIR)/cfg/xwcd.h:

Compile Board Modules

  • The build system recursively scans the $(XWOS_BM_DIR) folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR) directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro BMCFG_ in $(XWOS_BRD_DIR)/cfg/board.h:

Compile Third-party Software Modules

  • The build system recursively scans the xwem/ folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR) directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro XWEMCFG_ in $(XWOS_BRD_DIR)/cfg/xwem.h:

Compile Application Modules

  • The build system recursively scans the xwam/ folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR) directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro XWAMCFG_ in $(XWOS_BRD_DIR)/cfg/xwam.h:

Compile OEM Modules

  • The OEM folder path is specified by the configuration XWCFG_OEMPATH in the configuration file $(XWOS_BRD_DIR)/cfg/XWOS.h:
    • Can be a relative path relative to $(XWOS_BRD_DIR);
    • Can be an absolute path;
    • The path can point outside the XWOS root directory.
    • The configuration tool xwbs/util/el/mkcfg.el generates the variable $(XWOS_OEM_DIR) from this configuration;
  • The build system recursively scans the $(XWOS_OEM_DIR) folder and all subdirectories for xwmo.mk files, generating a module list, then compiles them into .a static libraries one by one, outputting to the $(XWOS_OBJ_DIR)/oem directory.
  • The header file search starts from $(XWOS_PATH).
  • Each xwmo.mk file represents a module. The build system spawns a separate make subprocess to compile it. The parameters defined within (source file list, additional compile options, header file paths, etc.) are independent for each module compilation subprocess.
  • Each middleware module has a compile switch macro OEMCFG_ in $(XWOS_BRD_DIR)/cfg/oem.h:

The build system ultimately links all .a static libraries generated above into the XWOS.elf file. The linker script is specified by XWCFG_LDSCRIPT in the configuration file $(XWOS_BRD_DIR)/cfg/XWOS.h.

Generate bin File

The build system converts the XWOS.elf file into .bin and .hex files.

Build Options

V

  • Effect: output the complete compilation process.
  • Values:
    • 1: enable option
    • 0: disable option (default)
  • Usage:
make V=1

D

  • Effect: optimize compilation, producing a smaller binary, but less suitable for debugging.
  • Values:
    • 1: enable option (default)
    • 0: disable option
  • Usage:
make D=0

XuanWu Module (XWMO)

XWOS’s device modules, middleware modules, third-party software modules, board modules, and OEM modules all use xwmo.mk to describe build rules, and are collectively referred to as XuanWu Modules (XWMO).

Each xwmo.mk file represents a XuanWu Module, similar in principle to Android.mk in the Android system. The build system spawns a separate make process to compile it, with the source file list, additional compile options, header file paths, etc., being independent for each XuanWu module compilation subprocess.

Example, xwmo.mk for xwam/example/cxx:

include $(XWOS_WKSPC_DIR)/XWOS.cfg                      # Include environment variables
include xwbs/functions.mk                               # Include Makefile functions defined by xwbs

XWMO_CSRCS := mif.c                                     # Specify C source files
XWMO_CFLAGS :=                                          # Specify additional C compile options

XWMO_CXXSRCS := task.cxx                                # Specify C++ source files
XWMO_CXXSRCS += test/vector.cxx                         # Add C++ source files
XWMO_CXXSRCS_gcc += test/literal.cxx test/exception.cxx # Add C++ source files only for gcc

XWMO_CXXFLAGS := -Wno-unused-value                      # Specify C++ compile options
XWMO_CXXFLAGS_gcc += -fexceptions                       # Add C++ compile options only for gcc

XWMO_INCDIRS := $(call getXwmoDir)                      # Specify additional header search paths, using getXwmoDir to get current xwmo path
include xwbs/xwmo.mk                                    # Include Makefile with build rules

Module Path Naming Rules

Since the module path needs to correspond to a C language macro as a compile switch, the module path must follow C language identifier rules, but may contain a few special characters:

  • Each level of the directory path must follow C language naming rules;
  • Directory levels are separated by /;
  • The path may contain ., but ../ and ./ are not allowed;
  • The path may contain -;

Module Compile Switch Naming Rules

A module must have a macro switch corresponding to its path, configured as 1, to be compiled:

  • Only the relative path portion of the module path needs to be converted:
    • Middleware module: take the path after xwmd/ (exclusive);
    • Device driver module: take the path after xwcd/ (exclusive);
    • Board module: take the path after $(XWOS_BRD_DIR)/bm/ (exclusive);
    • Third-party software module: take the path after xwem/ (exclusive);
    • Application module: take the path after xwam/ (exclusive);
    • OEM module: take the path after OEM folder/ (exclusive).
  • _ in the path must be represented by two _ characters;
  • / in the path is converted to _;
  • . in the path is converted to _;
  • - in the path is converted to _;
  • Add a prefix:
    • Middleware module: XWMDCFG_
    • Device module: XWCDCFG_
    • Board module: BMCFG_
    • Third-party software module: XWEMCFG_
    • Application module: XWAMCFG_
    • OEM module: OEMCFG_
  • Examples:
    • Middleware module: xwmd/isc/xwpcp –> XWMDCFG_isc_xwpcp;
    • Device driver module: xwcd/perpheral/ds/i2c/eeprom –> XWCDCFG_perpheral_ds_i2c_eeprom;
    • Board module: xwbd/WeActMiniStm32H750/bm/stm32_cube –> BMCFG_stm32__cube;
    • Third-party software module: xwem/vm/l__u_a-5.4.6 –> XWEMCFG_vm_l____u__a_5_4_6;
    • Application module: xwam/example/cxx –> XWAMCFG_example_cxx;
    • OEM module: oem/app –> OEMCFG_app;
  • The xwmc command from Auxiliary Functions can be used to generate the compile switch macro identifier.

Adding Modules

  1. Middleware Modules
  • Modules are located in the xwmd/ directory;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/xwmd.h, defined as 1 to compile the module.
  1. Device Driver Modules
  • Modules are located in the xwcd/ directory;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/xwcd.h, defined as 1 to compile the module.
  1. Board Modules
  • Modules are located in the $(XWOS_BRD_DIR)/bm/ directory;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/board.h, defined as 1 to compile the module.
  1. Third-party Software Modules
  • Modules are located in the xwem/ directory;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/xwem.h, defined as 1 to compile the module.
  1. Application Modules
  • Modules are located in the xwam/ directory;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/xwam.h, defined as 1 to compile the module.
  1. OEM Modules
  • Modules are located within the OEM folder;
  • The macro compile switch is in the file $(XWOS_BRD_DIR)/cfg/oem.h, defined as 1 to compile the module.

xwmo.mk Syntax Reference

xwmo.mk is part of the makefile. Its syntax is simplified to defining a few special variables to complete compilation.

C/C++

include $(XWOS_WKSPC_DIR)/XWOS.cfg              # Include configuration file
include xwbs/functions.mk                       # Include Makefile functions defined by xwbs
XWMO_ASRCS :=                                   # Assembly source files
XWMO_ASRCS_gcc :=                               # Assembly source files only for gcc
XWMO_ASRCS_llvm :=                              # Assembly source files only for llvm
XWMO_AFLAGS :=                                  # Assembly compile options
XWMO_AFLAGS_gcc :=                              # Assembly compile options only for gcc
XWMO_AFLAGS_llvm :=                             # Assembly compile options only for llvm
XWMO_CSRCS :=                                   # C source files
XWMO_CSRCS_gcc :=                               # C source files only for gcc
XWMO_CSRCS_llvm :=                              # C source files only for llvm
XWMO_CFLAGS :=                                  # C compile options
XWMO_CFLAGS_gcc :=                              # C compile options only for gcc
XWMO_CFLAGS_llvm :=                             # C compile options only for llvm
XWMO_CXXSRCS :=                                 # C++ source files
XWMO_CXXSRCS_gcc :=                             # C++ source files only for gcc
XWMO_CXXSRCS_llvm :=                            # C++ source files only for llvm
XWMO_CXXFLAGS :=                                # C++ compile options
XWMO_CXXFLAGS_gcc :=                            # C++ compile options only for gcc
XWMO_CXXFLAGS_llvm :=                           # C++ compile options only for llvm
XWMO_INCDIRS := $(call getXwmoDir)              # Get module path and add to header search path
XWMO_INCDIRS_gcc := $(call getXwmoDir)          # Get module path and add to gcc header search path
XWMO_INCDIRS_llvm := $(call getXwmoDir)         # Get module path and add to llvm header search path
XWMO_LUASRCS :=                                 # Lua script source files converted to C arrays
include xwbs/xwmo.mk                            # Include Makefile with build rules

RUST Module

include $(XWOS_WKSPC_DIR)/XWOS.cfg              # Include configuration file
include xwbs/functions.mk                       # Include Makefile functions defined by xwbs
include xwbs/xwmo.rust.mk                       # Include Makefile with build rules

Prebuilt Module

include $(XWOS_WKSPC_DIR)/XWOS.cfg              # Include configuration file
include xwbs/functions.mk                       # Include Makefile functions defined by xwbs
XWMO_PREBUILT :=                                # Pre-compiled .a file
include xwbs/xwmo.prebuilt.mk                   # Include Makefile with build rules

Functions Available in xwmo.mk

  • getXwmoDir

    • Usage: $(call getXwmoDir)
    • Description: get the current XuanWu Module path
  • getXwmoName

    • Usage: $(call getXwmoName)
    • Description: get the current XuanWu Module name
  • XwmoWildcard

    • Usage: $(call XwmoWildcard,WILDCARD,DIR)
    • Description: search for files matching wildcard WILDCARD in directory DIR, and output the file list.
    • Example
      • Example: XWMO_CSRCS += $(call XwmoWildcard,*.c,picolibc)
      • Explanation: search for all *.c files in picolibc, and assign the returned file list to XWMO_CSRCS
  • XwmoWildcardRecursively

    • Usage: $(call XwmoWildcardRecursively,WILDCARD,DIR)
    • Description: recursively search for files matching wildcard WILDCARD in directory DIR and its subdirectories, and output the file list.
  • XwmoReqCfg

    • Usage: $(call XwmoReqCfg,CFG,VALUE)
    • Description: test if configuration CFG equals VALUE; if not, report an error.
    • Example
      • Example: $(call XwmoReqCfg,XWCFG_LIBC,picolibc)
      • Explanation: report an error if configuration XWCFG_LIBC is not picolibc.
  • XwmoReqNotCfg

    • Usage: $(call XwmoReqNotCfg,CFG,VALUE)
    • Description: test if configuration CFG equals VALUE; if so, report an error.
    • Example
      • Example: $(call XwmoReqCfg,XWCFG_LIBC,picolibc)
      • Explanation: report an error if configuration XWCFG_LIBC is picolibc.

Auxiliary Functions

XWOS’s build system, similar to Android, also defines some build-related auxiliary commands.

Initialize Environment

source xwbd/WeActMiniStm32H750/env.sh # example for board WeActMiniStm32H750

Commands

  • xwmc
    • Function: get the C language macro identifier of a module’s compile switch.
    • Usage example:
cd xwem/vm/lua # enter the vm/lua module
xwmc
> XWEMCFG_vm_lua # output result
  • xwmn
    • Function: get the .a file name of a module.
    • Usage example:
cd xwem/vm/lua # enter the vm/lua module
xwmn
> xwem_vm_lua.a # output
  • xwm
    • Function: compile the entire XWOS project, similar to Android’s m command.
    • Usage: xwm [options] [target]
    • Option -B: full rebuild
    • Target: make target
    • Usage example:
xwm # compile entire project
xwm c # clean
xwm d # deep clean
  • xwmm
    • Function: compile a single module, similar to Android’s mm command, using the current path as the module path.
    • Option -B: full rebuild
    • Usage example:
cd xwem/vm/lua # enter the vm/lua module
xwmm
  • xwmmm
    • Function: compile a single module, similar to Android’s mmm command, requires specifying the module path.
    • Option -B: full rebuild
    • Usage example:
xwmmm xwem/vm/lua
  • xwcroot

    • Function: switch to the XWOS root directory, similar to Android’s croot command.
  • xwcbd

    • Function: switch to the board description layer directory.