Build System (LabBuildCompiler)

The Lab build system is the entry point and orchestration layer of the Chemical compiler. It reads build.lab or chemical.mod files, creates compilation jobs, manages dependencies, coordinates compiler plugins, and drives the entire multi-pass compilation pipeline.

Full Pipeline Overview

User Input
    │
    ├── chemical.mod → [ModToLabConverter] → build.lab (C code string)
    │
    └── build.lab → [2c backend] → C code → [TinyCC JIT] → executable build script
                        │
                        ▼
              LabBuildCompiler::process_modules()
                        │
                        ▼
              ┌─────────────────────────────────┐
              │         Job Execution            │
              │                                  │
              │  1. Parse all source files        │
              │     (lexer → parser → AST)        │
              │     Parallel per file             │
              │                                  │
              │  2. Symbol Resolution             │
              │     (6 parallel passes)           │
              │     See ASTProcessor section      │
              │                                  │
              │  3. Type Verification             │
              │     Parallel per file             │
              │                                  │
              │  4. C Translation (2c)            │
              │     Forward decl → Type aliases   │
              │     → Declare → Implement         │
              │                                  │
              │  5. Backend Codegen               │
              │     LLVM.cpp or TinyCC            │
              │                                  │
              │  6. Link                         │
              │     Produce executable/library    │
              └─────────────────────────────────┘

Phase 1: Input Loading

chemical.mod → build.lab (ModToLabConverter)

The ModToLabConverter (in compiler/lab/mod_conv/ModToLabConverter.cpp) translates a concise .mod file into a full .lab build script.

Input example (chemical.mod):

application my_app

source "src"
import "../lib_mod"
import "github.com/owner/repo" version "1.0.0"
link "mylib"
link c "helper.c"

Output (generated build.lab):

import lab;
import std;

import "../lib_mod/build.lab" as __mod_0_stmt;

public func build(ctx : *mut BuildContext, __chx_job : *mut LabJob) : *mut Module {
    var __curr_lab_path = lab::get_my_path();
    const __chx_already_exists = ctx.get_cached(__chx_job, &__curr_lab_path);
    if(__chx_already_exists != null) { return __chx_already_exists; }

    const deps : []ModuleDependency = [
        ModuleDependency { module: __mod_0_stmt.build(ctx, __chx_job), info: null },
    ];

    const mod = ctx.new_package(ModuleType.Directory, PackageKind.Application,
        "", "my_app",
        std::span<ModuleDependency>(deps, 1));
    ctx.set_cached(__chx_job, &__curr_lab_path, mod);

    // Remote imports
    ctx.fetch_mod_dependency(__chx_job, mod, ImportRepo {
        from: "github.com/owner/repo",
        version: "1.0.0",
    });

    // Source paths
    {
        var rel_path = lab::rel_path_to("src");
        ctx.add_path(mod, rel_path.to_view());
    }

    // Link statements
    ctx.link_system_lib(__chx_job, "mylib", mod);

    const c_file_path_0 = lab::rel_path_to("helper.c");
    const c_file_mod_0 = ctx.c_file_module("", "my_app_cfile_0", c_file_path_0.to_view(), std::span<*mut Module>());
    ctx.add_dependency(__chx_job, c_file_mod_0, null);

    return mod;
}

Key conversion logic in convertToBuildLab():

1. Imports for dependencies: Each import statement in .mod becomes a build.lab import with a unique identifier like __mod_N_stmt
2. Dependency list: Each import generates a ModuleDependency entry calling the imported module's build() function
3. Module creation: ctx.new_package() with the correct ModuleType and PackageKind
4. Remote imports: ctx.fetch_mod_dependency() with ImportRepo struct
5. Source paths: ctx.add_path(mod, path) for each source directory (path resolved via lab::rel_path_to)
6. Link statements: ctx.link_system_lib() for named libs, ctx.add_lib_search_path() for link paths, and ctx.c_file_module() + ctx.add_dependency() for C files
7. Conditional imports: if(windows) generates an if(__chx_job.getTarget().windows) block
8. Symbol info: For import Foo from "..." style, generates DependencySymbolInfo with alias and symbol list

build.lab → JIT-compiled executable

The build script itself is Chemical code that must be compiled and executed:

1. Lex, Parse: The build.lab is lexed and parsed like any Chemical file
2. C Translation (2c): The parsed AST is translated to C code via the 2c backend
3. TinyCC JIT: The generated C is compiled in-memory using TinyCC
4. Symbols exposed: LabBuildCompiler exposes CBI functions to the build script:
   • ctx.new_package() — create a module
   • ctx.set_cached() — cache the module
   • ctx.add_path() — add source paths
   • ctx.link_system_lib() — add link libraries
   • ctx.fetch_mod_dependency() — handle remote imports
   • etc.

The compiled build script is executed by calling its build() function, which returns a Module* containing all the module's metadata.

Phase 2: Job Creation

After the build script executes, LabBuildCompiler has a tree of LabModule objects representing the dependency graph. Each LabModule has:

struct LabModule {
    chem::string name;                    // Module name
    chem::string scope_name;              // Scope name (namespace)
    LabModuleType type;                   // Directory, Files, ObjFile, CFile, CPPFile
    PackageKind package_kind;             // Application or Library
    std::vector<ModuleDependency> dependencies;  // Dependencies
    std::vector<chem::string> paths;      // Source paths or file paths
    ModuleScope module_scope;             // Scope for symbol resolution
    std::vector<ASTFileMetaData> direct_files;  // Files belonging to this module
};

LabBuildCompiler::do_job_allocating() (via do_job) and process_modules() / process_job_tcc() / process_job_gen() process each module's files through the full pipeline.

Phase 3: ASTProcessor — The Compilation Orchestrator

ASTProcessor (in compiler/ASTProcessor.h/.cpp) is the core orchestrator that drives the compilation of each module. It owns the resolver, binder, allocators, and manages all passes.

Key Components in ASTProcessor

Member	Type	Purpose
resolver	SymbolResolver*	Symbol resolution
loc_man	LocationManager&	Source location management
controller	AnnotationController&	Annotation handling
binder	CompilerBinder&	CBI function binding
path_handler	ImportPathHandler&	Import resolution
container	InstantiationsContainer&	Generic instantiation tracking
file_allocator	ASTAllocator	Per-file arena allocator
mod_allocator	ASTAllocator	Per-module arena allocator
job_allocator	ASTAllocator	Per-job arena allocator
mod_storage	ModuleStorage&	Module storage and lookup
cache	std::unordered_map<unsigned int, std::unique_ptr<ASTFileResult>>	File result cache
import_mutex	std::mutex	Thread safety for imports
print_mutex	std::mutex	Thread safety for printing

The 6 Symbol Resolution Passes

ASTProcessor::sym_res_module() runs 6 passes in sequence:

Pass 1: Top-Level Declaration (Serial)

for(auto& file_ptr : module->direct_files) {
    file.private_symbol_range = sym_res_tld_declare_file(
        file.unit.scope.body, file.file_id, file.abs_path);
}

• What: Declares all top-level symbols (functions, structs, variants, namespaces, impls)
• Why serial: Declarations must happen in order to avoid race conditions on the shared symbol table
• Output: SymbolRange indicating start/end indices of file-private symbols
• Allocator cleared: file_allocator.clear() after pass

Pass 2: Link Signatures (Parallel per file)

// Parallel using thread pool
for(auto& file_ptr : module->direct_files) {
    futures.emplace_back(pool.push([this, &file](int id){
        auto res = link_sig_file_task(resolver, &file);
        // merge diagnostics with mutex
        return has_errors;
    }));
}

• What: Resolves all type signatures (function parameter types, return types, struct member types, variant member types, type aliases) — does NOT enter function bodies
• Why parallel: Each file has its own per-file SymbolTable and diagnoser
• Key detail: Uses sym_res_signature() which creates a per-file TopLevelLinkSignature with its own SymbolTable for file-private symbols (generic type params, using-imports, aliases)
• Output: SymResSignatureResult with inline instantiations collected
• Allocator cleared: file_allocator.clear() after pass

Between Pass 2 and Pass 3: generate_automatic_functions_for_module() runs a single serial recursive pass that generates automatic functions (constructors/destructors) for every container before generic instantiation, ensuring member destructors exist before dependent container destructors.

Pass 3: Generic Instantiation (Parallel per file)

for(auto& file_ptr : module->direct_files) {
    futures.emplace_back(pool.push([this, &file](int id){
        auto res = gen_inst_file_task(resolver, &file);
        return res.has_errors;
    }));
}

• What: Finalizes inline generic instantiations discovered during link signatures
• Key function: sym_res_generic_instantiation() in GenericInstantiationPass.cpp
  1. Calls GenericTypeDecl::finalize_signature() for each inline instantiation
  2. Calls GenericInstantiator::FinalizeSignature() to resolve generic type parameters
  3. Visits the scope to trigger nested generic instantiations
• Why separate: Generic types discovered during link signatures need to be instantiated BEFORE body resolution so that bodies can reference concrete types
• Registration only: This pass requires only InstantiationRequirement::Registration — not full body finalization
• Allocator cleared: file_allocator.clear() after pass

Pass 4: After Link Signature (Serial)

for(auto& file_ptr : module->direct_files) {
    sym_res_after_link_sig_file(file.unit.scope.body, file.file_id,
        file.abs_path, file.private_symbol_range);
}

• What: Calls sym_res_after_signature() — handles tasks that must happen after all signatures are linked but before bodies
• Why serial: May modify shared resolver state
• Allocator cleared: file_allocator.clear() after pass

Pass 5: Link Body — Generic Decls (Parallel per file)

for(auto& file_ptr : module->direct_files) {
    futures.emplace_back(pool.push([this, &file](int id){
        auto res = link_body_generic_decls_task(resolver, &file);
        return res.has_errors;
    }));
}

• What: Resolves the master bodies of generic declarations
• Why separate: Generic master bodies need to be resolved before instantiation-specific bodies
• Function: sym_res_link_body_generic_decls_pass()

Pass 6: Link Body — Full (Parallel per file)

for(auto& file_ptr : module->direct_files) {
    futures.emplace_back(pool.push([this, &file](int id){
        auto res = link_body_task(resolver, &file);
        return res.has_errors;
    }));
}

• What: The main body resolution pass. Resolves ALL remaining symbols in function bodies:
  • Variable references and identifiers
  • Function calls
  • Expressions and operators
  • Type references in expressions
  • Access chains (a.b.c())
  • Move semantics tracking
  • Operator overload resolution
  • Pattern matching
  • Lambdas and closures
  • Comptime blocks
• Function: sym_res_link_body_pass() which creates a per-file SymResLinkBody with its own SymbolTable, GenericInstantiatorAPI, and ASTDiagnoser
• Allocator cleared: file_allocator.clear() after pass

The Sequential Path

ASTProcessor::sym_res_module_seq() is an alternative path that processes files sequentially (one file at a time, all passes together). This is used for the build.lab itself (small, not worth parallel overhead). It calls declare_and_link_file() which runs all passes in order for a single file.

Type Verification (Parallel per file)

After symbol resolution, ASTProcessor::type_verify_module_parallel() runs type verification:

for(auto& fileData : module->direct_files) {
    futures.emplace_back(pool.push([this, &fileData](int id){
        return type_verify_file_task(this, fileData.result);
    }));
}

Each file gets its own ASTDiagnoser. Results are merged after all files complete.

C Translation

After type verification, ASTProcessor::declare_module() and ASTProcessor::implement_module() handle the 2c backend translation:

// Phase 1: Forward declarations
for(auto& file_ptr : module->direct_files) {
    c_visitor.fwd_declare(unit.scope.body.nodes);
}

// Phase 2: Type aliases
for(auto& file_ptr : module->direct_files) {
    c_visitor.declare_type_aliases(unit.scope.body.nodes);
}

// Phase 3: Declare (prototypes only — no bodies)
for(auto& file_ptr : module->direct_files) {
    c_visitor.declare_before_translation(unit.scope.body.nodes);
}

// Phase 4: Implement (function bodies)
for(auto& file_ptr : module->direct_files) {
    c_visitor.translate_after_declaration(unit.scope.body.nodes);
    c_visitor.file_level_reset();  // Clear per-file state
}

Generic instantiations are translated between the module files:

c_visitor.fwd_declare(container.get_current_module_instantiations());
c_visitor.declare_before_translation(container.get_current_module_instantiations());
c_visitor.translate_after_declaration(container.get_current_module_instantiations());
container.clear_current_module_instantiations();

File Import and Caching

ASTFileResult Cache

The cache map in ASTProcessor stores parsed file results:

std::unordered_map<unsigned int, std::unique_ptr<ASTFileResult>> cache;

When a file is first imported, it's:

1. Lexed and parsed into an ASTFileResult
2. Cached by file ID for reuse
3. Import statements in the file trigger recursive imports
4. Import resolution is handled by figure_out_direct_imports()

Recursive Import Pattern

import_chemical_file_recursive() handles recursive imports:

1. Lex and parse the current file
2. Print diagnostics
3. Call figure_out_direct_imports() to find local file imports
4. Recursively import those files via thread pool
5. Uses ConcurrentParsingState to track completion (atomic task counter)

Debug Feature: File Shuffling

In DEBUG builds, files within a directory are shuffled before compilation:

#ifdef DEBUG
    shuffle_files(filePaths);
#endif

This randomly reorders files to expose compilation order bugs. A seed is printed so failures can be reproduced:

File order seed: 12345678 (set FILE_ORDER_SEED to reproduce)

Job Types in LabBuildCompiler

Compilation Job

Standard compilation — produces executable or library:

1. Parse sources
2. Symbol resolve (via ASTProcessor)
3. Type verify
4. C translation (2c)
5. Backend codegen (LLVM or TinyCC)
6. Link

Interpretation Job

When --arg-interpret is passed:

int LabBuildCompiler::do_interpretation_job(LabJob* job) {
    // 1. Parse interpret/ + common/ modules
    // 2. Symbol resolve
    // 3. Type verify
    // 4. Initialize module-level VarInitStmt on global scope
    // 5. Find main() and call it via AST interpreter
    // No codegen, no linking
}

CBI Plugin Job

Compiles a compiler plugin:

int LabBuildCompiler::link_cbi_job(LabJob* job, std::vector<LabModule*>& dependencies) {
    // 1. Parse plugin source
    // 2. Symbol resolve
    // 3. Type verify
    // 4. C translation
    // 5. TinyCC JIT compile
    // 6. Register plugin in CompilerBinder
}

Thread Safety Architecture

What's Serial

• Top-level declaration (Pass 1)
• After-link-signature (Pass 4)
• Module scope start/end
• Main function no_mangle marking

What's Parallel

• File importing (lex + parse)
• Link signatures (Pass 2)
• Generic instantiation (Pass 3)
• Generic decl body linking (Pass 5)
• Full body linking (Pass 6)
• Type verification

Synchronization Points

• print_mutex: Guards diagnostic output — each parallel task acquires it before printing
• import_mutex: Guards the file cache during recursive imports
• generic_inst_reg_mutex: Guards generic instantiation registration maps (recursive mutex)
• implsIndex.index_mutex: A std::shared_mutex — shared_lock for reads, unique_lock for writes

Allocator Strategy

file_allocator.clear();  // Called after each pass — frees temporary AST nodes
mod_allocator;            // Lifetime = module compilation
job_allocator;            // Lifetime = entire job
ast_allocator;            // Lifetime = entire compilation session

Shared, unsynchronised build state (known gap)

mod_storage, coreNodes, implsIndex, current_job, executables and the three AST allocator pointers are build-wide state shared by the top-level build, any nested build.lab/chemical.mod build and every job. Nothing isolates them, so a nested build can clear state an enclosing build still uses. This is a known structural gap (it was not the cause of the built_lab_file crash, see below).

Pooled tasks must not outlive the stack state they point at

LabBuildCompiler::pool is the only thread pool. Every function that pushes to it must join all of its futures before returning — on error paths too. A std::future from ctpl::thread_pool is a packaged task, so destroying it does not wait (only std::async blocks). Tasks capture ASTProcessor*, SymbolResolver*, the three ASTAllocators and ConcurrentParsingState& — all stack objects — so a task left running after its spawner returns writes into stack memory that deeper calls have since reused, which corrupts unrelated locals and crashes far away from the cause.

Rules (these were violated and produced the intermittent built_lab_file SIGSEGV, fixed in compiler/ASTProcessor.cpp / ASTProcessor.h):

• Return only after joining: use the JoinedTasks<T> RAII guard (ASTProcessor.cpp), which joins every valid future in its vector on scope exit, and decide error handling after join_all(futures) — never return from inside the drain loop while futures remain (stop_on_file_error used to do exactly that).
• A task group that signals completion must count the pushing thread as a participant: state.pushed_task() before pushing, state.done_task() after everything is pushed (import_chemical_files_recursive). Otherwise the counter hits zero mid-loop, the promise is fulfilled early, and wait() returns with tasks still running.
• Never pass a stack object to a task unless the task is joined before that frame dies.

The full post-mortem (crash sites, corruption signature, the gdb harness that made it reproduce at ~40 %, why &ConcurrentParsingState == &local_vector, and the misleading investigation paths) is in lang/docs/lab-build-crash-investigation.md. It is a good template for debugging stack corruption: spin-loop detection, printing suspect addresses, and why hardware watchpoints on stack slots lie.

Key Files Reference

File	Lines	Role
compiler/lab/LabBuildCompiler.cpp	~5000	Full build system: job creation, execution, link, interpretation, plugin compilation
compiler/lab/LabBuildCompiler.h	~610	Core build compiler class declaration
compiler/ASTProcessor.cpp	~1780	The pass orchestration: sym_res_module, type_verify_module, declare/implement module
compiler/ASTProcessor.h	~645	ASTProcessor class declaration
compiler/lab/mod_conv/ModToLabConverter.cpp	~390	chemical.mod → build.lab conversion
compiler/lab/mod_conv/ModToLabConverter.h	~11	ModToLabConverter header
compiler/lab/LabJob.cpp	~125	Job implementation
compiler/lab/LabBuildContext.cpp	~240	Module and resource management
compiler/lab/LabBuildContext.h	~195	LabBuildContext header
compiler/lab/LabModule.h	~210	Module structure
compiler/lab/ModuleStorage.h	~105	Module storage and lookup

The build.lab API (Lab Module)

The lab module (lang/libs/lab/src/lab.ch) exposes the build system API to build.lab scripts. Every build.lab file imports this module and calls its API to define modules, dependencies, and jobs.

Key Interfaces

BuildContext Interface

The primary interface for build scripts. Available as ctx in the build() function:

@compiler.interface
public interface BuildContext {
    // Module creation
    func new_package(&self, type : ModuleType, package_kind : PackageKind,
        scope_name : &std::string_view, name : &std::string_view,
        dependencies : std::span<ModuleDependency>) : *mut Module

    // Caching
    func get_cached(&self, job : *LabJob, path : &std::string_view) : *mut Module
    func set_cached(&self, job : *LabJob, path : &std::string_view, module : *mut Module)

    // Module management
    func add_path(&self, module : *mut Module, path : &std::string_view)
    func add_dependency(&self, job : *mut LabJob, module : *mut Module, info : *mut DependencySymbolInfo)
    func add_module(&self, job : *mut LabJob, module : *mut Module)
    func put_job_before(&self, newJob : *mut LabJob, existingJob : *mut LabJob)

    // Linking
    func link_system_lib(&self, job : *mut LabJob, name : &std::string_view, module : *mut Module = null)
    func add_lib_search_path(&self, job : *mut LabJob, path : &std::string_view, module : *mut Module = null)
    func add_object(&self, job : *LabJob, path : &std::string_view)

    // Job creation (returns new LabJob*)
    func build_exe(&self, name : &std::string_view) : *mut LabJob
    func run_jit_exe(&self, name : &std::string_view) : *mut LabJob
    func build_dynamic_lib(&self, name : &std::string_view) : *mut LabJob
    func build_interpretation(&self, name : &std::string_view) : *mut LabJob
    func build_cbi(&self, name : &std::string_view) : *mut LabJobCBI
    func translate_to_c(&self, name : &std::string_view, output_path : &std::string_view) : *mut LabJob
    func translate_to_chemical(&self, module : *mut Module, output_path : &std::string_view) : *mut LabJob

    // Remote imports
    func fetch_mod_dependency(&self, job : *mut LabJob, mod : *mut Module,
        dep : &ImportRepo, strategy : ConflictResolutionStrategy = ...) : bool
    func fetch_job_dependency(&self, job : *mut LabJob,
        dep : &ImportRepo, strategy : ConflictResolutionStrategy = ...) : bool

    // CLI argument handling
    func has_arg(&self, name : &std::string_view) : bool
    func get_arg(&self, name : &std::string_view) : std::string_view
    func define(&self, job : *LabJob, name : &std::string_view) : bool
    func undefine(&self, job : *LabJob, name : &std::string_view) : bool

    // Testing
    func set_environment_testing(&self, job : *mut LabJob, value : bool)

    // Compiler interfaces (for CBI plugins)
    func add_compiler_interface(&self, module : *mut Module, interface : &std::string_view) : bool
    func contains_cbi(&self, key : &std::string_view) : bool
    func index_cbi_fn(&self, job : *mut LabJobCBI, key : &std::string_view,
        fn_name : &std::string_view, fn_type : CBIFunctionType) : bool

    // Tool invocation
    func build_path(&self) : std::string_view
    func invoke_ar(&self, string_arr : std::span<std::string_view>) : int
    func invoke_ranlib(&self, string_arr : std::span<std::string_view>) : int
}

LabJob Interface

Represents a compilation job:

public interface LabJob {
    func getType(&self) : LabJobType
    func getName(&self) : std::string_view
    func getAbsPath(&self) : std::string_view
    func getBuildDir(&self) : std::string_view
    func getStatus(&self) : LabJobStatus
    func getTargetTriple(&self) : std::string_view
    func getMode(&self) : OutputMode
    func getTarget(&self) : &TargetData
    func setAbsPath(&self, path : std::string_view)
}

Module Interface

Represents a compiled module:

public interface Module {
    func getType(&self) : ModuleType
    func getScopeName(&self) : std::string_view
    func getName(&self) : std::string_view
    func getBitcodePath(&self) : std::string_view
    func setBitcodePath(&self, path : &std::string_view)
    func getObjectPath(&self) : std::string_view
    func setObjectPath(&self, path : &std::string_view)
    // ... etc
}

Key Enums

public enum ModuleType { File, CFile, CPPFile, ObjFile, Directory }
public enum PackageKind { Library, Application }
public enum LabJobType {
    Executable, JITExecutable, Library, ToCTranslation,
    ToChemicalTranslation, ProcessingOnly, CBI, Interpretation
}
public enum LabJobStatus { Pending, Launched, Success, Failure }
public enum OutputMode : int {
    Debug, DebugQuick, DebugComplete,
    ReleaseFast, ReleaseSmall, ReleaseSafe
}

Common Patterns in build.lab

Pattern 1: Standard Executable

import lab

func build(ctx : *mut AppBuildContext) : *mut LabJob {
    const exe_job = ctx.build_exe("my_app")
    const deps = [/* dependency modules */]
    const mod = ctx.directory_app_module("", "main", "src", deps)
    ctx.add_module(exe_job, mod)
    return exe_job
}

Pattern 2: Remote Dependency

ctx.fetch_mod_dependency(job, mod, ImportRepo{
    from : "github.com/owner/repo",
    version : "v1.0",
    location : intrinsics::get_raw_location()
})

Pattern 3: CBI Plugin

const cbi_name = std::string_view("my_plugin")
const cbi_job = ctx.build_cbi(&cbi_name)
ctx.add_module(cbi_job, my_plugin_mod)

Pattern 4: Conditional Build

if(ctx.has_arg("interpret")) {
    const interp_job = ctx.build_interpretation("my-interp")
    // ... add modules
    return interp_job
}

Pattern 5: Build Path Helpers (in lab namespace)

// Path to the current build.lab's directory
lab::curr_dir()

// Absolute path relative to current build.lab
lab::rel_path_to("src")

// Build directory paths
ctx.build_job_dir_path("job_name")
ctx.build_mod_file_path("job", "scope", "mod", "file")
ctx.build_llvm_ir_path("job", "scope", "mod")

// Common helpers
const mod = ctx.directory_app_module("", "main", lab::rel_path_to("src"), deps)

How Tests Are Wired (lang/tests/build.lab Example)

See the Testing Guide for how lang/tests/build.lab wires:

• Interpretation tests (--arg-interpret)
• Compiled tests (default)
• Library tests (--arg-test-plugins)
• Individual library tests (--arg-test-html, --arg-test-css, etc.)

The test wiring uses ctx.build_interpretation(), ctx.build_exe(), set_environment_testing(), has_arg(), and define() to configure the build environment.

CLI Arguments — How They Connect to the Build Pipeline

Entry Point: compiler_main()

File: core/main/CompilerMain.cpp

The CLI entry point is the single compiler_main(int argc, char* argv[]) function declared in core/main/CompilerMain.h. It parses all command-line options and routes them to the appropriate subsystem.

Option Parsing Infrastructure

File: utils/CmdUtils.h, utils/CmdUtils.cpp

struct CmdOption {
    std::string_view large_opt;      // Long name (e.g., "verbose")
    std::string_view small_opt;      // Short alias (e.g., "v")
    CmdOptionType type;              // NoValue, SingleValue, MultiValued, SubCommand
};

class CmdOptions {
    std::unordered_map<std::string_view, CmdOption&> data;  // Registered options
    std::vector<std::string_view> arguments;                // Positional arguments
    tsl::ordered_map<std::string_view, std::string_view> options;  // Parsed values
    
    void register_options(CmdOption options_data[], unsigned size);
    void parse_cmd_options(int argc, char* argv[], int skip = 0);
    
    // Access parsed values:
    bool has_value(const std::string_view& opt, const std::string_view& small_opt = "");
    std::optional<std::string_view>& option_new(const std::string_view& opt, const std::string_view& small_opt = "");
    CmdOption& cmd_opt(const std::string_view& opt);  // For subcommands / multi-values
};

Full Option Reference

Options are registered in CompilerMain.cpp as a CmdOption[] array:

CLI Flag	Alias	Type	Effect
--include	—	MultiValued	Add include path or file (.ch = source, else = header)
--build-dir	—	SingleValue	Set build output directory
--library	-l	MultiValued	Link a system library
configure	—	SubCommand	Run OS configuration (sets CHEMICAL_HOME, PATH)
--cc	—	SubCommand	Forward remaining args to Clang
--run	—	SubCommand	Run a build.lab or chemical.mod file (chemical run file.mod)
--linker	—	SubCommand	Forward to system linker
--ar	—	SubCommand	Forward to LLVM ar
--dlltool	—	SubCommand	Forward to LLVM dlltool
--ranlib	—	SubCommand	Forward to LLVM ranlib
--lib	—	SubCommand	Forward to LLVM lib
--tcc-jit	—	SubCommand	Forward to TinyCC JIT runner
--mode	-m	SingleValue	Output mode: debug, debug_quick, debug_complete, release, release_fast, release_small
--plugin-mode	-pm	SingleValue	Plugin compilation mode (same values as --mode)
--version	—	NoValue	Print version and exit
--help	—	NoValue	Print help message
--verbose	-v	NoValue	Verbose output
--verbose-link	-vl	NoValue	Verbose linker output
--output	-o	SingleValue	Output file path
--out-ll <path>	—	SingleValue	Emit LLVM IR to path
--out-bc <path>	—	SingleValue	Emit LLVM bitcode to path
--out-obj <path>	—	SingleValue	Emit object file to path
--out-asm <path>	—	SingleValue	Emit assembly to path
--out-bin <path>	—	SingleValue	Emit binary executable to path
--out-ll-all	—	NoValue	Emit LLVM IR for ALL modules
--out-asm-all	—	NoValue	Emit assembly for ALL modules
--emit-c	--emit-c	NoValue	Keep generated C files
--minify-c	--minify-c	NoValue	Minify generated C output
--incremental	--incremental	NoValue	Per-file C translation (not merged)
--keepc	--keepc	NoValue	Keep C files after compilation
--ignore-extension	—	NoValue	Output file extension is NOT used to determine output type
--jobs	-j	SingleValue	Thread count for parallel compilation
--job-type	-jt	SingleValue	Force job type: exe, jit-exe, lib, 2c, 2ch, inter, proc
--target	-t	SingleValue	Target triple (default: host triple)
--lto	—	NoValue	Enable Link Time Optimization
--assertions	—	NoValue	Enable IR assertions for codegen validation
--debug-ir	—	NoValue	Don't crash on potentially bad IR
--benchmark	-bm	NoValue	Benchmark lexing/parsing/compilation
--benchmark-files	-bm-files	NoValue	Per-file benchmark breakdown
--benchmark-modules	-bm-modules	NoValue	Per-module benchmark breakdown
--test	--test	NoValue	Set is_testing_env flag
--no-cache	—	NoValue	Disable module caching
--frecompile-plugins	--frecompile-plugins	NoValue	Force recompilation of all CBI plugins
--sanitize	-fsanitize	SingleValue	Enable sanitizers: address, memory, thread, undefined, leak, hwaddress, dataflow (comma-separated)
--tsan	—	NoValue	Enable ThreadSanitizer
--no-pie	--no-pie	NoValue	Disable position-independent executable
--jit	--jit	NoValue	JIT compilation mode (TinyCC)
--use-tcc	--use-tcc	NoValue	Use TinyCC backend (C translation + libtcc)
--use-c	--use-c	NoValue	Translate to C + compile with embedded Clang
--use-lld	--use-lld	NoValue	Use LLD for linking
--use-bc	--use-bitcode	NoValue	Use bitcode format for module objects
--resources	--res	SingleValue	Override resources directory path
--mod	—	MultiValued	Specify module files
--cbi-m	--cbi-m	MultiValued	CBI module names
--download	--download	NoValue	Download-only (don't build)
--check	--check	NoValue	Check-only (don't build)
--run-negative-tests	—	NoValue	Run negative lifetime tests (DEBUG-only)
--ignore-errors	--ignore-errors	NoValue	Continue despite errors
-g	—	NoValue	Emit debug info
-c	—	NoValue	Compile to object files only (no link)
--fno-unwind-tables	—	NoValue	Disable unwind tables (cleaner IR)
--fno-asynchronous-unwind-tables	—	NoValue	Disable async unwind tables
--arg-<name>	-arg-<name>	String	Pass arbitrary args to build.lab scripts (e.g., --arg-interpret)
--res <dir>	-res <dir>	SingleValue	Resources directory

How CLI Args Flow Through the Pipeline

CLI (argc, argv)
    │
    ▼
CmdOptions::parse_cmd_options()
    │
    ├── SubCommand Detected?
    │   ├── "configure" → configure_exe()
    │   ├── "cc" → chemical_clang_main2()
    │   ├── "ar"/"ranlib"/"lib"/"dlltool" → llvm_ar_main2()
    │   ├── "run" → LabBuildCompiler::run_invocation()
    │   ├── "tcc-jit" → LabBuildCompiler::tcc_run_invocation()
    │   └── "linker" → forward to system linker
    │
    ├── No Input?
    │   └── print_usage(), exit 1
    │
    ├── Input is .lab or .mod file?
    │   ├── .mod → build via LabBuildCompiler::build_mod_file()
    │   ├── .lab → build via LabBuildCompiler::build_lab_file()
    │   └── .mod → .lab → convert via ModToLabConverter
    │
    └── Input is .ch file(s)?
        └── Single-file compilation path:
            1. Create LabModule, LabJob, LabBuildCompilerOptions
            2. prepare_options() fills options from parsed CLI
            3. Handle output file extension → set module type:
               .o → obj output, .s → asm, .ll → llvm ir, .bc → bitcode
            4. compiler.do_job_allocating(&job)

The prepare_options() Lambda

In CompilerMain.cpp, the prepare_options lambda maps CLI flags to LabBuildCompilerOptions:

auto prepare_options = [&](LabBuildCompilerOptions* opts) -> void {
    opts->benchmark = options.has_value("benchmark", "bm");
    opts->benchmark_files = options.has_value("benchmark-files", "bm-files");
    opts->benchmark_modules = options.has_value("benchmark-modules", "bm-modules");
    opts->verbose = verbose;
    opts->verbose_link = options.has_value("verbose-link", "vl");
    opts->minify_c = options.has_value("minify-c");
    opts->emit_c = options.has_value("emit-c", "emit-c") || options.has_value("keepc", "keep-c");
    opts->debug_info = options.has_value("", "g") || (opts->out_mode == OutputMode::Debug || opts->out_mode == OutputMode::DebugComplete);
    if (options.has_value("use-tcc", "use-tcc")) { opts->use_tcc = true; opts->use_c = true; }
    else if (options.has_value("use-c", "use-c")) { opts->use_c = true; }
    opts->use_lld = options.has_value("use-lld", "use-lld");
    opts->is_testing_env = options.has_value("test");
    opts->ignore_errors = options.has_value("ignore-errors", "ignore-errors");
    if(options.has_value("no-cache")) { opts->is_caching_enabled = false; }
    if(options.has_value("frecompile-plugins")) { opts->force_recompile_plugins = true; }
    if(options.has_value("debug-ir")) { opts->debug_ir = true; }
    if(options.has_value("lto")) { opts->def_lto_on = true; }
    if(options.has_value("assertions")) { opts->def_assertions_on = true; }
    opts->fno_unwind_tables = options.has_value("", "fno-unwind-tables");
    opts->no_pie = options.has_value("no-pie", "no-pie");
    auto mode_opt = options.option_new("plugin-mode", "pm");
    if(mode_opt.has_value()) { opts->def_plugin_mode = get_output_mode(mode_opt, false); }

    // Sanitizers — parsed inline from --sanitize=addr,undef format
};

LabBuildCompilerOptions Structure

File: compiler/lab/LabBuildCompilerOptions.h

It inherits the processing flags (benchmark, benchmark_files, benchmark_modules, verbose, verbose_link, ignore_errors, debug_info, emit_c, stop_on_file_error, is64Bit, target_triple, resources_path, exe_path) from ASTProcessorOptions.

class LabBuildCompilerOptions : public ASTProcessorOptions {
public:
    std::string build_dir;                 // Build output directory
    LabJobAttributes default_job_attrs;    // download_only / check_only

#ifdef COMPILER_BUILD
    bool use_c = false;                    // Translate to C (+ embedded Clang on the LLVM build)
#else
    bool use_c = true;
#endif
#ifdef COMPILER_BUILD
    bool use_tcc = false;                  // Force TinyCC (implies use_c)
#else
    bool use_tcc = true;
#endif
    bool is_caching_enabled = true;
    bool is_build_lab_caching_enabled = true;
    bool force_recompile_plugins = false;
    bool translate_to_single_file = true;
    bool use_mod_obj_format = true;        // .o vs .bc format
    OutputMode out_mode = OutputMode::Debug;
    OutputMode def_out_mode = OutputMode::Debug;
    OutputMode def_plugin_mode = OutputMode::Debug;   // ReleaseFast in non-DEBUG builds
    bool minify_c = false;
    bool debug_ir = false;
    bool def_lto_on = false;
    bool def_assertions_on = false;
#ifdef COMPILER_BUILD
    bool no_pie = false;
    SanitizerType sanitizers = SanitizerType::None;
    bool use_lld = false;
    bool out_ll_all = false;
    bool out_asm_all = false;
    bool fno_unwind_tables = false;
    bool fno_asynchronous_unwind_tables = false;
#endif
    bool is_testing_env = false;

    LabBuildCompilerOptions(std::string exe_path, std::string target_triple,
        std::string build_dir, bool is64Bit);
};

Build Args Passthrough to build.lab

When running a .lab file, CLI --arg-* flags are forwarded to LabBuildContext.build_args:

for(auto& opt : options.options) {
    if(opt.first.starts_with("arg-")) {
        context.build_args[opt.first.data() + 4] = opt.second;
    }
}

The build.lab script then checks these via ctx.has_arg() and ctx.get_arg().

Output Extension Routing

The compiler determines what type of compilation to do based on the output file extension:

Extension	Behavior
.exe (or --out-bin)	Link into an executable
.o	Compile to object file only (set as module.object_path)
.ll	Compile to LLVM IR only (set as module.llvm_ir_path)
.bc	Compile to LLVM bitcode only (set as module.bitcode_path)
.s	Compile to assembly only (set as module.asm_path)
.c	Translate to C (set job.type = ToCTranslation)
.ch	Translate to Chemical (set job.type = ToChemicalTranslation)
-c flag	Processing only (compile to object, no link)
None / default	a.exe (Win) or a (Unix) — compile and link

Entry Point Decision Flow (Simplified)

compiler_main(argc, argv)
    │
    ├── if "-cc1" → chemical_clang_main()
    ├── if configure → configure_exe()
    ├── if version → print version
    ├── if help → print_help()
    ├── if run → LabBuildCompiler::run_invocation()
    ├── if cc → chemical_clang_main2()
    ├── if ar/dlltool/ranlib/lib → llvm_ar_main2()
    ├── if tcc-jit → LabBuildCompiler::tcc_run_invocation()
    │
    └── else (normal compilation)
        ├── Parse options → LabBuildCompilerOptions
        ├── Input = .mod file?
        │   ├── → output.c? → translate to C (job type = ToCTranslation)
        │   └── → default → build_mod_file()
        ├── Input = .lab file?
        │   └── → build_lab_file()
        └── Input = .ch/.c/.o files?
            └── → Single-file compilation via do_job_allocating()

Related Skills

• Symbol Resolution (.agents/skills/symres/SKILL.md) — Detailed symres pipeline
• Generic Instantiation (.agents/skills/generics/SKILL.md) — Generic monomorphization
• C Codegen (.agents/skills/c_codegen/SKILL.md) — 2c translation backend
• Performance (.agents/skills/performance/SKILL.md) — Optimization and parallelization patterns
• Compiler Bindings (.agents/skills/compiler_bindings/SKILL.md) — CBI and TinyCC integration
• Testing Guide (.agents/skills/testing/SKILL.md) — How tests are wired and executed