nexus
/
nip
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
nip/docs/vision.md

21 KiB
Raw Blame History

NexusOS Vision

🚀 Abstract & Vision

NexusOS is not an experiment. It is the operating system of Libertaria’s future: Military-grade security and modularity. Fast boot. No bloat. Hardened kernel. Zero GNUs given. Healthcare integrity. Deterministic builds. Verifiable artifacts. Runtime predictability. Government architecture. Layered policy via .npk. Immutable roots. Declarative runtime control. Philosophical payload. A system designed to mean something.

NexusOS is engineered to fuse the expressive power and efficiency of the Nim programming language with the atomic reliability of AerynOS, the profound reproducibility of Nix/Guix, and the organizational clarity of GoboLinux’s filesystem hierarchy. Designed for pragmatic simplicity, radical flexibility, and uncompromising developer control, NexusOS leverages existing software ecosystems to rapidly deliver a potent, highly programmable, and next-generation OS and package management platform.

🎯 The Problem & The NexusOS Solution

Traditional package and OS management systems often force a compromise between simplicity, reproducibility, and programmability. NexusOS aims to transcend these limitations.

Feature Domain Apt/DNF (Traditional) NixOS/Guix (Declarative) Gentoo (Source-Based) AerynOS (Modern Atomic) NexusOS (Hybrid Declarative)
Simplicity (End-User UX) ✅ 〰️ (Steep Initial Curve) ❌ ✅ ✅ (Tiered UX)
Reproducibility ❌ ✅ ✅ ✅ ✅
Atomic Updates ❌ ✅ ❌ ✅ ✅
Powerful Programmability ❌ ✅ (Specialized DSL/LISP) ✅ (Full Nim DSL) 〰️ (YAML + Build Scripts) ✅ (Full Nim DSL)
Flexible Version Mgmt ❌ ✅ ✅ 〰️ ✅
Typed, Modern Core Codebase ❌ 〰️ (Nix DSL/C++/Perl) ❌ (Shell/Python/C) ✅ (Rust) ✅ (Nim)

NexusOS Solution: A unified paradigm delivering all desired traits through a Nim-first, layered architecture.

🧩 Core Innovations & Pillars

1. Nim-Powered Core & Hybrid DSL

  • A robust, statically-typed foundation built in Nim, ensuring performance and reliability.
  • Hybrid Recipe Approach:
    • An elegant, high-level Nim-based Domain Specific Language (DSL) leveraging metaprogramming for concise, powerful, and type-safe package definitions and system configurations ("Mastery Mode").
    • A simplified, declarative recipe format (e.g., YAML-like or a restricted Nim syntax, inspired by AerynOS "stones") for common use cases and as a target for imported recipes. This "Easy Mode" format translates internally to the full Nim DSL, ensuring clarity and human-readability for basic package maintenance.
  • Direct NimScripting Capability: Alongside the DSL, NexusOS provides the ability to use NimScript directly for highly sophisticated or bespoke package and system building tasks. This offers maximum flexibility and an immediate power ramp, especially during early development phases.

2. Universal Recipe Ingestion & Translation

  • Strategic ingestion and translation of package definitions from established ecosystems, primarily Nixpkgs (treating .nix as a declarative "Kalkül") and Arch PKGBUILDs.
  • Ensures immediate, vast software availability, bootstrapping a rich ecosystem from day one.

3. GoboLinux-Inspired Filesystem Model

  • Intuitive, versioned application directories (e.g., /Programs/AppName/Version), enhancing clarity and simplifying manual inspection.
  • Effortless activation and rollback of application versions via managed Current symlinks.

4. Atomic Transactions, System Generations & Bootloader Integration

  • Default transactional integrity for all system modifications, creating immutable "generations" (inspired by NixOS/Guix/AerynOS).
  • Guarantees system consistency: operations complete fully or not at all.
  • Bootloader Integration: Native support for Limine to list and boot into previous system generations, enabling seamless A/B style rollbacks at boot time for maximum system recovery capability.
  • Pragmatic overrides (e.g., --allow-partial-update) for expert-driven, targeted operations.

5. Sophisticated Version, Channel & Retention Management

  • Native support for distinct software channels (stable, LTS, testing, git-dev).
  • User-configurable, fine-grained retention policies (defined in the Nim DSL) for managing package versions and disk space intelligently.

6. Programmable, Guix-Inspired Build System with Type-Safe "Macros"

  • Moving beyond shell-script dominance, build logic is expressed directly in typed Nim.

  • Define reusable "build systems" (for Autotools, CMake, Meson, Cargo etc.) as Nim modules.

  • Type-Safe Build Templates (Nim Metaprogramming): Instead of simple text-based macros (like AerynOS's %make), NexusOS will utilize Nim's templates and macros to create structured, type-safe, and highly maintainable build abstractions. This captures the simplicity and standardization benefits while adding compile-time checking and greater extensibility.

    • Example (Conceptual Nim DSL): buildPackage("zlib", "1.3.1"): source("https://zlib.net/zlib-1.3.1.tar.gz", sha256 = "...") useCMake(opts = ["-DBUILD_TESTING=OFF"]) // Type-safe template

  • Offers deep customization and control akin to Guix, but with Nim's paradigms.

🛠️ Core Engine Architecture: Leveraging Nim's Advanced Capabilities

The choice of Nim as the foundational language for NexusOS is a deliberate, strategic decision rooted in its unique and powerful capabilities. The nexus and nip tools are not simple scripts; they are sophisticated, high-performance pieces of system infrastructure. Their architecture is designed to leverage Nim's most advanced features to deliver deterministic performance and massive concurrency.

1. Memory Management: Determinism via ARC/ORC

This choice is about performance predictability and suitability for systems programming.

  • Garbage Collection (GC): The default, traditional GC prioritizes developer convenience. While robust, it can introduce non-deterministic "stop-the-world" pauses to clean up memory. For a simple command-line tool that runs and exits, this is often acceptable. For a long-running daemon or a core system utility where responsiveness is key, these pauses are a liability.
  • ARC/ORC (Automatic Reference Counting / Ownership): This is our choice for NexusOS. It is about deterministic performance. Memory is freed the instant it is no longer referenced, similar to the RAII model in C++ or Rust's ownership system, but without the manual memory management or the complexity of a borrow checker. This eliminates unpredictable pauses, resulting in smoother, more consistent performance. The trade-off—a tiny, constant overhead for updating reference counts and the need to manually break rare reference cycles—is a worthwhile price for the robustness and predictability required of core system infrastructure.

Verdict for NexusOS: We will build with ARC/ORC from the start (--gc:orc). A package manager and system orchestrator must be as predictable and real-time as possible. The deterministic nature of ARC/ORC is a perfect match for the low-level, high-performance tasks we will implement, such as file hashing, process management, and data deduplication.

2. Concurrency: A Hybrid Model for Maximum Throughput

This is not an "either/or" question. The correct architecture uses both of Nim's concurrency models strategically for different tasks.

  • async is for I/O-bound work.
    • Use Case: Fetching dozens of source tarballs from the network, downloading pre-built .npk binaries, checking hashes of large files on disk.
    • Why: These tasks spend most of their time waiting for the network or disk. async allows a single OS thread to efficiently juggle hundreds of these waiting operations without the heavy overhead of creating a dedicated thread for each one. It's about maximizing throughput when the CPU is not the bottleneck.
  • spawn is for CPU-bound work.
    • Use Case: Compiling source code (e.g., running GCC on a large C++ project), compressing final .npk packages.
    • Why: These tasks will max out a CPU core. Running them in an async context would block the entire event loop, freezing all other I/O operations. spawn creates a true OS-level thread that the kernel can schedule on a separate CPU core. This allows for true parallelism, enabling us to build multiple packages simultaneously and fully utilize a multi-core processor (the equivalent of make -j8).

3. The Hybrid Architecture in Practice

The nexus build orchestrator will be architected as follows:

  1. The Orchestrator (Main Thread using async): The main nexus process runs an async event loop. Its job is to manage the overall workflow. It will parse the dependency graph and use a pool of async workers to fetch all sources and check all hashes concurrently. This phase is all about I/O saturation.
  2. The Build Farm (Thread Pool using spawn): Once the sources for a package are ready, the async orchestrator will not build it directly. Instead, it will submit a "build job" to a thread pool. This pool will consist of a fixed number of worker threads (e.g., matching the number of CPU cores). Each worker thread will pull a job from the queue, spawn the compilation process, and wait for it to finish. This phase is all about CPU saturation.
  3. Deployment (Back to async): Once a build is complete, the worker thread notifies the main orchestrator. The orchestrator can then handle the final deployment—atomically moving the build output to the final GoboLinux-style path—using async file I/O.

This hybrid model gives us the best of all worlds: maximum I/O throughput for fetching and maximum CPU parallelism for building, all orchestrated cleanly by a single, responsive main process. Nim is exceptionally well-suited for building exactly this kind of sophisticated, multi-paradigm system.

🛠️ Core Engine Architecture: The Nim & Janet Duality

NexusOS is built on a principle of architectural integrity: using the right tool for the right job. While the core engine, build orchestrator, and the primary NimPak EDSL are forged in statically-typed Nim for maximum performance and compile-time correctness, we explicitly recognize that not all system logic is static or predictable. For this, we embed Janet—a modern, lightweight, and powerful Lisp—as a first-class citizen within our toolchain from the very beginning.

Janet serves as the dynamic superglue of the system, the designated successor to fragile and opaque Bash scripts for handling runtime logic. Its role is to manage tasks where the inputs are "fuzzy," unclear, or only available at execution time. This includes fetching dynamic configuration values from a web service, interfacing with non-standard APIs, or executing conditional deployment logic based on live system state that cannot be known at compile time. Unlike shelling out to Bash, the embedded Janet interpreter operates within a controlled context, allowing for structured data exchange (not just text streams) with the core Nim engine and offering a safer, more robust, and debuggable environment for dynamic tasks.

Nim provides the immutable, verifiable skeleton of the system, while Janet provides the adaptive, intelligent nervous system that allows it to react dynamically. This hybrid approach is not a compromise; it is a deliberate design for creating a system that is both resilient and responsive from day one.

📦 Additional NexusOS Capabilities & Strategic Features

The following concepts, inspired by the best aspects of Guix and the potential for Nim-centric packaging (conceptualized as "NimPak"), will significantly enhance NexusOS’s practicality, usability, and user autonomy, offering paths for broader adoption.

1. Per-User Package Management (Guix-inspired)

By default, NexusOS empowers every user on the same machine to independently manage their own software environment without the need for superuser privileges:

  • User-Level Autonomy: Users can safely install, upgrade, and manage packages in their own namespace, entirely isolated from the global system and other users.
  • No Root Required for User Packages: Eliminates security risks and administrative friction associated with global privilege escalations for routine user-specific package operations.
  • Ideal for Multi-User Environments: Facilitates shared server scenarios, research clusters, and academic environments where user software autonomy and system stability are paramount.

2. Robust Reproducible Environments (Guix-inspired)

NexusOS guarantees reliable reproducibility of computing environments, critical for software developers, researchers, and system administrators alike:

  • Environment Snapshots: Easily build and capture complete software environments that include applications, libraries, runtime configurations, and their precise dependencies.
  • Reproduce in Time: Restore an exact snapshot of your software environment at any future point, ensuring the same outcomes years from now, regardless of intervening system changes.
  • Containers & Virtualization: Seamlessly create and deploy containerized environments (e.g., OCI images) or VM images based on reproducible NexusOS definitions, significantly enhancing CI/CD pipelines and reproducible research.
  • Example Usage:
    • Build an environment today.
    • Reproduce precisely the same environment years from now.
    • Guarantee consistent results regardless of system updates or underlying hardware changes.

3. NimPak — Unified Nim-Centric Package Management (Portable Core)

"NimPak" embodies NexusOS’s powerful Nim-native package management vision, designed as a robust, unified API ecosystem within the Nim language, with potential for portability:

  • Single Language, Infinite Possibilities: Leverage NimPak’s APIs, including high-level embedded domain-specific languages (EDSLs), to define and manage packages, dependencies, and even whole-system configurations with exceptional clarity and type safety.
  • Potential Integration with Existing Distros: The core NimPak tooling could be designed to integrate atop any UNIX/Linux distribution without interference or conflicts with the host distribution’s native package manager (APT, DNF, Pacman, etc.).
  • Cross-Distribution Utility: This portability simplifies adoption by allowing users and developers to experience NimPak's benefits (e.g., for managing development projects or specific application stacks) on existing systems, rather than requiring an immediate wholesale migration to NexusOS.
  • Example Conceptual Usage (NimPak API):

    Conceptual NimPak definition

    definePackage("myapp", "1.0.0"): source(git = "https://github.com/example/myapp.git", tag = "v1.0.0") dependencies: useCMake(["-DBUILD_EXAMPLES=ON"]) # A type-safe build system template

    This definition could potentially be built and managed by NimPak tools

    on Ubuntu, Fedora, Arch, or NexusOS.

🚩 Integration & Advantages of These Additional Features for NexusOS

By incorporating these powerful ideas, NexusOS uniquely combines:

  • User-level autonomy and isolated environments (Guix-inspired).
  • Deep reproducibility and environmental consistency (Guix-inspired).
  • Elegant Nim-native interfaces and a unified tooling vision (NimPak concept).
  • A potential path for seamless compatibility and integration of its core package management technology across existing Linux/UNIX distributions.

This holistic approach enables a gentle, incremental transition path toward full NexusOS adoption for users, encourages immediate experimentation and use of its package management features on current systems, and offers clear value propositions for both end-users and system administrators from the outset.

🎛️ Layered User Experience

NexusOS caters to diverse expertise through a tiered interface:

User Level Target Audience Primary Interaction & Capability Focus
Easy Mode End-Users, Basic Maintainers Simple CLI (ni install nginx, nu upgrade-system). Focus on pre-built binaries, atomic updates, effortless rollbacks. Recipe creation via simplified declarative format (e.g., YAML-like).
Medium Mode SysAdmins, Power Users Extended CLI, source builds from recipes, system configuration via declarative Nim files, channel/version policy management. Modification of simplified recipes. User-level package management.
Mastery Mode DevOps, Architects, Devs Full Nim DSL programmability for packages & build systems (plus direct NimScripting), type-safe build templates, reproducible environment definitions, complex cross-compilation, OS variant creation, fleet management tooling.

🚀 Strategic Direction

We are not building "just another distro."

We are designing:

  • A verifiable OS
  • A programmable system
  • A modular foundation
  • A reproducibility-driven ecosystem
  • A licensing and packaging architecture fit for open infrastructure and commercial clarity

Development Philosophy

  • Pragmatic over purist: adopt practical solutions that work
  • Foundation first: establish solid groundwork before advancing
  • Incremental progress: build, test, document, commit regularly
  • Security by design: integrate security considerations from the start

Core Technical Stack

  • Language: Nim (primary), Janet (shell scripting replacement for Bash)
  • Package Manager: nip CLI tool with .npk format
  • Architecture: GoboLinux-style /Programs/App/Version/ hierarchy
  • Metadata: KDL format for configuration and package metadata
  • Hashing: BLAKE3 for all integrity verification
  • License: ACUL compliance enforcement required

Code Quality Standards

  • Use Nim's type system extensively, avoid any types
  • Implement Result[T, E] pattern for error handling
  • Document all public procedures with comments
  • Follow modular structure: separate concerns into focused modules
  • Use ARC/ORC memory management (--gc:orc)
  • Implement comprehensive logging for debugging and audit

Package Management Principles

  • All operations must be atomic and rollback-capable
  • Support grafting from existing package managers (Pacman, Nix)
  • Maintain full provenance tracking from source to installation
  • Generate ACUL-compliant metadata for all packages
  • Implement cryptographic verification via nip verify

Development Workflow Reminders

  • Always implement documentation alongside code
  • Commit changes regularly with descriptive messages
  • Set up proper project structure before diving deep
  • Test each component before moving to next feature
  • Consider security implications in every design decision

System Architecture Goals

  • Immutable root filesystem with layered policy control
  • Deterministic builds with reproducible artifacts
  • Runtime predictability and fast boot times
  • User-space package installation without root filesystem modification
  • Hardware-backed security where available

AI Assistant Behavior

  • Provide actionable, time-conscious guidance
  • Focus on practical implementation over theoretical discussion
  • Remind about project management tasks (documentation, commits, structure)
  • Teach Nim programming through hands-on OS development
  • Maintain awareness of next steps and their dependencies

🔐 Licensing

NexusOS operates under a three-tier licensing model to balance openness, commercial viability, and community freedom:

  1. Core Codebase (e.g., nexus CLI, nip package manager, NimPak DSL, Fragments):

    • License: Dual-licensed under EUPL-1.2 (open-source, copyleft) or ACUL (commercial, closed-source with conditions).
    • Details:
      • EUPL-1.2: Requires source code sharing for derivative works.
      • ACUL: Allows closed-source use with attribution, reproducibility, and Nexus Foundation membership.
      • Applies to core tools, internal Fragments, and native NexusForge image generator.

  2. Community Contributions (NexusRecipes, NexusArtifacts, community docs):

    • License: CC0 (public domain).
    • Details: NexusRecipes (user-contributed recipes) and NexusArtifacts (signed binaries) in community/ are freely usable. Community documentation is also CC0.
    • Hosting:
      • NexusArtifacts (signed .npk, VM images) are downloaded from NexusHub.
      • NexusRecipes (.nexusrecipe.yaml, .nexusrecipe.nim) are sourced from NexusCommons.
    • External Sources: nip can ingest Nix Flakes and AUR PKGBUILDs with --i-know-what-i-do, translating to NexusRecipes under CC0.

  3. Documentation:

    • Core docs (e.g., vision, architecture): EUPL-1.2.
    • Community docs (in community/docs/): CC0.

Nexus Foundation Membership:

  • Commercial users of the core codebase under ACUL require Nexus Foundation membership. See Nexus Foundation.
  • NexusArtifacts and NexusRecipes under CC0 can be used without membership.

Contribution Guidelines:

  • Contributions to community/recipes/ or community/artifacts/ are CC0-licensed.
  • Core codebase contributions (e.g., src/) require a Contributor License Agreement (CLA) under EUPL-1.2/ACUL.

🗺️ Roadmap

For a detailed development plan, please refer to the roadmap.md document.

🏷️ Naming Scheme

For a comprehensive overview of the NexusOS naming conventions, including CLI tools, package formats, and core concepts, please refer to the nimbleos_overview.md document.