18 KiB

Raw Blame History

Nippels Developer Guide

Complete guide for developers working with Nippels (NimPak Cells)

Architecture Overview
Core Components
API Reference
Extension Points
Development Setup
Testing
Contributing

Architecture Overview

High-Level Architecture

┌─────────────────────────────────────────────────────────┐
│                    Nippel Manager                       │
│  (nippels.nim - Orchestration & Public API)            │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │   Profile    │  │  Namespace   │  │     XDG      │ │
│  │   Manager    │  │  Subsystem   │  │   Enforcer   │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
│                                                         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │     CAS      │  │    Merkle    │  │     UTCP     │ │
│  │   Backend    │  │     Tree     │  │   Protocol   │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
│                                                         │
│  ┌──────────────┐  ┌──────────────┐                   │
│  │   Nexter     │  │ Decentralized│                   │
│  │     Comm     │  │  Architecture│                   │
│  └──────────────┘  └──────────────┘                   │
└─────────────────────────────────────────────────────────┘

Module Dependencies

nippels.nim (main orchestrator)
├── nippel_types.nim (shared types)
├── profile_manager.nim (security profiles)
├── namespace_subsystem.nim (Linux namespaces)
├── xdg_enforcer.nim (XDG directory enforcement)
├── cas_backend.nim (content-addressable storage)
├── merkle_tree.nim (integrity verification)
├── utcp_protocol.nim (AI-addressability)
├── nexter_comm.nim (Nexter communication)
└── decentralized.nim (P2P features)

Core Components

1. Nippel Manager (nippels.nim)

Purpose: Main orchestration and public API

Key Types:

type
  NippelManager* = ref object
    cells*: Table[string, Nippel]
    profileManager*: ProfileManager
    activeCells*: HashSet[string]
    merkleTrees*: Table[string, MerkleTree]

  Nippel* = object
    name*: string
    profile*: SecurityProfile
    settings*: ProfileSettings
    rootPath*: string
    state*: NippelState
    namespaceHandle*: Option[NamespaceHandle]
    merkleRoot*: string
    utcpAddress*: UTCPAddress

Key Functions:

proc createNippel*(manager: NippelManager, name: string,
                   profile: SecurityProfile): Result[Nippel, NippelError]

proc activateNippel*(manager: NippelManager, name: string): Result[void, NippelError]

proc deactivateNippel*(manager: NippelManager, name: string): Result[void, NippelError]

proc removeNippel*(manager: NippelManager, name: string,
                   purge: bool = false): Result[void, NippelError]

2. Profile Manager (profile_manager.nim)

Purpose: Security profile management

Key Types:

type
  SecurityProfile* = enum
    Workstation, Homestation, Satellite, NetworkIOT, Server

  ProfileSettings* = object
    isolationLevel*: IsolationLevel
    desktopIntegration*: bool
    networkAccess*: NetworkAccess
    resourceLimits*: ResourceLimits
    xdgStrategy*: XDGStrategy

  ProfileOverrides* = object
    isolationLevel*: Option[IsolationLevel]
    desktopIntegration*: Option[bool]
    networkAccess*: Option[NetworkAccess]

Key Functions:

proc loadProfile*(manager: ProfileManager,
                  profile: SecurityProfile): ProfileSettings

proc applyOverrides*(settings: ProfileSettings,
                     overrides: ProfileOverrides): ProfileSettings

3. Namespace Subsystem (namespace_subsystem.nim)

Purpose: Linux namespace management

Key Types:

type
  IsolationLevel* = enum
    None, Standard, Strict, Quantum

  NamespaceConfig* = object
    mountNS*: bool
    pidNS*: bool
    ipcNS*: bool
    networkNS*: bool
    utsNS*: bool
    userNS*: bool

  NamespaceHandle* = object
    pid*: int
    namespaces*: NamespaceConfig

Key Functions:

proc createNamespaces*(config: NamespaceConfig): Result[NamespaceHandle, string]

proc enterNamespace*(handle: NamespaceHandle): Result[void, string]

proc exitNamespace*(handle: NamespaceHandle): Result[void, string]

proc destroyNamespaces*(handle: NamespaceHandle): Result[void, string]

4. XDG Enforcer (xdg_enforcer.nim)

Purpose: XDG directory enforcement

Key Types:

type
  XDGStrategy* = enum
    Portable, SystemIntegrated

  XDGDirectories* = object
    dataHome*: string
    configHome*: string
    cacheHome*: string
    stateHome*: string

Key Functions:

proc createXDGStructure*(rootPath: string,
                         strategy: XDGStrategy): Result[XDGDirectories, string]

proc setXDGEnvironment*(dirs: XDGDirectories): Result[void, string]

proc redirectLegacyPaths*(dirs: XDGDirectories): Result[void, string]

5. CAS Backend (cas_backend.nim)

Purpose: Content-addressable storage with deduplication

Key Types:

type
  CASBackend* = ref object
    storePath*: string
    hashAlgorithm*: HashAlgorithm
    refCounts*: Table[string, int]

  HashAlgorithm* = enum
    Xxh3_128, Blake2b512

Key Functions:

proc storeFile*(cas: CASBackend, filePath: string): Result[string, string]

proc retrieveFile*(cas: CASBackend, hash: string,
                   destPath: string): Result[void, string]

proc garbageCollect*(cas: CASBackend): Result[int, string]

6. Merkle Tree (merkle_tree.nim)

Purpose: Integrity verification

Key Types:

type
  MerkleTree* = ref object
    root*: MerkleNode
    algorithm*: HashAlgorithm

  MerkleNode* = ref object
    hash*: string
    path*: string
    children*: seq[MerkleNode]

Key Functions:

proc buildTreeFromFiles*(files: seq[string],
                         algorithm: HashAlgorithm): Result[MerkleTree, string]

proc verifyTree*(tree: MerkleTree): Result[bool, string]

proc updateTree*(tree: MerkleTree,
                 changes: seq[FileChange]): Result[MerkleTree, string]

proc diffTrees*(tree1, tree2: MerkleTree): seq[FileDiff]

7. UTCP Protocol (utcp_protocol.nim)

Purpose: AI-addressability

Key Types:

type
  UTCPAddress* = object
    scheme*: string
    host*: string
    resource*: string
    port*: int

  UTCPRequest* = object
    address*: UTCPAddress
    method*: string
    headers*: Table[string, string]
    body*: string

  UTCPResponse* = object
    status*: int
    headers*: Table[string, string]
    body*: string

Key Functions:

proc assignUTCPAddress*(nippelName: string): UTCPAddress

proc handleUTCPRequest*(request: UTCPRequest): Result[UTCPResponse, string]

proc formatUTCPAddress*(address: UTCPAddress): string

API Reference

Creating a Nippel

import nippels, nippel_types, profile_manager

# Create manager
let manager = newNippelManager()

# Create Nippel with default profile
let result = manager.createNippel("my-app", Workstation)
if result.isOk:
  echo "Nippel created successfully"
else:
  echo "Error: ", result.error

# Create with custom overrides
let overrides = ProfileOverrides(
  isolationLevel: some(Strict),
  networkAccess: some(Limited)
)
let result2 = manager.createNippelWithOverrides("secure-app", Satellite, overrides)

Activating a Nippel

# Activate Nippel
let activateResult = manager.activateNippel("my-app")
if activateResult.isOk:
  echo "Nippel activated"
  # Now in isolated environment
else:
  echo "Activation failed: ", activateResult.error

# Check if active
if manager.isNippelActive("my-app"):
  echo "Nippel is active"

# Deactivate
let deactivateResult = manager.deactivateNippel("my-app")

Working with Profiles

import profile_manager

let profileMgr = newProfileManager()

# Load profile
let settings = profileMgr.loadProfile(Workstation)
echo "Isolation level: ", settings.isolationLevel
echo "Desktop integration: ", settings.desktopIntegration

# Apply overrides
let overrides = ProfileOverrides(
  desktopIntegration: some(false)
)
let customSettings = settings.applyOverrides(overrides)

Using CAS Backend

import cas_backend

let cas = newCASBackend("/var/lib/nip/cas", Xxh3_128)

# Store file
let storeResult = cas.storeFile("/path/to/file.txt")
if storeResult.isOk:
  let hash = storeResult.value
  echo "File stored with hash: ", hash

# Retrieve file
let retrieveResult = cas.retrieveFile(hash, "/path/to/dest.txt")

# Garbage collect
let gcResult = cas.garbageCollect()
if gcResult.isOk:
  echo "Removed ", gcResult.value, " unreferenced entries"

Building Merkle Trees

import merkle_tree

# Build tree from files
let files = @["/path/to/file1.txt", "/path/to/file2.txt"]
let treeResult = buildTreeFromFiles(files, Xxh3_128)

if treeResult.isOk:
  let tree = treeResult.value
  echo "Merkle root: ", tree.root.hash

  # Verify tree
  let verifyResult = tree.verifyTree()
  if verifyResult.isOk and verifyResult.value:
    echo "Tree verified successfully"

UTCP Protocol

import utcp_protocol

# Assign UTCP address
let address = assignUTCPAddress("my-app")
echo "UTCP address: ", formatUTCPAddress(address)
# Output: utcp://localhost/nippel/my-app

# Create request
let request = UTCPRequest(
  address: address,
  method: "GET",
  headers: {"Accept": "application/json"}.toTable,
  body: ""
)

# Handle request
let response = handleUTCPRequest(request)
if response.isOk:
  echo "Response: ", response.value.body

Extension Points

Custom Security Profiles

You can define custom security profiles:

# In profile_manager.nim

proc loadCustomProfile*(name: string): ProfileSettings =
  case name
  of "my-custom-profile":
    result = ProfileSettings(
      isolationLevel: Strict,
      desktopIntegration: true,
      networkAccess: Limited,
      resourceLimits: ResourceLimits(
        maxMemory: 2_000_000_000,  # 2GB
        maxCPU: 50
      ),
      xdgStrategy: Portable
    )
  else:
    raise newException(ValueError, "Unknown profile: " & name)

Custom Hash Algorithms

Add support for new hash algorithms:

# In cas_backend.nim

type
  HashAlgorithm* = enum
    Xxh3_128, Blake2b512, MyCustomHash

proc computeHash*(data: string, algorithm: HashAlgorithm): string =
  case algorithm
  of Xxh3_128:
    result = xxh3_128(data)
  of Blake2b512:
    result = blake2b_512(data)
  of MyCustomHash:
    result = myCustomHashFunction(data)

Custom UTCP Methods

Extend UTCP protocol with custom methods:

# In utcp_protocol.nim

proc handleCustomMethod*(request: UTCPRequest): Result[UTCPResponse, string] =
  case request.method
  of "CUSTOM_METHOD":
    # Handle custom method
    let response = UTCPResponse(
      status: 200,
      headers: {"Content-Type": "application/json"}.toTable,
      body: """{"result": "success"}"""
    )
    return ok(response)
  else:
    return err("Unknown method: " & request.method)

Custom Namespace Configurations

Define custom namespace configurations:

# In namespace_subsystem.nim

proc getCustomNamespaceConfig*(level: string): NamespaceConfig =
  case level
  of "my-custom-level":
    result = NamespaceConfig(
      mountNS: true,
      pidNS: true,
      ipcNS: true,
      networkNS: false,  # Custom: no network isolation
      utsNS: true,
      userNS: false
    )
  else:
    raise newException(ValueError, "Unknown isolation level: " & level)

Development Setup

Prerequisites

Nim 2.0.0 or later
Linux kernel with namespace support
xxHash library (nimble install xxhash)
Standard Nim libraries

Building from Source

# Clone repository
git clone https://github.com/nexusos/nip.git
cd nip

# Install dependencies
nimble install -d

# Build
nimble build

# Run tests
nimble test

# Install
nimble install

Development Environment

# Set up development environment
export NIP_DEV_MODE=1
export NIP_LOG_LEVEL=debug

# Build with debug symbols
nim c -d:debug --debugger:native nip/src/nimpak/nippels.nim

# Run with debugger
gdb --args ./nippels

Code Style

Follow Nim standard style guide:

# Good
proc createNippel*(manager: NippelManager, name: string): Result[Nippel, NippelError] =
  ## Creates a new Nippel with the given name
  if name.len == 0:
    return err(NippelError(code: InvalidName, message: "Name cannot be empty"))

  # Implementation
  ok(nippel)

# Bad
proc CreateNippel(manager:NippelManager,name:string):Result[Nippel,NippelError]=
  if name.len==0:return err(NippelError(code:InvalidName,message:"Name cannot be empty"))
  ok(nippel)

Documentation Standards

All public APIs must be documented:

proc createNippel*(manager: NippelManager, name: string,
                   profile: SecurityProfile): Result[Nippel, NippelError] =
  ## Creates a new Nippel with the specified security profile.
  ##
  ## Parameters:
  ##   - manager: The NippelManager instance
  ##   - name: Unique name for the Nippel
  ##   - profile: Security profile to use
  ##
  ## Returns:
  ##   - Ok(Nippel) on success
  ##   - Err(NippelError) on failure
  ##
  ## Example:
  ## ```nim
  ## let manager = newNippelManager()
  ## let result = manager.createNippel("my-app", Workstation)
  ## if result.isOk:
  ##   echo "Created: ", result.value.name
  ## ```

  # Implementation

Testing

Unit Tests

# test_nippels.nim
import unittest, nippels, nippel_types

suite "Nippel Manager Tests":
  setup:
    let manager = newNippelManager()

  test "Create Nippel":
    let result = manager.createNippel("test-app", Workstation)
    check result.isOk
    check result.value.name == "test-app"

  test "Activate Nippel":
    discard manager.createNippel("test-app", Workstation)
    let result = manager.activateNippel("test-app")
    check result.isOk
    check manager.isNippelActive("test-app")

  test "Deactivate Nippel":
    discard manager.createNippel("test-app", Workstation)
    discard manager.activateNippel("test-app")
    let result = manager.deactivateNippel("test-app")
    check result.isOk
    check not manager.isNippelActive("test-app")

Integration Tests

# test_integration.nim
import unittest, nippels, profile_manager, namespace_subsystem

suite "Integration Tests":
  test "Full Nippel Lifecycle":
    let manager = newNippelManager()

    # Create
    let createResult = manager.createNippel("integration-test", Workstation)
    check createResult.isOk

    # Activate
    let activateResult = manager.activateNippel("integration-test")
    check activateResult.isOk

    # Verify active
    check manager.isNippelActive("integration-test")

    # Deactivate
    let deactivateResult = manager.deactivateNippel("integration-test")
    check deactivateResult.isOk

    # Remove
    let removeResult = manager.removeNippel("integration-test")
    check removeResult.isOk

Performance Tests

# test_performance.nim
import unittest, nippels, times

suite "Performance Tests":
  test "Nippel Creation Performance":
    let manager = newNippelManager()
    let start = cpuTime()

    for i in 1..100:
      discard manager.createNippel("perf-test-" & $i, Workstation)

    let elapsed = cpuTime() - start
    let avgTime = elapsed / 100.0

    echo "Average creation time: ", avgTime * 1000, " ms"
    check avgTime < 0.1  # Should be < 100ms

  test "Nippel Activation Performance":
    let manager = newNippelManager()
    discard manager.createNippel("perf-test", Workstation)

    let start = cpuTime()
    discard manager.activateNippel("perf-test")
    let elapsed = cpuTime() - start

    echo "Activation time: ", elapsed * 1000, " ms"
    check elapsed < 0.05  # Should be < 50ms

Running Tests

# Run all tests
nimble test

# Run specific test suite
nim c -r tests/test_nippels.nim

# Run with coverage
nim c -d:coverage -r tests/test_nippels.nim

# Run performance tests
nim c -d:release -r tests/test_performance.nim

Contributing

Contribution Guidelines

Fork the repository
Create a feature branch
```
git checkout -b feature/my-new-feature
```
Make your changes
- Follow code style guidelines
- Add tests for new features
- Update documentation
Run tests
```
nimble test
```

Commit your changes

git commit -am "Add new feature: description"

Push to your fork
```
git push origin feature/my-new-feature
```
Create a Pull Request

Code Review Process

All PRs require at least one review
Tests must pass
Documentation must be updated
Code style must be consistent

Areas for Contribution

New security profiles
Additional hash algorithms
UTCP protocol extensions
Performance optimizations
Documentation improvements
Bug fixes
Test coverage improvements

18 KiB Raw Blame History

Nippels Developer Guide

Table of Contents

Architecture Overview

High-Level Architecture

Module Dependencies

Core Components

1. Nippel Manager (nippels.nim)

2. Profile Manager (profile_manager.nim)

3. Namespace Subsystem (namespace_subsystem.nim)

4. XDG Enforcer (xdg_enforcer.nim)

5. CAS Backend (cas_backend.nim)

6. Merkle Tree (merkle_tree.nim)

7. UTCP Protocol (utcp_protocol.nim)

API Reference

Creating a Nippel

Activating a Nippel

Working with Profiles

Using CAS Backend

Building Merkle Trees

UTCP Protocol

Extension Points

Custom Security Profiles

Custom Hash Algorithms

Custom UTCP Methods

Custom Namespace Configurations

Development Setup

Prerequisites

Building from Source

Development Environment

Code Style

Documentation Standards

Testing

Unit Tests

Integration Tests

Performance Tests

Running Tests

Contributing

Contribution Guidelines

Code Review Process

Areas for Contribution

See Also

18 KiB

Raw Blame History