20 KiB

Raw Blame History

NIP Dependency Resolver - Visual Guide

Version: 1.0
Status: Production Ready
Last Updated: November 26, 2025

Overview

This guide provides visual diagrams and flowcharts to help understand the NIP dependency resolution system architecture and workflows.

Resolution Pipeline

High-Level Architecture

graph TB
    subgraph "User Interface"
        CLI[CLI Commands]
        API[Public API]
    end
    
    subgraph "Resolution Orchestrator"
        ORCH[Orchestrator]
        CACHE[Resolution Cache]
        METRICS[Metrics Tracker]
    end
    
    subgraph "Phase 1: Variant Unification"
        COLLECT[Collect Demands]
        UNIFY[Unify Variants]
        HASH[Calculate Hash]
    end
    
    subgraph "Phase 2: Graph Construction"
        FETCH[Fetch Manifests]
        BUILD[Build Graph]
        CYCLE[Detect Cycles]
    end
    
    subgraph "Phase 3: Resolution"
        TOPO[Topological Sort]
        SYNTH[Build Synthesis]
        ORDER[Installation Order]
    end
    
    subgraph "Storage Layer"
        CAS[Content-Addressable Storage]
        REPOS[Repositories]
    end
    
    CLI --> ORCH
    API --> ORCH
    
    ORCH --> CACHE
    ORCH --> COLLECT
    
    COLLECT --> UNIFY
    UNIFY --> HASH
    HASH --> FETCH
    
    FETCH --> REPOS
    FETCH --> BUILD
    BUILD --> CYCLE
    
    CYCLE --> TOPO
    TOPO --> SYNTH
    SYNTH --> CAS
    SYNTH --> ORDER
    
    ORDER --> ORCH
    ORCH --> METRICS

Three-Phase Resolution

Phase 1: Variant Unification

flowchart TD
    START([Start: Multiple Demands]) --> COLLECT[Collect All Variant Demands]
    COLLECT --> GROUP[Group by Package Name]
    GROUP --> LOOP{For Each Package}
    
    LOOP --> DOMAINS[Extract Domains]
    DOMAINS --> CHECK{Domain Type?}
    
    CHECK -->|Exclusive| EXCLUSIVE[Check for Conflicts]
    CHECK -->|Non-Exclusive| ACCUMULATE[Accumulate Flags]
    
    EXCLUSIVE --> CONFLICT{Conflict?}
    CONFLICT -->|Yes| ERROR[Return Conflict Error]
    CONFLICT -->|No| MERGE[Merge Domain]
    
    ACCUMULATE --> MERGE
    MERGE --> MORE{More Domains?}
    
    MORE -->|Yes| DOMAINS
    MORE -->|No| HASH[Calculate Variant Hash]
    
    HASH --> NEXT{More Packages?}
    NEXT -->|Yes| LOOP
    NEXT -->|No| SUCCESS([Return Unified Profiles])
    
    ERROR --> END([End: Conflict])
    SUCCESS --> END2([End: Success])
    
    style START fill:#e1f5e1
    style SUCCESS fill:#e1f5e1
    style ERROR fill:#ffe1e1
    style END fill:#ffe1e1
    style END2 fill:#e1f5e1

Phase 2: Graph Construction

flowchart TD
    START([Start: Unified Profiles]) --> INIT[Initialize Empty Graph]
    INIT --> QUEUE[Add Root Packages to Queue]
    
    QUEUE --> LOOP{Queue Empty?}
    LOOP -->|No| POP[Pop Package from Queue]
    
    POP --> VISITED{Already Visited?}
    VISITED -->|Yes| LOOP
    VISITED -->|No| MARK[Mark as Visited]
    
    MARK --> FETCH[Fetch Package Manifest]
    FETCH --> FOUND{Found?}
    
    FOUND -->|No| NOTFOUND[Package Not Found Error]
    FOUND -->|Yes| DEPS[Extract Dependencies]
    
    DEPS --> UNIFY[Unify Dependency Variants]
    UNIFY --> CONFLICT{Conflict?}
    
    CONFLICT -->|Yes| CONFERR[Variant Conflict Error]
    CONFLICT -->|No| NODE[Create Graph Node]
    
    NODE --> EDGE[Add Edges to Dependencies]
    EDGE --> ADDQ[Add Dependencies to Queue]
    ADDQ --> LOOP
    
    LOOP -->|Yes| CYCLE{Has Cycle?}
    CYCLE -->|Yes| CYCERR[Circular Dependency Error]
    CYCLE -->|No| SUCCESS([Return Complete Graph])
    
    NOTFOUND --> END([End: Error])
    CONFERR --> END
    CYCERR --> END
    SUCCESS --> END2([End: Success])
    
    style START fill:#e1f5e1
    style SUCCESS fill:#e1f5e1
    style NOTFOUND fill:#ffe1e1
    style CONFERR fill:#ffe1e1
    style CYCERR fill:#ffe1e1
    style END fill:#ffe1e1
    style END2 fill:#e1f5e1

Phase 3: Topological Sort

flowchart TD
    START([Start: Dependency Graph]) --> INDEG[Calculate In-Degree for All Nodes]
    INDEG --> ZERO[Find Nodes with In-Degree = 0]
    ZERO --> QUEUE[Add to Queue]
    
    QUEUE --> LOOP{Queue Empty?}
    LOOP -->|No| POP[Pop Node from Queue]
    
    POP --> ADD[Add to Result List]
    ADD --> EDGES[Get Outgoing Edges]
    EDGES --> DEC[Decrement In-Degree of Neighbors]
    
    DEC --> CHECK{In-Degree = 0?}
    CHECK -->|Yes| ADDQ[Add Neighbor to Queue]
    CHECK -->|No| NEXT{More Neighbors?}
    
    ADDQ --> NEXT
    NEXT -->|Yes| DEC
    NEXT -->|No| LOOP
    
    LOOP -->|Yes| VERIFY{Result Size = Node Count?}
    VERIFY -->|No| CYCLE[Cycle Detected Error]
    VERIFY -->|Yes| REVERSE[Reverse Result List]
    
    REVERSE --> SUCCESS([Return Installation Order])
    CYCLE --> END([End: Error])
    SUCCESS --> END2([End: Success])
    
    style START fill:#e1f5e1
    style SUCCESS fill:#e1f5e1
    style CYCLE fill:#ffe1e1
    style END fill:#ffe1e1
    style END2 fill:#e1f5e1

Variant System

Variant Profile Structure

graph TB
    subgraph "Variant Profile"
        PROFILE[Variant Profile<br/>Hash: xxh3-abc123]
        
        subgraph "Exclusive Domains"
            LIBC[libc Domain<br/>Exclusive: true<br/>Flags: musl]
            ALLOC[allocator Domain<br/>Exclusive: true<br/>Flags: jemalloc]
            ARCH[arch Domain<br/>Exclusive: true<br/>Flags: x86_64]
        end
        
        subgraph "Non-Exclusive Domains"
            FEAT[features Domain<br/>Exclusive: false<br/>Flags: ssl, http2, brotli]
            BUILD[build Domain<br/>Exclusive: false<br/>Flags: lto, pgo]
        end
    end
    
    PROFILE --> LIBC
    PROFILE --> ALLOC
    PROFILE --> ARCH
    PROFILE --> FEAT
    PROFILE --> BUILD
    
    style PROFILE fill:#e1e1ff
    style LIBC fill:#ffe1e1
    style ALLOC fill:#ffe1e1
    style ARCH fill:#ffe1e1
    style FEAT fill:#e1ffe1
    style BUILD fill:#e1ffe1

Variant Unification Example

graph LR
    subgraph "Demand 1: nginx"
        D1[features: ssl, http2<br/>libc: musl]
    end
    
    subgraph "Demand 2: nginx"
        D2[features: brotli<br/>allocator: jemalloc]
    end
    
    subgraph "Unification Process"
        MERGE[Merge Non-Exclusive<br/>Check Exclusive]
    end
    
    subgraph "Unified Profile"
        UNIFIED[features: ssl, http2, brotli<br/>libc: musl<br/>allocator: jemalloc<br/>Hash: xxh3-abc123]
    end
    
    D1 --> MERGE
    D2 --> MERGE
    MERGE --> UNIFIED
    
    style D1 fill:#e1f5e1
    style D2 fill:#e1f5e1
    style MERGE fill:#fff4e1
    style UNIFIED fill:#e1e1ff

Variant Conflict Example

graph LR
    subgraph "Demand 1: nginx"
        D1[libc: musl<br/>features: ssl]
    end
    
    subgraph "Demand 2: nginx"
        D2[libc: glibc<br/>features: http2]
    end
    
    subgraph "Unification Process"
        MERGE[Check Exclusive Domain<br/>libc: musl vs glibc]
    end
    
    subgraph "Result"
        CONFLICT[❌ CONFLICT<br/>Exclusive domain 'libc'<br/>has conflicting values]
    end
    
    D1 --> MERGE
    D2 --> MERGE
    MERGE --> CONFLICT
    
    style D1 fill:#e1f5e1
    style D2 fill:#e1f5e1
    style MERGE fill:#fff4e1
    style CONFLICT fill:#ffe1e1

Dependency Graph Examples

Simple Chain

graph TD
    APP[Application<br/>v1.0.0<br/>variant: default]
    LIB1[Library A<br/>v2.3.0<br/>variant: ssl]
    LIB2[Library B<br/>v1.5.0<br/>variant: default]
    BASE[Base Library<br/>v3.0.0<br/>variant: default]
    
    APP --> LIB1
    APP --> LIB2
    LIB1 --> BASE
    LIB2 --> BASE
    
    style APP fill:#e1e1ff
    style LIB1 fill:#e1ffe1
    style LIB2 fill:#e1ffe1
    style BASE fill:#ffe1e1

Installation Order: Base Library → Library A → Library B → Application

Diamond Dependency

graph TD
    ROOT[Root Package<br/>v1.0.0]
    LEFT[Left Dependency<br/>v2.0.0<br/>requires: common >= 1.0]
    RIGHT[Right Dependency<br/>v3.0.0<br/>requires: common >= 1.5]
    COMMON[Common Library<br/>v1.5.0<br/>✓ Satisfies both]
    
    ROOT --> LEFT
    ROOT --> RIGHT
    LEFT --> COMMON
    RIGHT --> COMMON
    
    style ROOT fill:#e1e1ff
    style LEFT fill:#e1ffe1
    style RIGHT fill:#e1ffe1
    style COMMON fill:#ffe1e1

Resolution: Common Library v1.5.0 satisfies both constraints (>= 1.0 and >= 1.5)

Circular Dependency (Error)

graph TD
    A[Package A<br/>depends on B]
    B[Package B<br/>depends on C]
    C[Package C<br/>depends on A]
    
    A --> B
    B --> C
    C -.->|❌ Cycle!| A
    
    style A fill:#ffe1e1
    style B fill:#ffe1e1
    style C fill:#ffe1e1

Error: Circular dependency detected: A → B → C → A

Conflict Detection

Conflict Types

graph TB
    subgraph "Conflict Detection"
        DETECT[Conflict Detector]
        
        subgraph "Conflict Types"
            VERSION[Version Conflict<br/>Incompatible version requirements]
            VARIANT[Variant Conflict<br/>Incompatible variant flags]
            CIRCULAR[Circular Dependency<br/>Cycle in dependency graph]
            MISSING[Missing Package<br/>Package not found]
        end
    end
    
    DETECT --> VERSION
    DETECT --> VARIANT
    DETECT --> CIRCULAR
    DETECT --> MISSING
    
    style DETECT fill:#e1e1ff
    style VERSION fill:#ffe1e1
    style VARIANT fill:#ffe1e1
    style CIRCULAR fill:#ffe1e1
    style MISSING fill:#ffe1e1

Version Conflict Example

graph LR
    subgraph "Package A"
        A[Requires: libssl >= 3.0]
    end
    
    subgraph "Package B"
        B[Requires: libssl < 3.0]
    end
    
    subgraph "Resolution"
        CONFLICT[❌ Version Conflict<br/>No version satisfies both<br/>>= 3.0 AND < 3.0]
    end
    
    A --> CONFLICT
    B --> CONFLICT
    
    style A fill:#e1f5e1
    style B fill:#e1f5e1
    style CONFLICT fill:#ffe1e1

Caching System

Three-Tier Cache Architecture

graph TB
    subgraph "Resolution Request"
        REQ[Resolution Request<br/>Package + Constraints]
    end
    
    subgraph "L1 Cache - Memory"
        L1[LRU Cache<br/>Capacity: 1000<br/>Speed: ~0.1ms<br/>Hit Rate: 85%]
    end
    
    subgraph "L2 Cache - CAS"
        L2[Content-Addressable Storage<br/>Speed: ~1-5ms<br/>Hit Rate: 10%]
    end
    
    subgraph "L3 Cache - SQLite"
        L3[Persistent Cache<br/>Speed: ~10-50ms<br/>Hit Rate: 5%]
    end
    
    subgraph "Resolution Engine"
        ENGINE[Full Resolution<br/>Speed: ~50-800ms]
    end
    
    REQ --> L1
    L1 -->|Miss| L2
    L2 -->|Miss| L3
    L3 -->|Miss| ENGINE
    
    L1 -->|Hit| RESULT[Return Result]
    L2 -->|Hit| RESULT
    L3 -->|Hit| RESULT
    ENGINE --> RESULT
    
    ENGINE -.->|Store| L3
    ENGINE -.->|Store| L2
    ENGINE -.->|Store| L1
    
    style REQ fill:#e1e1ff
    style L1 fill:#e1ffe1
    style L2 fill:#fff4e1
    style L3 fill:#ffe1e1
    style ENGINE fill:#e1e1ff
    style RESULT fill:#e1f5e1

Cache Invalidation

flowchart TD
    START([Repository Update]) --> CALC[Calculate New Repo State Hash]
    CALC --> COMPARE{Hash Changed?}
    
    COMPARE -->|No| SKIP[Skip Invalidation]
    COMPARE -->|Yes| INVALID[Invalidate Affected Entries]
    
    INVALID --> L1[Clear L1 Cache Entries]
    L1 --> L2[Mark L2 Entries as Stale]
    L2 --> L3[Update L3 Metadata]
    
    L3 --> DONE([Cache Invalidated])
    SKIP --> DONE
    
    style START fill:#e1e1ff
    style INVALID fill:#ffe1e1
    style DONE fill:#e1f5e1

Source Adapters

Source Class Hierarchy

graph TB
    subgraph "Source Adapters"
        BASE[Source Adapter<br/>Base Interface]
        
        subgraph "Frozen Sources"
            NIX[Nix Adapter<br/>Binary packages<br/>Fixed variants]
            ARCH[Arch Adapter<br/>Binary packages<br/>Fixed variants]
        end
        
        subgraph "Flexible Sources"
            GENTOO[Gentoo Adapter<br/>Source builds<br/>Configurable variants]
            NPK[NPK Adapter<br/>Source builds<br/>Full flexibility]
        end
    end
    
    BASE --> NIX
    BASE --> ARCH
    BASE --> GENTOO
    BASE --> NPK
    
    style BASE fill:#e1e1ff
    style NIX fill:#ffe1e1
    style ARCH fill:#ffe1e1
    style GENTOO fill:#e1ffe1
    style NPK fill:#e1ffe1

Source Selection Strategy

flowchart TD
    START([Package Request]) --> STRATEGY{Resolution Strategy?}
    
    STRATEGY -->|PreferBinary| FROZEN[Check Frozen Sources]
    STRATEGY -->|PreferSource| FLEXIBLE[Check Flexible Sources]
    STRATEGY -->|Balanced| BOTH[Check Both]
    
    FROZEN --> FOUND1{Variant Match?}
    FOUND1 -->|Yes| USE1[Use Frozen Package]
    FOUND1 -->|No| FALLBACK1[Fallback to Flexible]
    
    FLEXIBLE --> BUILD[Build from Source]
    
    BOTH --> FROZEN2[Check Frozen First]
    FROZEN2 --> FOUND2{Found & Recent?}
    FOUND2 -->|Yes| USE2[Use Frozen Package]
    FOUND2 -->|No| FLEXIBLE2[Use Flexible Source]
    
    FALLBACK1 --> BUILD
    FLEXIBLE2 --> BUILD
    
    USE1 --> RESULT([Return Package])
    USE2 --> RESULT
    BUILD --> RESULT
    
    style START fill:#e1e1ff
    style USE1 fill:#e1f5e1
    style USE2 fill:#e1f5e1
    style BUILD fill:#fff4e1
    style RESULT fill:#e1f5e1

NipCell Fallback

Conflict Resolution with NipCells

flowchart TD
    START([Unresolvable Conflict]) --> DETECT[Detect Conflict Severity]
    DETECT --> SEVERE{Severe Conflict?}
    
    SEVERE -->|No| REPORT[Report Conflict to User]
    SEVERE -->|Yes| SUGGEST[Suggest NipCell Isolation]
    
    SUGGEST --> USER{User Accepts?}
    USER -->|No| REPORT
    USER -->|Yes| CREATE[Create Separate NipCells]
    
    CREATE --> CELL1[Cell 1: Package A<br/>with dependencies]
    CREATE --> CELL2[Cell 2: Package B<br/>with dependencies]
    
    CELL1 --> RESOLVE1[Resolve in Cell 1]
    CELL2 --> RESOLVE2[Resolve in Cell 2]
    
    RESOLVE1 --> SUCCESS([Both Packages Installed])
    RESOLVE2 --> SUCCESS
    REPORT --> END([User Must Resolve])
    
    style START fill:#ffe1e1
    style SUGGEST fill:#fff4e1
    style CELL1 fill:#e1ffe1
    style CELL2 fill:#e1ffe1
    style SUCCESS fill:#e1f5e1
    style END fill:#ffe1e1

Cell Isolation

graph TB
    subgraph "Main System"
        MAIN[Main Environment<br/>Shared packages]
    end
    
    subgraph "Cell 1: Firefox"
        CELL1[Firefox Environment]
        FF[Firefox<br/>libssl 3.0]
        DEPS1[Dependencies<br/>compatible with libssl 3.0]
    end
    
    subgraph "Cell 2: Chromium"
        CELL2[Chromium Environment]
        CH[Chromium<br/>libssl 2.8]
        DEPS2[Dependencies<br/>compatible with libssl 2.8]
    end
    
    MAIN -.->|Shared| CELL1
    MAIN -.->|Shared| CELL2
    
    CELL1 --> FF
    CELL1 --> DEPS1
    
    CELL2 --> CH
    CELL2 --> DEPS2
    
    style MAIN fill:#e1e1ff
    style CELL1 fill:#e1ffe1
    style CELL2 fill:#ffe1e1
    style FF fill:#e1ffe1
    style CH fill:#ffe1e1

Performance Optimization

Resolution Performance

graph LR
    subgraph "Without Cache"
        COLD[Cold Resolution<br/>~800ms]
    end
    
    subgraph "With L1 Cache"
        L1HIT[L1 Hit<br/>~0.1ms<br/>600x faster]
    end
    
    subgraph "With L2 Cache"
        L2HIT[L2 Hit<br/>~3ms<br/>267x faster]
    end
    
    subgraph "With L3 Cache"
        L3HIT[L3 Hit<br/>~30ms<br/>27x faster]
    end
    
    COLD -.->|Cache| L3HIT
    L3HIT -.->|Promote| L2HIT
    L2HIT -.->|Promote| L1HIT
    
    style COLD fill:#ffe1e1
    style L3HIT fill:#fff4e1
    style L2HIT fill:#e1ffe1
    style L1HIT fill:#e1f5e1

Optimization Techniques

graph TB
    subgraph "Performance Optimizations"
        OPT[Optimization Layer]
        
        subgraph "Techniques"
            BIT[Bit Vector Unification<br/>10-100x speedup<br/>O(1) flag operations]
            INDEX[Indexed Conflict Detection<br/>O(n) instead of O(n²)<br/>Hash table indexing]
            HASH[Cached Hash Calculation<br/>~99% hit rate<br/>Memoization]
            POOL[Memory Pool Allocation<br/>2-5x faster<br/>Reduced allocations]
        end
    end
    
    OPT --> BIT
    OPT --> INDEX
    OPT --> HASH
    OPT --> POOL
    
    style OPT fill:#e1e1ff
    style BIT fill:#e1ffe1
    style INDEX fill:#e1ffe1
    style HASH fill:#e1ffe1
    style POOL fill:#e1ffe1

CLI Command Flow

nip resolve Command

sequenceDiagram
    participant User
    participant CLI
    participant Orchestrator
    participant Cache
    participant Resolver
    participant CAS
    
    User->>CLI: nip resolve nginx --use-flags=ssl,http2
    CLI->>CLI: Parse arguments
    CLI->>Orchestrator: resolve(demands)
    
    Orchestrator->>Cache: Check L1 cache
    alt Cache Hit
        Cache-->>Orchestrator: Return cached result
        Orchestrator-->>CLI: Resolution result
    else Cache Miss
        Orchestrator->>Resolver: Perform resolution
        Resolver->>Resolver: Phase 1: Unify variants
        Resolver->>Resolver: Phase 2: Build graph
        Resolver->>Resolver: Phase 3: Topological sort
        Resolver->>CAS: Calculate build hashes
        CAS-->>Resolver: Build hashes
        Resolver-->>Orchestrator: Resolution result
        Orchestrator->>Cache: Store in cache
        Orchestrator-->>CLI: Resolution result
    end
    
    CLI->>CLI: Format output
    CLI-->>User: Display results

nip conflicts Command

sequenceDiagram
    participant User
    participant CLI
    participant Orchestrator
    participant ConflictDetector
    participant Graph
    
    User->>CLI: nip conflicts
    CLI->>Orchestrator: detectConflicts()
    
    Orchestrator->>Graph: Get installed packages
    Graph-->>Orchestrator: Package list
    
    Orchestrator->>ConflictDetector: Analyze conflicts
    ConflictDetector->>ConflictDetector: Check version conflicts
    ConflictDetector->>ConflictDetector: Check variant conflicts
    ConflictDetector->>ConflictDetector: Check circular dependencies
    
    ConflictDetector-->>Orchestrator: Conflict list
    Orchestrator-->>CLI: Conflict report
    
    CLI->>CLI: Format conflicts
    CLI->>CLI: Generate suggestions
    CLI-->>User: Display conflicts + suggestions

State Diagrams

Resolution State Machine

stateDiagram-v2
    [*] --> Idle
    
    Idle --> Resolving: resolve()
    Resolving --> CacheCheck: Check cache
    
    CacheCheck --> CacheHit: Found in cache
    CacheCheck --> Unifying: Not in cache
    
    CacheHit --> Complete: Return result
    
    Unifying --> UnifySuccess: Variants unified
    Unifying --> UnifyFailed: Conflict detected
    
    UnifySuccess --> Building: Build graph
    Building --> BuildSuccess: Graph complete
    Building --> BuildFailed: Error occurred
    
    BuildSuccess --> Sorting: Topological sort
    Sorting --> SortSuccess: Order determined
    Sorting --> SortFailed: Cycle detected
    
    SortSuccess --> Synthesizing: Synthesize builds
    Synthesizing --> Complete: Success
    
    UnifyFailed --> Error: Report conflict
    BuildFailed --> Error: Report error
    SortFailed --> Error: Report cycle
    
    Complete --> Idle: Done
    Error --> Idle: Done
    
    Error --> [*]
    Complete --> [*]

Cache State Machine

stateDiagram-v2
    [*] --> Empty
    
    Empty --> Checking: Lookup request
    Checking --> Hit: Key found
    Checking --> Miss: Key not found
    
    Hit --> Valid: Check validity
    Valid --> Returning: Valid entry
    Valid --> Stale: Stale entry
    
    Stale --> Miss: Treat as miss
    Miss --> Resolving: Perform resolution
    
    Resolving --> Storing: Resolution complete
    Storing --> Cached: Store in cache
    
    Cached --> Checking: Next request
    Returning --> Checking: Next request
    
    Cached --> Invalidating: Repo update
    Invalidating --> Empty: Clear entries

20 KiB Raw Blame History