nip/docs/source-build-guide.md

# NIP Source Build System - User Guide

## Introduction

The NIP Source Build System enables building packages from source using multiple package management systems (Nix, PKGSRC, Gentoo Portage) with full variant support (USE flags and OVERRIDES). This guide will help you get started and make the most of the system.

## Quick Start

### Basic Build

The simplest way to build a package:

```bash
nip build firefox
```

NIP will:
1. Auto-detect available source systems
2. Select the best source (priority: Nix > PKGSRC > Gentoo)
3. Build the package with default settings
4. Install to `/Programs`
5. Create system symlinks

### Build with Variants

Add variants to customize the build:

```bash
nip build firefox +wayland +lto
```

This builds Firefox with:
- Wayland display server support
- Link-time optimization enabled

### Choose Source System

Specify which source system to use:

```bash
nip build nginx --source=nix
nip build nginx --source=pkgsrc
nip build nginx --source=gentoo
```

## Understanding Variants

### What are Variants?

Variants are build-time options that customize how a package is compiled. They're organized into semantic domains:

- **Graphics:** Display server and graphics API options
- **Audio:** Audio server options
- **Optimization:** Compiler optimization options
- **Security:** Security hardening options

### Variant Syntax

Variants use the `+domain=value` syntax:

```bash
nip build <package> +domain=value
```

For common variants, you can omit the domain:

```bash
nip build firefox +wayland +lto +pipewire
```

### Available Variants

**Graphics:**
- `+wayland` - Wayland display server
- `+X` - X11 display server
- `+vulkan` - Vulkan graphics API

**Audio:**
- `+pipewire` - PipeWire audio server
- `+pulseaudio` - PulseAudio sound server
- `+alsa` - ALSA audio support

**Optimization:**
- `+lto` - Link-time optimization
- `+pgo` - Profile-guided optimization

**Security:**
- `+pie` - Position-independent executable
- `+hardened` - Full security hardening

### Combining Variants

You can combine multiple variants:

```bash
nip build firefox +wayland +vulkan +pipewire +lto +pie
```

## Source Systems

### Nix

**Advantages:**
- Largest package collection (~100,000+)
- Reproducible builds
- Binary cache available
- Excellent documentation

**When to use:**
- You need the latest versions
- You want reproducible builds
- You have Nix installed

**Example:**
```bash
nip build firefox +wayland --source=nix
```

### PKGSRC

**Advantages:**
- Portable (BSD, Linux, macOS)
- Stable and well-tested
- Good BSD support
- ~27,000 packages

**When to use:**
- You're on BSD
- You want portable builds
- You prefer traditional make-based builds

**Example:**
```bash
nip build nginx --source=pkgsrc
```

### Gentoo Portage

**Advantages:**
- Highly customizable
- USE flags for fine control
- ~20,000 packages
- Optimized for your system

**When to use:**
- You're on Gentoo
- You want maximum customization
- You need USE flag control

**Example:**
```bash
nip build vim --source=gentoo
```

## Build Caching

### How Caching Works

NIP caches builds based on variant fingerprints. If you build the same package with the same variants, NIP reuses the cached build instantly.

**First build:**
```bash
$ nip build firefox +wayland +lto
🔨 Building from source (this may take a while)...
⏱️  Duration: 245 seconds
✅ Build successful!
```

**Second build (cache hit):**
```bash
$ nip build firefox +wayland +lto
♻️  Using cached build
✅ Installed to: /Programs/firefox/...
⏱️  Duration: 0 seconds
```

### Cache Management

**View cache statistics:**
```bash
nip cache stats
```

**Clean old builds (30+ days):**
```bash
nip cache clean
```

**Clear all cache:**
```bash
nip cache clear
```

**Force rebuild (skip cache):**
```bash
nip build firefox +wayland --rebuild
```

## Advanced Usage

### Custom Variant Mappings

Create `~/.config/nip/variant-mappings.json` to define custom mappings:

```json
{
  "mypackage": {
    "feature": {
      "enabled": {
        "nix": "enableFeature = true",
        "pkgsrc": "feature",
        "gentoo": "feature",
        "description": "Enable custom feature"
      }
    }
  }
}
```

Then use it:
```bash
nip build mypackage +feature=enabled
```

### Build Without Installing

Build a package but don't install it:

```bash
nip build test-package --no-install
```

The artifact will be built and validated, but not grafted to `/Programs`.

### Keep Intermediate Files

Keep build work directories for debugging:

```bash
nip build firefox --keep-work
```

### Verbose Mode

See detailed build output:

```bash
nip build firefox +wayland --verbose
```

This shows:
- Nix expressions generated
- Build commands executed
- Detailed build output
- Grafting steps
- Symlink creation

## Package Discovery

### List Available Sources

See which source systems are installed:

```bash
nip sources
```

Output:
```
📚 Available Package Sources (by priority):

1. 🔵 Nix (nixpkgs)
   Status: ✅ Available
   Packages: ~100,000+

2. 🟢 PKGSRC (NetBSD)
   Status: ❌ Not installed
   Install: https://www.pkgsrc.org/

3. 🟣 Gentoo Portage
   Status: ❌ Not installed
   Install: https://www.gentoo.org/
```

### Search for Package

Search for a package across all sources:

```bash
nip sources bash
```

Output shows whether the package is available in each source system.

## Installation Structure

### Directory Layout

Built packages are installed to `/Programs`:

```
/Programs/
└── firefox/
    ├── 1.0.0/
    │   └── blake2b-abc123.../
    │       └── nix-blake2b-def456.../
    │           ├── bin/
    │           │   └── firefox
    │           ├── lib/
    │           └── share/
    └── Current -> 1.0.0/blake2b-abc123.../nix-blake2b-def456...
```

### System Symlinks

Executables and libraries are automatically symlinked:

```
/System/Links/
├── Executables/
│   └── firefox -> /Programs/firefox/Current/bin/firefox
└── Libraries/
    └── libfoo.so -> /Programs/firefox/Current/lib/libfoo.so
```

### Variant Tracking

All installed variants are tracked in the variant database:

- Package name and version
- Variant domains and values
- Source system used
- Installation path
- Timestamp
- Variant fingerprint

## Common Workflows

### Building a Web Browser

```bash
# Firefox with Wayland and optimizations
nip build firefox +wayland +lto +pipewire

# Chromium with X11
nip build chromium +X +lto
```

### Building a Web Server

```bash
# NGINX with IPv6
nip build nginx +ipv6

# From specific source
nip build nginx --source=pkgsrc
```

### Building Development Tools

```bash
# GCC with LTO
nip build gcc +lto

# Vim with all features
nip build vim +X +python +ruby
```

### Experimenting with Variants

```bash
# Build without installing to test
nip build test-package +experimental --no-install

# If it works, build and install
nip build test-package +experimental
```

## Best Practices

### 1. Start Simple

Build without variants first to ensure the package builds:

```bash
nip build firefox
```

Then add variants:

```bash
nip build firefox +wayland +lto
```

### 2. Use Cache

Let NIP cache builds. Don't use `--rebuild` unless necessary.

### 3. Check Sources

Before building, check if the package is available:

```bash
nip sources firefox
```

### 4. Use Verbose for Debugging

When troubleshooting, use verbose mode:

```bash
nip build firefox +wayland --verbose
```

### 5. Clean Cache Periodically

Remove old builds to save space:

```bash
nip cache clean
```

### 6. Custom Mappings for Frequent Packages

If you frequently build a package with specific variants, create custom mappings.

## Tips and Tricks

### Tip 1: Auto-Detection is Smart

NIP automatically selects the best source system. Trust it unless you have a specific reason to override.

### Tip 2: Cache is Your Friend

The cache makes rebuilds instant. Use `--rebuild` only when you need fresh builds.

### Tip 3: Variants are Portable

The same variant flags work across all source systems. NIP translates them automatically.

### Tip 4: Check Unmapped Variants

If a variant doesn't work, NIP will warn you. Add a custom mapping to fix it.

### Tip 5: Verbose Mode for Learning

Use `--verbose` to see exactly what NIP is doing. Great for learning and debugging.

## Getting Help

### Command Help

```bash
nip build --help
nip sources --help
nip cache --help
```

### Documentation

- This guide: `nip/docs/source-build-guide.md`
- Help reference: `nip/docs/build-system-help.md`
- Configuration: `nip/docs/configuration.md`

### Troubleshooting

See the "Troubleshooting" section above or the dedicated troubleshooting guide.

## Next Steps

1. **Try a simple build:** `nip build bash`
2. **Experiment with variants:** `nip build firefox +wayland`
3. **Explore sources:** `nip sources`
4. **Check cache:** `nip cache stats`
5. **Read advanced docs:** See configuration and troubleshooting guides

Happy building! 🚀