02 / Local AI Systems

SHADOW

An offline, local-first intelligent assistant loop in Python utilizing Ollama, deterministic tool routing, and persistent SQLite context storage on Apple Silicon.

Discipline Local AI / Agent Architecture
Runtime / Stack Python 3.12, Ollama, SQLite, AsyncIO
Status Active Development
Source Code Private Repository
SHADOW offline AI command center interface with central neural computational sphere and local execution telemetry

Overview

Most conversational AI interfaces depend entirely on remote cloud APIs. This architecture introduces recurring latency, recurring billing costs, and unavoidable privacy compromises when working with sensitive source code or personal local files.

SHADOW is an experiment in building a fully offline personal computing assistant. By leveraging local model runners like Ollama running directly on Apple Silicon unified memory, SHADOW executes reasoning loops, tool dispatches, and session state serialization entirely on the local machine with zero external network dependencies.

Runtime Loop & Tool Routing

The core orchestration engine is built around a deterministic state machine rather than open-ended prompt recursion. Key characteristics include:

  • Local Model Execution: Interfaces with local quantized weights (Llama 3, Mistral) served through Ollama's local HTTP endpoint on localhost:11434.
  • Strict Typed Tool Dispatch: Tools are declared as typed Python dataclasses. Function arguments are validated against strict JSON schemas before any execution attempt, eliminating hallucinated function calls.
  • Permission Boundaries: File system mutations, external script invocations, and terminal commands must pass through an explicit local confirmation gate before execution.
  • Non-Blocking I/O: Built on Python's asyncio to ensure terminal UI responsiveness while models stream token responses.

The Persistence Layer

Rather than adopting complex vector database engines or remote vector search endpoints, SHADOW relies on embedded SQLite with WAL (Write-Ahead Logging) mode enabled:

  • Structured Session Store: Conversation turns, tool invocations, and command execution traces are saved to indexed relational tables.
  • FTS5 Full-Text Indexing: Fast, lightweight lexical search across historical conversations and tool outputs without GPU embedding overhead.
  • Deterministic Context Pruning: When approaching context limits, SHADOW computes sliding window summaries and structured state checkpoints rather than blindly truncating historical tokens.

Technical Specifications

Primary Language Python 3.12+ (Type-annotated, AsyncIO)
Inference Provider Ollama (Local Apple Silicon Metal acceleration)
State Storage SQLite 3 with WAL mode & FTS5 enabled
Network Policy Zero telemetry, completely offline capable
License MIT License