Drev CI
A self-hosted, container-native continuous integration engine written in Go. Built to run deterministic, DAG-based pipelines on your own infrastructure without opaque SaaS abstraction layers.
01. Overview
Modern continuous integration platforms have gradually evolved into multi-layered distributed monoliths. Running basic test suites, linters, and binary compilations for small-to-medium codebases frequently involves configuring external agents, negotiating metered SaaS credits, or provisioning complex container orchestrators.
I built Drev CI to explore the mechanics of containerized pipeline execution from first principles. The objective was to design a self-contained, compiled daemon in Go that can run directly on a developer workstation or a dedicated Linux host. It treats pipeline workflows not as opaque scripts, but as a Directed Acyclic Graph (DAG) executed in clean, ephemeral container sandboxes.
By communicating directly with the Docker daemon over a Unix domain socket, Drev CI eliminates intermediate shell wrappers, captures multiplexed execution logs at the byte stream level, and guarantees deterministic workspace teardown upon completion.
02. Daemon Architecture
Drev CI is engineered as a single Go binary that functions both as a long-running daemon and as a direct CLI executor. When a pipeline manifest is submitted, the engine coordinates four dedicated subsystems:
- DAG Dependency Resolver: Ingests declarative pipeline manifests, computes topological execution orders using graph traversal algorithms, and validates that no cyclic dependencies or deadlocks exist between jobs before execution begins.
-
Docker Engine API Client: Connects directly to
/var/run/docker.sockusing the official Go Docker SDK (github.com/docker/docker/client). Bypassing external shell invocations (e.g.docker run) prevents shell escaping bugs and gives the daemon low-level control over container creation, network isolation, and process status signals. - Job Scheduler & Volume Manager: Provisions isolated container instances per job with dedicated working directories. Manages shared host bind mounts and ephemeral volume attachments so artifacts can flow between dependent jobs safely.
-
Multiplexed Log Streaming Bus: Hooks into the raw output channels of running containers, demultiplexing interleaved
stdoutandstderrstreams. Broadcasts real-time, line-buffered execution events to connected web clients via Server-Sent Events (SSE).
03. Pipeline Specification (.drev.yml)
Rather than forcing jobs into rigid sequential phases, Drev CI uses a declarative DAG specification. Individual jobs declare explicit dependencies via the needs key, allowing independent jobs to run concurrently:
name: build-and-test
jobs:
checkout:
image: alpine/git:latest
steps:
- run: git clone https://github.com/KoushikSagarr/drevci.git .
lint:
image: golangci/golangci-lint:v1.59-alpine
needs: [checkout]
steps:
- run: golangci-lint run ./...
test:
image: golang:1.22-alpine
needs: [checkout]
steps:
- run: go test -v -race ./...
compile:
image: golang:1.22-alpine
needs: [lint, test]
steps:
- run: go build -o bin/drev ./cmd/drev
Each job runs within its own declared container image. Because lint and test both depend only on checkout, the scheduler provisions and executes them concurrently across available CPU cores, reducing total pipeline turnaround time.
04. Execution Lifecycle
Each pipeline execution traverses five deterministic phases:
-
1. Manifest Ingestion & Graph Validation: The daemon reads
.drev.yml, validates YAML syntax, and constructs an in-memory adjacency list. A cycle detection algorithm verifies that the graph is a valid DAG. -
2. Container Provisioning & Resource Quotas: For each scheduled job node, the engine issues container creation requests to the Docker socket. Strict memory and CPU resource limits are applied via
container.HostConfigto prevent runaway processes from starving host resources. -
3. Execution & Line-Buffered Log Streaming: Containers are attached and started. Output streams are read via
stdcopy.StdCopy, decoded, and pushed into buffered Go channels. A lightweight HTTP endpoint streams these events directly to the user interface via Server-Sent Events (SSE). -
4. Signal Interception & Safe Teardown: The engine traps POSIX signals (
SIGINT,SIGTERM). If a user aborts a run or the daemon shuts down, it issues graceful container stops followed by immediate removal (ContainerRemove), eliminating orphan container leaks. - 5. Exit Status Evaluation & Artifact Archival: Container exit codes are captured deterministically. If any prerequisite job yields a non-zero exit code, downstream dependent jobs are marked as skipped, and execution terminates with a concise failure summary.
05. Engineering Decisions & Trade-Offs
Several deliberate architectural choices shaped the design of Drev CI:
-
Direct Socket API vs. Shell Invocation: Interfacing with
/var/run/docker.sockdirectly requires managing API version handshakes and raw byte streams. However, it avoids shell injection risks, ensures deterministic lifecycle tracking, and provides granular visibility into container states that shell wrappers cannot deliver. - Single-Binary Architecture vs. Heavy Microservices: Choosing Go allowed compiling the daemon, API server, and CLI into a single, dependency-free binary. There is no requirement for external key-value stores, distributed message queues (Kafka/RabbitMQ), or runtime interpreters.
- Local Daemon Scope vs. Cluster Orchestration: Drev CI is explicitly focused on single-node execution for homelabs, dedicated servers, and edge environments. It purposefully avoids the operational complexity of Kubernetes operators in favor of speed and operational simplicity.
06. Implementation State
To maintain technical integrity, here is an accurate distinction between what is currently implemented in the Go repository and what is planned on the roadmap:
-
Implemented & Tested: Core Go runner daemon, CLI tool (
drev exec),.drev.ymlYAML parser with topological sort and cycle detection, Docker Engine API client integration over Unix socket, ephemeral container provisioning and teardown, live line-buffered SSE log streaming, and deterministic signal handling. - In Progress / Architectural Intent: Multi-node runner clustering, distributed artifact caching across builds, and automated webhook listeners for GitHub/GitLab repository push events.
07. Recognition & Reach
In November 2026, Drev CI was selected for inclusion in the public XERJ Corpus Hub following a personal invitation from its founder, Ivan.
The Corpus Hub is an open reference-code repository designed to make concrete, real-world engineering implementations discoverable to AI coding agents and automated development workflows. Rather than indexing generic boilerplates, the corpus indexes verified codebases that demonstrate practical architectural patterns.
The selected snapshot captures a pinned, stable commit of Drev CI:
-
Pinned Commit:
6f5f2c6742f735d6566dc48fea4c84178319a84(mac transfer v1). -
Indexed Implementation Scope: 12 core files spanning 1,091 lines of Go and TypeScript across the web dashboard, persistent storage engine (
internal/store), and REST/SSE API router (internal/api). - Public Attribution & Indexation: Author attribution is maintained through the public contributor directory, providing discoverability for autonomous agent environments under the MIT licence.
The indexed snapshot, public author attribution, and source code can be explored directly:
08. Technical Specifications
| Language & Toolchain | Go 1.22+ |
|---|---|
| Container Runtime | Docker Engine API via /var/run/docker.sock |
| Scheduling Engine | Directed Acyclic Graph (DAG) with Topological Sort |
| Telemetry Transport | HTTP Server-Sent Events (SSE) + Multiplexed Channels |
| Binary Distribution | Single compiled static binary with zero external runtime dependencies |
| Corpus Indexation | XERJ Corpus Hub (Pinned Commit 6f5f2c6) ↗ |
| Source Repository | github.com/KoushikSagarr/drevci ↗ |
| License | MIT License |