Architecture

HitKeep is designed to give you full control over every layer of the analytics stack — without requiring you to operate multiple services. The entire pipeline from HTTP ingest to dashboard query is embedded in a single Go binary. Nothing runs outside it; nothing phones home.

High-Level Data Flow

Data flows through four stages, all inside the single process:

Ingestion: The HTTP server receives a pageview or event from the tracking script.
Buffering: The hit is published to an in-memory NSQ queue, decoupling the HTTP response from the write path.
Processing: An internal consumer reads from the queue in micro-batches.
Storage: Hits are written to embedded DuckDB, a columnar OLAP database.

graph LR
    A[Browser / Server] -->|HTTP POST /ingest| B(Go HTTP Server)
    B -->|Publish| C{Embedded NSQ}
    C -->|Consume batches| D[Ingest Worker]
    D -->|INSERT| E[(DuckDB — hitkeep.db)]

The tracking script (hk.js) is also served from your instance — no CDN, no third-party domain.

1. Storage: DuckDB

HitKeep uses DuckDB, an in-process OLAP database engine, as its primary store.

Why DuckDB instead of PostgreSQL or ClickHouse?

Runs inside the Go process — no separate server, no socket, no authentication
Columnar storage optimizes analytical queries (aggregations, time-bucketing) over row-based access patterns
A single file (hitkeep.db) — backup is a file copy, restore is a file copy
~120 MB per million hits compressed — efficient for VPS-class storage
Queryable directly with the DuckDB CLI or any Parquet-compatible tool — your data is portable by nature

# Read your analytics database directly — no HitKeep running required
duckdb /var/lib/hitkeep/data/hitkeep.db \
  "SELECT date_trunc('day', timestamp), count(*) FROM hits GROUP BY 1 ORDER BY 1 DESC LIMIT 7;"

2. Ingestion Buffering: Embedded NSQ

Writing to a columnar database synchronously per HTTP request creates lock contention at scale. HitKeep solves this by embedding NSQ, a distributed messaging platform, in-process.

Decoupling: The ingest HTTP handler validates and enqueues the hit in memory, completing the request in microseconds.
Burst absorption: Traffic spikes (a site goes viral, a product launches) queue up without database pressure. The writer consumes at a steady, optimal pace.
Zero configuration: NSQ runs inside the process on configurable loopback ports. No Kafka cluster, no broker to manage.

NSQ’s TCP and HTTP interfaces bind to 127.0.0.1 by default and are not exposed externally.

3. Clustering: Leader / Follower

For high availability, HitKeep supports a Leader/Follower topology using HashiCorp Memberlist (gossip protocol) for node discovery.

Roles

Role	Behavior
Leader	Holds the `hitkeep.db` file lock, runs the NSQ consumer, writes to disk
Follower	Accepts HTTP traffic; proxies `/ingest` requests to the Leader

There is exactly one Leader at any time. If the Leader goes down, the cluster elects a new one — provided the PVC or data volume can be re-attached (Kubernetes StatefulSets handle this automatically).

Request Flow

Load Balancer → Follower → [proxy /ingest] → Leader → NSQ → DuckDB
                         → [serve API reads] → DuckDB read replica (same file, read-only)

For single-server deployments (Docker Compose, systemd), the node acts as Leader implicitly.

4. Security Layer

The HTTP server enforces multiple security controls before any data is processed:

Rate limiting: Per-IP token bucket limiters on /ingest, /api/*, and /api/login. Configurable rate and burst.
Sec-Fetch validation: Checks Sec-Fetch-* headers on state-changing requests.
JWT authentication: HTTP-only cookies signed with a configurable secret. Short expiry by default.
WebAuthn: FIDO2/Passkey challenge-response for passwordless login.
TOTP: RFC 6238 time-based one-time passwords for second-factor auth.

5. Frontend Architecture

The dashboard is a Single Page Application served from the same binary:

Framework: Angular v21 with Signals for reactive state
UI library: PrimeNG with Tailwind CSS
API contract: All dashboard functionality uses the same JSON REST API as external clients
Tracking snippet: hk.js is minified with esbuild (already part of the Angular toolchain) to stay under 2 KB. It is served from your instance — no third-party CDN, no cross-origin requests from your visitors’ browsers.

What the Binary Contains

hitkeep (single executable, ~80 MB)
├── Go HTTP server
├── Embedded NSQ broker + consumer
├── DuckDB engine + SQL migrations
├── Angular dashboard (compiled, embedded)
├── hk.js tracking snippet (embedded)
└── Background workers (retention, rollups, reports)

Every layer is auditable. The full source is on GitHub under the MIT license.

HitKeep Cloud runs this exact binary stack in dedicated infrastructure — EU (Frankfurt) or US (Virginia) — with the same auditability guarantees. Join the waitlist →