Performance and Memory

HitKeep’s embedded DuckDB engine caches database pages in memory to keep dashboards fast, and by default DuckDB allows itself up to 80% of system RAM for that cache. On a dedicated analytics host that is desirable, but on a small or shared VPS it looks like a leak.

Since HitKeep 2.10.1, it is bounded by default.

What HitKeep does automatically

Memory limit. When HITKEEP_DUCKDB_MEMORY_LIMIT is not set, HitKeep derives a limit at startup:

1. If GOMEMLIMIT is set, HitKeep treats it as the process budget and gives DuckDB that budget minus roughly 512 MiB for the Go side.
2. Otherwise it takes the effective memory, usually container (cgroup) limit when running in Docker or Kubernetes, otherwise physical RAM and gives DuckDB half of it, clamped between 1 GiB and 16 GiB.

The resolved value appears in the startup configuration log and in the system report. Set HITKEEP_DUCKDB_MEMORY_LIMIT=none to restore DuckDB’s own unbounded default.

Threads. When HITKEEP_DUCKDB_THREADS is not set, HitKeep uses Go’s GOMAXPROCS, which respects container CPU quotas. On bare metal this equals the core count; in a CPU-limited container it prevents DuckDB from over-parallelizing, which also lowers peak query memory.

Self-healing. If DuckDB ever hits an unrecoverable in-memory fault (for example an out-of-memory during a write on an overloaded host), HitKeep now detects the condition, drains the affected connections, and reopens the database automatically. Before 2.10.1 this state required a manual restart.

Leaner analytics tables. 2.10.1 removes the internal indexes from the raw hits, events, and web_vitals tables. They made up roughly 60% of the database file, consumed memory proportional to row count, and slowed down every insert. The migration runs automatically on first start after the upgrade and rewrites those tables; expect one startup that takes a few extra seconds per few million rows.

Existing database files do not shrink on disk. New installs, backups, and exports are substantially smaller, and ingest is measurably faster.

Tuning for small hosts

# Rule of thumb: about twice your hitkeep.db file size.
HITKEEP_DUCKDB_MEMORY_LIMIT=2GB

Tips so far:

• Do not set the limit far below your database size. Query intermediates spill to disk, but writes need real headroom; an overly tight limit fails dashboard queries under concurrent load.
• Reducing HITKEEP_DUCKDB_THREADS lowers peak query memory and lets a given limit go further — try halving it before raising the memory limit.
• Expected resident memory is roughly the DuckDB limit plus 0.5–1 GiB for the application itself (HTTP server, embedded NSQ, IP metadata tables).
• In multi-team installs the limit applies per team database. See Teams and Data Isolation. This is a deliberate design decision for now.

All flags are listed in the configuration reference.

Reading the memory breakdown

System Status → Storage shows DuckDB’s own accounting (duckdb_memory()): buffer-managed table data, in-memory index structures, query intermediates, and temporary storage. The Copy system report button on the System Status page copies a full markdown report — version, configuration, storage, memory breakdown, ingest counters for you to paste into GitHub issue or support request.

If you would rather not think about memory limits, database sizing, or upgrade migrations at all, HitKeep Cloud runs and sizes instances automatically.