Every cached download wrote its hit in its own UPDATE. SQLite runs on
one connection, so each download was a write transaction the rest
queued behind, holding a warm npm cache near 1,000 requests a second
with the CPU two thirds idle.
Hits are now counted in memory and written in one transaction every
database.hit_flush_interval, "1s" by default, "0" for a write per hit.
A failed write keeps the hits for the next flush, and Close writes
what is pending. The same test then served about 2,200 requests a
second.
Closes#322.
* fix(database): set connection pool limits for Postgres
OpenPostgres returned sqlx.Open's handle with database/sql's defaults:
no cap on open connections and two idle ones. Under load nearly every
request opened a new Postgres session, ran its few statements and closed
it again, paying a backend fork and SCRAM authentication each time.
Set the pool limits the issue suggests: 32 open and 32 idle connections,
idle connections closed after 5 minutes and every connection recycled
after 30 minutes. The values live in named constants because the mnd
linter rejects the literals inline.
The test (skipped without PROXY_DATABASE_URL, like the other Postgres
tests) takes 16 connections from the pool, releases them and checks that
all 16 stay idle; with the default pool only two survive.
Fixes#323
* test(database): pin the pool properties instead of the wiring
Review pointed out that the pool test compared MaxOpenConnections to the
constant it was set from, so the assertion followed the constant and
would have accepted postgresMaxOpenConns = 0 (unlimited), and that its
burst of 16 only proved MaxIdleConns >= 16.
The burst now takes postgresMaxIdleConns connections, so every
configured idle slot has to survive the release, and the open cap is
checked to be finite and large enough for that burst before any
connection is taken, so a cap below the idle count fails fast instead
of blocking in db.Conn. The doc comment now says which settings the
test covers; the idle-time and lifetime settings only show up in
DBStats.MaxIdleTimeClosed and MaxLifetimeClosed after minutes of
wall-clock time and stay unexercised.
* test(database): guard the pool test against a vacuous burst
Review showed that with the burst tied to postgresMaxIdleConns the test
also passed for a constant of 2, database/sql's default, or of 0, where
it took no connections at all. It now fails outright unless the
configured idle count exceeds the default, and every connection it
takes is released in a cleanup, so an assertion failure mid-burst no
longer leaves sessions open for the rest of the test binary.
* Fix all golangci-lint issues across the codebase
Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.
* Suppress staticcheck SA1019 for intentional deprecated field usage
The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
Replace raw database/sql with jmoiron/sqlx for cleaner query handling.
Support both SQLite (default) and PostgreSQL as configurable backends.
Configuration via:
- CLI flags: -database-driver, -database-path, -database-url
- Environment: PROXY_DATABASE_DRIVER, PROXY_DATABASE_PATH, PROXY_DATABASE_URL
- Config file: database.driver, database.path, database.url
Tests run against both databases when PROXY_DATABASE_URL is set.
