A common pattern in networked applications is maintaining a pool of reusable connections — database handles, HTTP clients, IMAP sessions. The challenge: how many connections should the pool hold? Too few and callers wait. Too many and you waste server resources and connection limits.

This article describes an adaptive pool implementation that automatically sizes itself between a configured minimum and maximum, driven by two independent signals measured passively during normal operation.

A fixed pool of N connections is always wrong for at least part of the time. During a traffic burst you need more; during quiet periods you're holding open connections that could be closed. Dynamic sizing is the right answer, but most approaches get the signal wrong.

Queue length (how many callers are waiting) is a lagging indicator. You only know there's a problem after callers are already blocked. And it only captures contention at the semaphore — if you have 10 idle connections and 10 simultaneous requests, all 10 get served instantly with zero queue length, yet the pool might still be the right size.

Utilisation ratio (outstanding / size) has the opposite problem: it fires on every burst even when nobody is waiting, causing grow/shrink oscillation. A burst that uses all 10 connections but serves them all in 1ms looks the same as one where callers wait 500ms.

The solution is to separate the two concerns:

These are independent questions with independent answers.

pressure = EMA of min(log(1 + waitMs) / log(1 + targetMs), 1.0)

waitTime is the actual wall-clock time each caller spent blocked waiting for a slot. targetWait is your acceptable latency budget per connection acquisition (e.g. 20ms if your page load target is 100ms).

The signal uses a logarithmic scale rather than a linear one. Connection wait times span many orders of magnitude: nanoseconds when a slot is immediately available, milliseconds under moderate load, seconds under heavy load. On a linear scale, the difference between 0ms and 1ms is lost in the noise when the target is 20ms (sample = 0.05). On a log scale that same difference registers clearly.

With targetWait = 20ms (log1p(20) ≈ 3.045):

Wait time linear sample log sample
0ns (instant) 0.000 0.000
1ms 0.050 0.228
5ms 0.250 0.589
10ms 0.500 0.787
20ms (target) 1.000 1.000
50ms+ → 1.0 (capped) → 1.0 (capped)

A 1ms wait produces a log sample of 0.228 vs linear's 0.050 — four times more sensitive to short waits that would otherwise be invisible.

Grow by 1 when pressure > 0.3 — callers are consistently experiencing meaningful wait times relative to the target latency.

The key insight: this is sampled after unblocking from the semaphore, so it directly measures the thing you care about. No proxies.

idleness = EMA of (idleConnections / size)

Sampled in release, after the connection is returned to the pool. This measures what fraction of the pool is currently sitting idle.

Shrink by 1 when idleness > 0.5 — more than half the pool is consistently idle after releases.

Consider a burst of 10 simultaneous requests against a pool of 10 connections:

  1. All 10 requests arrive, all 10 connections are claimed instantly. pressure ≈ 0 (no waiting), idleness ≈ 0 (all in use). No grow, no shrink.

  2. Requests complete, connections are released one by one. After each release, len(conns) grows: 1/10, 2/10, 3/10... Idleness EMA starts rising.

  3. After the burst, all 10 connections are idle. idleness → 1.0. Shrink fires — pool reduces toward the appropriate steady-state size.

  4. Next burst arrives. If waiting occurs, pressure rises → grow fires.

The pool tracks actual usage patterns rather than instantaneous snapshots.

type pool struct {
    min, max int

    mu          sync.Mutex
    size        int     // current target: min ≤ size ≤ max
    outstanding int     // connections held by callers
    draining    int     // slots scheduled for permanent removal
    pressure    float64 // EMA grow signal
    idleness    float64 // EMA shrink signal

    // Semaphore: RECEIVE to claim a slot, SEND to return one.
    // Buffered at max so size can grow up to max without reallocation.
    sem   chan struct{}
    conns chan *Connection
}

The semaphore channel uses an important convention: receive = claim a slot, send = return a slot. This is the opposite of the more common "send to acquire" pattern. The reason: we want <-sem to block when no slots are available (pool is full), which means the channel must be empty when full. Pre-filling with initSize items makes initSize slots available immediately.

func newPool(min, max int) *pool {
    initSize := min(min*3, max) // start warmed up but not at max
    p := &pool{
        min:  min,
        max:  max,
        size: initSize,
        sem:  make(chan struct{}, max), // buffered at max, never reallocated
        conns: make(chan *Connection, max),
    }
    for i := 0; i < initSize; i++ {
        p.sem <- struct{}{} // pre-fill initSize slots
    }
    return p
}

Starting at min * 3 (capped at max) gives the pool a warm start — enough capacity for a typical burst without opening connections upfront.

func (p *pool) acquire() (*Connection, error) {
    start := time.Now()
    <-p.sem                    // block until a slot is available
    waited := time.Since(start)

    p.mu.Lock()
    p.outstanding++

    // Update pressure EMA using log-scaled wait time.
    // log1p(x) = log(1+x), avoids log(0) and spreads out short waits.
    waitMs := waited.Seconds() * 1000
    targetMs := targetWait.Seconds() * 1000
    sample := min(math.Log1p(waitMs)/math.Log1p(targetMs), 1.0)
    p.pressure = alpha*sample + (1-alpha)*p.pressure

    // Grow if callers are waiting too long.
    if p.pressure > growThreshold && p.size < p.max {
        p.size++
        p.sem <- struct{}{} // mint a new slot
    }
    p.mu.Unlock()

    // Reuse idle connection or open new one.
    select {
    case c := <-p.conns:
        return c, nil
    default:
        return openNewConnection()
    }
}

The pressure sample is taken after unblocking. If we waited, the sample is high. If we got a slot instantly, it's near zero. This correctly distinguishes "pool is big enough" from "pool is too small".

Growing is done by sending one extra item to sem — this is safe because sem is buffered at max, and the mutex ensures size ≤ max is maintained.

func (p *pool) release(c *Connection) {
    go func() { // fire-and-forget
        // Health check before touching pool state.
        if err := c.Ping(); err != nil {
            c.Close()
            // Return the slot — pool size is unchanged, a fresh connection
            // will be opened on next acquire.
            p.mu.Lock(); p.outstanding--; p.mu.Unlock()
            p.sem <- struct{}{}
            return
        }

        p.mu.Lock()
        p.outstanding--
        shouldDrain := p.draining > 0
        if shouldDrain { p.draining-- }
        p.mu.Unlock()

        if shouldDrain {
            c.Close()
            return // slot NOT returned — pool permanently shrinks by 1
        }

        // Return connection.
        p.conns <- c

        // Sample idleness and check for shrink.
        p.mu.Lock()
        idleSample := float64(len(p.conns)) / float64(max(p.size, 1))
        p.idleness = alpha*idleSample + (1-alpha)*p.idleness

        if p.idleness > shrinkThreshold && p.size > p.min && p.draining == 0 {
            p.size--
            p.draining++
        }
        p.mu.Unlock()

        p.sem <- struct{}{} // return the slot
    }()
}

release is fire-and-forget — the caller returns immediately. A goroutine performs the health check and idleness sampling in the background.

The shrink mechanism: instead of immediately removing a slot, we increment draining. The next release goroutine that finds draining > 0 discards its connection and does not return the slot to sem. This permanently reduces the pool's capacity by 1. The draining counter ensures we only shrink by 1 per decision, even if multiple releases happen before the drain completes.

The health check: sending a NOOP (or Ping) verifies the connection is still alive. If it fails, the connection is discarded but the slot is returned to sem — the pool size is unchanged, and the next acquire will open a fresh connection. This is important for resilience: if the laptop sleeps overnight and all connections go stale, they are discarded one by one on use, each returning their slot, so the pool remains fully functional.

Here is a real log trace from the implementation serving an email web client, showing two load bursts separated by a quiet period:

19:23:19 grow size=11 pressure=0.32 waited=94ms
19:23:19 grow size=12 pressure=0.45 waited=93ms   ← cascade: each waiter
19:23:19 grow size=13 pressure=0.36 waited=1µs    ← unblocks and samples
19:23:19 acquire new (size=13 outstanding=13 waited=752ns)
                                                   ← pressure now < 0.3, stops
19:23:27 shrink size=12 idleness=0.54
19:23:29 shrink size=11 idleness=0.61             ← quiet period: slow drain
19:24:22 shrink size=10 idleness=0.60
  ... (continues shrinking every ~30-60s) ...
19:25:07 shrink size=5  idleness=0.53

19:25:08 grow size=6  pressure=0.36 waited=343ms  ← new burst hits small pool
19:25:08 grow size=7  pressure=0.49 waited=299ms
19:25:16 grow size=8  pressure=0.39 waited=932ns  ← further grow from next request
19:25:16 grow size=9  pressure=0.31 waited=1µs
19:25:23 shrink size=8 idleness=0.51              ← settles, starts shrinking again

When multiple requests are waiting simultaneously, each one that unblocks contributes an EMA sample:

This is the self-limiting property: each grow immediately unblocks one waiter, whose lower wait time brings the pressure back down. The pool grows by exactly as many steps as needed to drain the queue, then stops.

After the burst, the 8 remaining idle connections produce idleness ≈ 0.6 on each release. The EMA smoothing means it takes ~5-10 releases before the threshold is reliably exceeded, so shrinks fire roughly every 30-60 seconds during quiet periods — gradual, not immediate. This is intentional: a pool that shrinks aggressively after every quiet moment would oscillate.

The two signals operate on different timescales by design. Pressure spikes fast (a single 94ms wait pushes the EMA well above threshold) so growth is quick. Idleness accumulates slowly (each release only nudges the EMA by alpha=0.2) so shrinkage is gradual. The asymmetry means the pool is always ready to serve a burst but doesn't hold excess connections forever.

Parameter Description Typical value
alpha EMA smoothing factor 0.2 (≈ last 5 events)
targetWait Acceptable wait per acquire 20ms
growThreshold Pressure level that triggers grow 0.3
shrinkThreshold Idleness level that triggers shrink 0.5
min Minimum pool size 2–3
max Maximum pool size 10–25
initSize Starting size min(min*3, max)

The EMA alpha of 0.2 means recent events have more influence but a single anomalous event doesn't cause an immediate resize. Lower alpha = more conservative (slower to react), higher = more aggressive (faster but noisier).

No oscillation: grow and shrink are driven by different signals measured at different times. A burst that causes all connections to be claimed but serves them quickly will see pressure ≈ 0 (no waiting) and idleness → 0 (all in use) — neither threshold fires during the burst.

Lazy: no connections are opened until needed. The pool starts with initSize slots (semaphore capacity) but opens actual connections on demand.

Self-healing after sleep/disconnect: dead connections are discarded and their slots returned, keeping the pool at full capacity. The pool seamlessly replenishes as connections are reused.

Bounded: the pool never exceeds max connections. sem is buffered at max and the mutex ensures size ≤ max before any grow operation.

Non-blocking release: callers are never delayed by the health check. The background goroutine handles the round-trip independently.