Skip to content

NewsroomEngineering notesNew

Idempotency is a product feature, not a database detail

Why every usage row, seat snapshot, and bank statement line in Sonic carries an explicit identity — and what happens when a file gets uploaded twice.

Engineering · Platform engineering27 Aug 20262 min read
  1. 01

    Configure the key, not guess it

    Idempotency columns are picker options an operator chooses per template — never auto-checked, never silently 'all columns'.

  2. 02

    Stage, then compute uniqueness

    Process computes the identity and the preview explicitly. Uploading a sample does nothing on its own.

  3. 03

    Re-upload is safe

    The same file processed twice does not double-bill. A changed file with the same rows is still recognised as the same rows.

Engineers usually file idempotency under "backend correctness." In a billing product it is a customer-facing promise: if ops accidentally uploads the same usage extract twice, nobody gets double-billed, and if they upload a corrected version of the same file, Sonic does not treat every row as new.

The design choice we made is to keep the idempotency key visible and operator-configured rather than inferred. When you build a usage or seat template, the columns available for uniqueness are shown as picker options — they are never auto-checked, and they are never silently "every column," because a stray text ID or an Excel #REF! in an unrelated column would break arithmetic that never mentions it. That failure mode is common in real vendor exports, not a theoretical edge case.

A previous selection survives a sample or worksheet switch only if the column still exists in the new sheet, or if it genuinely was not part of the previous sheet at all — a column picked from the old sheet with no match in the new one gets dropped rather than silently pointing at nothing. The same merge logic is shared between usage and seat templates, so the behaviour does not drift between the two ingestion paths.

For seat data specifically, the identity problem is different in kind: a seat file is a daily snapshot, not an event stream. We compare each snapshot to the customer's balance on record and emit an event only when the count changes, skipping already-imported rows by a stored snapshot key while still advancing the running count underneath. That is what lets a backfill or a re-upload behave correctly instead of re-emitting every historical add.

None of this is visible in a demo unless you ask for it. It is exactly the kind of engineering that never shows up in a screenshot and is the difference between a system finance can trust with a monthly file and one that needs a human to babysit every upload.

Read this on your own numbers.

Bring a contract or a carrier file — we'll run it live.

Book a demo