Titen: A Survey App That Assumes the Signal Will Drop

An enumerator walks into a village to register small businesses. Somewhere between the main road and the third house, the signal goes. They keep working, because the job does not stop for a bar of reception. Two hours later they get signal back on the ride home, in a moving vehicle, on a connection that comes and goes every few seconds.

Everything hard about this project lives in that last sentence.

A survey app that works offline is easy. You write to a local database and sync later. The difficulty is what happens during the sync, when the connection is bad enough to half-complete a request but good enough to keep retrying. Get that wrong and you do not lose data. You get something worse: duplicate data that looks real, and nobody notices until the totals are wrong at the end of the month.

So I built Titen around a single assumption. The network will fail mid-request, and the client will retry, and the server has to be fine with that.

The decisions that mattered

An outbox per mutation, not a sync button

The naive design queues “unsent records” and pushes them when online. It falls apart the moment one record in the batch fails: do you retry the batch, or the record, and what state is the queue in now?

Titen writes every change as its own outbox entry the moment it happens. Each entry carries what changed, when, and a retry count. A background worker drains the outbox with exponential backoff, one entry at a time, and an entry only leaves the outbox when the server has confirmed it.

The enumerator never presses sync. There is no sync screen. The app’s job is to make the queue empty eventually, and to be honest in the meantime about what has not landed yet.

Idempotent on a client-generated UUID

This is the part that makes retries safe, and it is one line of design that removes an entire category of bug.

The client generates a UUID when the record is created, not when it is sent. Every endpoint treats that UUID as the identity of the operation. Send the same mutation five times and the server creates one row and returns the same result five times.

That means the client is free to be stupid about retries. It does not need to know whether the previous attempt reached the server before the connection died, which is a question it fundamentally cannot answer. It just sends again.

Measured across 200 test submissions on deliberately unstable signal: 0% data loss, and 95% of syncs completed without anyone touching the app. The remaining 5% finished on the next launch.

Photos on their own path

Photos are a different problem from form data. They are large, they are slow, and losing one matters less than losing the survey it belongs to.

So they sync on a separate path with their own queue, compressed hard enough to move on 2G. A photo is deleted from the device only after the server confirms it has it, which means a phone that runs out of storage degrades by refusing new photos rather than by silently dropping ones already taken.

Two binaries, so the panel can fail alone

The API and the admin panel are separate binaries against the same database. It would have been less work to serve both from one process.

The reason not to: the admin panel is where an unbounded query gets written. Someone filters a dashboard across a date range nobody tested and pins the process. If that process is also the one accepting data from the field, an admin’s bad afternoon becomes a day of lost submissions.

Separating them means the panel can fall over and intake keeps running. The enumerators never find out.

The stack, and why

Self-hosted throughout. The client’s data stays on infrastructure they control, which for survey work covering named businesses was not a negotiable point.

Four weeks, alone

Product design, both apps, the API, the database, the infrastructure and the deployment. The schedule is the reason some of the decisions above look austere.

There is no realtime collaboration, no conflict resolution UI, no partial-record merge. Two enumerators editing the same record is not a case Titen handles, because in this deployment it cannot happen: records belong to the person who created them, and that constraint bought me a week.

I would make the same call again. Most offline-first complexity is paid to support conflicts that the domain does not actually produce.

Honest status

Titen is live and in use at titen-app.my.id. The reliability numbers above come from 200 test submissions on throttled and interrupted connections, not from a year of field telemetry. They are a measurement, not a track record.

What is not there yet: no multi-device editing of one record, no offline map tiles, and the retry backoff is a fixed curve rather than adapting to the connection it observes.

What I took from it

The idempotency key is the smallest decision in the project and it carried the most weight. Everything else — the outbox, the backoff, the separate photo path — exists to be allowed to fail safely, and all of it only works because a repeated request is harmless.

It is also the pattern I keep reaching for since. Any integration that can be retried, and every integration can be retried, wants an identity for the operation rather than an identity for the record. It costs one column and it removes a class of bug you would otherwise find in production, months later, in a spreadsheet that does not add up.