Letting a model do bookkeeping without letting it do damage
A conversational platform that performs real accounting operations — building a chart of accounts, classifying transactions, raising invoices and bills, reconciling against bank statements — rather than just answering questions about them.
Problem
Small business owners need bookkeeping done correctly but cannot justify a full-time bookkeeper, and general ledger software assumes accounting knowledge they do not have.
The useful version of this product is not a chatbot that explains accounting. It is a system that performs the work: generating a chart of accounts for the specific business, classifying incoming transactions, raising invoices and bills, tracking fixed assets and amortization, and reconciling records against bank statements.
That means the language model is not producing text for a human to read. It is producing operations against financial records.
Constraints
- Correctness is not a quality metric here, it is the product. A misclassified transaction is a real accounting error with downstream tax consequences.
- Bank statements arrive as PDFs with no consistent layout — every institution formats differently, and some are scans rather than text.
- Transaction categorization is genuinely ambiguous. The same merchant can be a legitimate expense or an owner's draw depending on context the model does not have.
- Users are not accountants, so the system cannot resolve ambiguity by asking a question the user cannot answer.
- Conversations are long-running and stateful. Onboarding, account setup and reconciliation happen across sessions.
My role
Primary engineer. I designed the orchestration layer, the service dispatch model, the write-safety mechanism and the extraction pipeline, and wrote the majority of the accounting domain logic.
Architecture
A phase-based orchestrator moves a business through onboarding, chart-of-accounts generation and confirmation, transaction upload, reconciliation setup, and ongoing advisory. Each phase constrains which operations are available, so the system cannot attempt reconciliation before accounts exist.
A service dispatcher exposes the accounting operations as a catalog the model selects from. The model emits a structured call; the dispatcher validates it, executes it, and returns a result the model can respond to.
State lives in PostgreSQL accessed asynchronously, with Redis holding conversation history and cached summaries so long sessions do not re-read everything on each turn.
Request correlation IDs run through every layer, and a circuit breaker isolates failures in the external model providers from the rest of the system.
Key decisions
Confirm-then-commit instead of direct writes
The obvious design gives the model tools that write to the database. I did not do that. Instead every mutation the model proposes becomes a pending action — a described, structured, reversible intent that is held until a human explicitly confirms it. Only confirmation executes it. This costs a round trip and some conversational friction, and it is worth it: the failure mode of a bad LLM call becomes a rejected suggestion rather than a corrupted ledger. The general principle I now apply elsewhere is that a model may propose anything and commit nothing.
A second model validates the first
Chart-of-accounts generation is the highest-leverage step in onboarding — everything downstream inherits its mistakes — and it is also the step where the model has the most freedom. So generation and validation are split across two different providers: one model generates the structure, a second independently reviews it against the business profile before a human ever sees it. Disagreement between them is a useful signal in itself. Using the same model twice would mostly reproduce the same blind spots.
Deterministic code wherever the answer is checkable
Transfers between a user's own accounts are matched by rules, not by the model, because that relationship is verifiable from amounts and dates. Semantic matching is reserved for cases where the text genuinely has to be interpreted. Every model call in the pipeline that could be a deterministic check instead is one.
Batch classification with a rule-based fallback
Transaction classification runs concurrently across many statement lines. Concurrency makes provider rate limits and transient failures normal rather than exceptional, so a deterministic fallback path handles anything the model pass does not return cleanly. The system degrades to worse categorization rather than to no categorization.
Result
The platform performs the full bookkeeping loop conversationally: onboarding a business, generating and confirming its chart of accounts, ingesting and classifying transactions, managing invoices and bills, and reconciling against uploaded statements.
The design goal that mattered most was met — there is no path by which a single bad model response silently changes a customer's financial records.
What I learned
- The interesting engineering in an LLM product is usually not the prompt. It is the set of things you decide the model is not allowed to do on its own.
- Human review is a design decision, not a fallback. Choosing review points by cost-of-error produces a better system than gating on model confidence, which is not well calibrated for this kind of task anyway.