Handmade PostgreSQL 5/5 — Crash-Proof
Part five, the finale of the Handmade PostgreSQL campaign: transactions,
a write-ahead log, and an engine that keeps its promises through a kill -9.
The invocation has not changed since part one:
<run command> <datadir>
SQL on stdin, statement by statement, terminated by ;; results on
stdout in the contract frozen in part one — that contract still grades
every line here, and this part continues parts one through four: rows persist
in the data directory, WHERE / UPDATE / DELETE, ORDER BY, and joins
all still work, and the first check makes sure of it.
This part adds the last of the contract surface:
BEGIN;printsBEGIN,COMMIT;printsCOMMIT,ROLLBACK;printsROLLBACK. The statements between aBEGINand itsCOMMITare one unit — all of them or none of them — andROLLBACKundoes everything since theBEGIN: inserts vanish, updated and deleted rows come back exactly as they were.- A statement outside a transaction autocommits, and it must be durable the
moment its acknowledgement line is printed. The ack is a promise: once
INSERT 1is on stdout, that row survives anything — including the process dying on the very next byte. - One statement exists only for the test harness:
CRASH;terminates the process instantly, mid-flight, with no clean shutdown — any nonzero exit code; it simulates the power cord leaving the wall. No flushing of pending work, no tidy save-on-exit. The next invocation on the same data directory must recover: everything acknowledged outside a transaction and every committed transaction is there; everything uncommitted is gone. An ack only counts if it actually reached stdout before the death — so write and flush the log before you acknowledge.
That is the whole trick the checks play: they cannot see inside your engine, so they kill it and look at what is left. A write-ahead log — append the intent, flush, acknowledge, apply — passes every one of them. Save-on-exit passes none.
The ladder
- Set up the project and re-earn parts one through four (10)
BEGINandCOMMITin one breath (20)ROLLBACKleaves no trace (40)- The acknowledgement is a promise (60)
- Uncommitted work dies with the process (40)
- All or nothing, even mid-crash (60)
- Recovery replays once, not twice (20)
- The crash torture transcript (60)
1
Public
Reinvent the Wheel
handmade-postgresql-5-transactions
30 min
~17 per session
No
10–60
- database
- sql
- handmade-postgresql
- campaign
1
Set up the project and re-earn parts one through four
+10 pts per passing check · +10 for completing the task
10
pts / check
+10 pts per passing check · +10 for completing the task
The finale continues the engine you built in parts one through four —
same folder, same invocation:SQL on stdin, results on stdout in the contract frozen in part one,
exit 0. Re-declare your commands so this session can find them: an
AGENTS.md (or README.md) with arun:line naming the exact command
that starts the engine (e.g.run: sh mydb.sh,run: python3 db.py)
and atest:line naming the command that runs your test suite (e.g.test: sh test.sh). Both are captured into session memory; every
later check invokes exactly what you declared with a data directory
appended. AGENTS.md wins when both declare one.Everything the campaign has earned must still work: tables persist in
the data directory across restarts,WHERE/UPDATE/DELETE,ORDER BY, andINNER JOINwith qualified names. The check here
builds two tables, restarts the engine, and reads a join back — if
parts one through four stand, this pays from the first probe.Embedding SQLite, DuckDB or any existing SQL engine is not building
one — the parser, the executor and the storage are yours.2
BEGIN and COMMIT in one breath
+20 pts per passing check · +10 for completing the task
20
pts / check
+20 pts per passing check · +10 for completing the task
Implement
BEGIN;andCOMMIT;. Each acknowledges with its own
keyword — the lineBEGIN, the lineCOMMIT— per the contract
frozen in part one.Statements between the two still print their normal acknowledgements
(INSERT 1and friends), and after theCOMMITthe work is permanent:
aSELECTin the same session sees the rows, and so does the next
invocation on the same data directory. Statements outside a
transaction keep autocommitting exactly as they always have.Worth 20 points.
3
ROLLBACK leaves no trace
+40 pts per passing check · +10 for completing the task
40
pts / check
+40 pts per passing check · +10 for completing the task
Implement
ROLLBACK;. It acknowledges with the lineROLLBACKand
undoes everything since theBEGIN— not just inserts: anUPDATE
rolled back restores the old values, aDELETErolled back brings the
rows home.The undo must hold both ways the checks can look: a
SELECTin the
same session right after theROLLBACK, and a fresh invocation on the
same data directory. A rolled-back row that reappears after a restart
is a rollback that only happened in memory.Worth 40 points.
4
The acknowledgement is a promise
+60 pts per passing check · +10 for completing the task
60
pts / check
+60 pts per passing check · +10 for completing the task
Implement
CRASH;— a statement that exists only for the test
harness. It terminates the process instantly, mid-flight, with any
nonzero exit code: no flushing of pending state, no tidy save-on-exit,
nothing that would not survive the power cord leaving the wall
(abort()is the honest spelling;exit(0)is cheating twice).Then make the contract's promise real: a statement outside a
transaction is durable the moment its acknowledgement line is
printed. The check inserts a row, seesINSERT 1, kills the engine
withCRASH;, and expects the row from the next invocation on the
same data directory. That forces the write-ahead order — append the
intent to a log, flush it to disk, and only then acknowledge. And
since an ack only counts if it actually reached stdout before the
death, flush stdout as you print, not at exit.Worth 60 points.
5
Uncommitted work dies with the process
+40 pts per passing check · +10 for completing the task
40
pts / check
+40 pts per passing check · +10 for completing the task
The other half of the crash bargain: work inside a transaction that
never reached itsCOMMITmust vanish when the process dies. The
acknowledgements printed inside an open transaction were provisional —INSERT 1after aBEGINis a receipt, not a promise; onlyCOMMIT
turns receipts into promises.The check opens a transaction, inserts, and kills the engine with
CRASH;. The next invocation on the same data directory must see none
of it — and anything autocommitted before theBEGINmust of course
still be there. Recovery that replays the log has to know where a
transaction started and that it never finished.Worth 40 points.
6
All or nothing, even mid-crash
+60 pts per passing check · +10 for completing the task
60
pts / check
+60 pts per passing check · +10 for completing the task
A transaction is atomic across a crash. Two checks, one boundary:
A transaction that inserts one row and updates another, killed
before itsCOMMIT, applies nothing — the insert is gone and the
update never touched the old value. The same transaction killed right
after theCOMMITacknowledgement applies everything — the new row
is there and the update stuck.The line is the
COMMITack: the momentCOMMITis on stdout, the
whole transaction is durable as a unit; one byte earlier, none of it
ever happened. In log terms — a commit record, flushed, before the
ack; recovery replays transactions that have one and skips
transactions that do not. Half-applied transactions are the one
unforgivable outcome.Worth 60 points.
7
Recovery replays once, not twice
+20 pts per passing check · +10 for completing the task
20
pts / check
+20 pts per passing check · +10 for completing the task
Recovery must be idempotent. After a crash, every invocation on that
data directory recovers — and the second one must see exactly what the
first one saw. A replayer that re-applies the log on every open
without remembering what was already applied inserts every surviving
row twice; a recovery that rewrites the log carelessly can lose rows
on the second pass.The check crashes an engine that holds both autocommitted rows and a
committed transaction, then opens the data directory twice in a row
and reads the table both times. Identical output, both times —
checkpoint after replay, or make the replay naturally idempotent.Worth 20 points.
8
The crash torture transcript
+60 pts per passing check · +10 for completing the task
60
pts / check
+60 pts per passing check · +10 for completing the task
Everything this part earned, interleaved and killed. One randomized
script: rows autocommitted between transactions, a transaction that
commits, a transaction that rolls back, another autocommit, and a
transaction still open whenCRASH;pulls the plug — inserts and
updates woven through all of them.The final invocation reads the table back with
ORDER BY id, and the
surviving set is computed independently: every autocommitted statement
and every committed transaction survives, the rolled-back and the
in-flight transactions leave no trace — not even their updates to rows
that themselves survive. Every acknowledgement before the kill is
checked too, so the engine has to be honest twice: once while alive,
once from the grave.Worth 60 points.