Engineering Fundamentals

Summary

A cluster of links on the bedrock of software engineering — the problems that don’t go away regardless of AI tooling. Idempotency, build caching, code reading, formal modeling, and now: structural correctness as a design philosophy for AI-generated code. W21 added a significant new thread via the backpressure piece — the argument that invariants belong in the substrate (types, compilers, tests, proof checkers), not in prompts. This directly connects to the formal methods work (TLA+) and distributed systems testing methodology. The theme emerging: the engineering fundamentals that make human-written systems reliable are the same ones that keep AI-generated systems correct. W22 adds two more pieces: platform capture of communication channels (push notifications are no longer developer-controlled delivery — Apple and Google have inserted AI intermediaries that summarize, reorder, and rewrite before delivery) and database-native durable workflows (DBOS argues that external orchestrators like Temporal are a needless abstraction — Postgres itself can coordinate workflows via standard locking and integrity constraints, making scalability and availability DB engineering problems with decades of prior art).

Key Sources

W22 2026 · 23-May-26 → 29-May-26

• Postgres-backed Durable Workflow Execution | DBOS — external orchestrators (Temporal, Airflow, Step Functions) are structurally overcomplicated; Postgres itself can serve as orchestrator — workers dequeue from a workflows table, checkpoint step outputs, detect duplicates via DB integrity constraints; hard problems like scalability and availability become well-understood DB engineering problems; eliminates a separate orchestration tier (engineering-blog · #durable-workflows, #postgres, #distributed-systems, #orchestration)
• What Apple and Google are doing to your push notifications — deep history of how APNs and FCM became active intermediaries: from permissive delivery (2009–2017) to AI-powered on-device summarization, reordering, and rewriting; developers have lost direct control over their own communication channel — platform intermediation is now the default, not the exception (engineering-blog · #push-notifications, #platform-control, #mobile, #distributed-systems)
• What I’m Hearing About Cognitive Debt (So Far) — Margaret-Anne Storey synthesizes practitioner responses to her cognitive debt concept: the gap between a system’s evolving structure and the team’s shared understanding of how and why it works; key distinction — technical debt lives in code, cognitive debt lives in people; manifests as confidence loss, heavier review burden, debugging friction, slower onboarding, fatigue; Fowler agrees it must be repaid; repayment means restoring the distributed theory of the system across people, docs, tests, tooling (opinion · #cognitive-debt, #ai-coding, #code-comprehension, #developer-wellbeing)
• Staged publishing and new install-time controls for npm — npm CLI 11.15.0: staged publishing now GA — packages go to a queue requiring maintainer approval before becoming installable; new --allow-file / --allow-remote / --allow-directory install source flags; direct supply-chain security improvements for the npm ecosystem (news · #npm, #supply-chain-security, #package-management)
• How Shamir’s Secret Sharing Works — Ente’s visual explainer: threshold secret sharing using polynomial interpolation; two points fix a line, so two shares recover the secret; any k-of-n shares reconstruct; fewer than k shares reveal literally nothing (not just “hard to crack”); Shamir (the S in RSA) published this in 1979; excellent primer for practical use in key management, recovery flows, and distributed trust systems (engineering-blog · #cryptography, #secret-sharing, #security, #distributed-trust)
• How Shamir’s Secret Sharing Works — Hacker News — HN discussion; community adds real-world applications: hardware security modules, cold wallet recovery, team key custody; discussion of finite field arithmetic for exact (not floating-point) computation (hn-thread · #cryptography, #security)
• White Rabbit — CERN/OHWR open hardware project: sub-nanosecond accuracy and picosecond precision synchronization for large distributed systems over Ethernet (gigabit, deterministic delivery); connects thousands of nodes over typical 10km distances; fully open hardware, firmware, and software; commercially produced multi-vendor; a remarkable case of open hardware enabling science-grade distributed systems (tool · #open-hardware, #distributed-systems, #timing, #networking, #cern)
• A Circuit Prompt Programming Language (CPPL) — paper: compiler-mediated framework that turns LLM-assisted hardware generation into a statically checkable problem; extends the structural correctness principle into RTL/hardware design; another data point that structural gates (compilers, type systems) + LLMs is the correct architecture (paper · #formal-methods, #hardware-design, #llm-coding, #structural-correctness)

W21 2026 · 16-May-26 → 22-May-26

• Intro to TLA+ for the LLM Era — practical guide: LLMs now write TLA+ specs, removing the syntax barrier; the hard parts — understanding your system and defining temporal correctness — remain human work; includes beans puzzle walkthrough and Claude prompt template
• shenli/distributed-system-testing — SKILL.md skills for claim-driven distributed/stateful system testing; covers partial partitions, crash-recovery, idempotency under replay, concurrency; produces structured test plan + blame classification (SUT/harness/checker/environment); emphasizes coverage adequacy as a deliverable
• Structural Backpressure Beats Smarter Agents — key primitive: move invariants out of prompts into the substrate; structural gates (compiler, type checker, test runner, linter, proof checker) produce concrete answers the model must satisfy; the refusal is the point
• Remote Cache CDC: Reusing Bytes — BuildBuddy’s Content-Defined Chunking in Bazel remote cache; 40% less data uploaded, 40% smaller disk cache; moves build caching from “skip actions” toward “skip bytes”; merged into Bazel 8.7/9.1+
• Content-defined chunking added to Bazel — HN thread on the Bazel CDC merge; community discussion on implications for large monorepos
• Reading code instead of writing code: The underestimated senior discipline — code reading is the defining senior skill; LLMs tip the balance further: they generate code faster than humans can reason about it

W20 2026 · 09-May-26 → 15-May-26

• Can LLMs model real-world systems in TLA+? — ACM SIGOPS research on whether LLMs can produce useful TLA+ specs for distributed systems; explores the formal verification + language model intersection; empirical evaluation of correctness
• Idempotency Is Easy Until the Second Request Is Different — the “happy path” idempotency is not the hard part; the hard part: same key, different content, concurrent retry, partial side-effect; strong take: same key + different canonical command should be a hard error

W19 2026 · 02-May-26 → 08-May-26

• lsferreira42/nfsdiag — NFS diagnostic/doctor application; pragmatic tool for NFS troubleshooting

Open Questions / Tensions

• LLMs and formal reasoning: The ACM SIGOPS TLA+ piece and the emptysqua.re intro raise complementary questions — can LLMs produce syntactically valid TLA+ (yes) vs semantically correct TLA+ for real distributed systems (unclear). The backpressure piece implicitly argues that only machine-checkable structural gates close this gap reliably.
• Structural backpressure scope: Brooks’ argument is compelling for a specific class of invariants (access control, type safety). It’s less clear how it applies to emergent properties (performance, scalability, user experience) that don’t reduce to machine-checkable predicates.
• Distributed systems testing as an agent skill: shenli’s approach — encapsulating testing methodology in SKILL.md files for agents — is a new pattern. It suggests agents can carry methodology, not just execute code. The quality of the methodology matters enormously.
• Build caching and AI codegen: CDC in Bazel is motivated by “small changes, large artifacts.” AI-generated code may change artifacts in ways that disrupt chunking locality — an unexamined downstream effect.
• Code reading as a teachable skill: Connecting to Osmani’s “don’t outsource learning” — if AI writes more and reading remains undervalued, engineers may become increasingly unable to audit their own codebases.
• Postgres as orchestrator vs. purpose-built orchestration: The DBOS argument is elegant but has trade-offs — a Postgres-backed system may hit limits under very high workflow fan-out or when workflows need advanced scheduling semantics. The question is whether Postgres ergonomics are better than Temporal’s complexity — the answer likely depends on team maturity and existing DB expertise.
• Platform capture of communication channels: The push notification piece documents a structural shift: Apple and Google now insert AI processing between developer intent and user experience. This pattern (platform as active intermediary) applies beyond push — email (Google/Yahoo filtering), search ranking, app store review. Engineers building on platform channels should assume the platform may rewrite, summarize, or suppress without notice.