S3E9: Testing Policy Debt

Chris and Nick talk about how organizational growth introduces process inefficiencies and inter-departmental friction, which can be diagnosed and addressed using value stream mapping. They emphasize that automating systems successfully relies on robust exception handling rather than just building for the "happy path," sharing a banking case study where classifying and systematically resolving exception classes (via a system dubbed "X-Man") drastically improved efficiency. This macro-systems view is compared to code testing, where they advocate for a structured test hierarchy. They highlight the high cost of slow end-to-end tests and stress the importance of maintaining a fast, ten-minute feedback loop dominated by cheap, local unit and functional tests to prevent developer context from being lost.

They also preview the concept of "policy debt," illustrating it through recent updates to NIST's password guidelines. They discuss how legacy organizations often remain trapped under outdated and cumbersome rules—such as mandatory 90-day password rotation, because they lack the institutional memory or documentation to explain why these policies were created in the first place. Nick shares a past success using modern guidelines to negotiate policy changes with a regulator, contrasting it with larger, older organizations that struggle to unpick legacy commitments. The episode concludes with a historical anecdote about Chester Carlson’s invention of the Xerox machine and the creation of "key operators" to manage paper and toner, highlighting that scaling friction and operational adaptations are universal challenges for any growing technology.