These are bits and pieces. Not fully formed thoughts. But I want to quote two smart people. Steve Yegge 2 years ago. And Dan Hon a few days ago.
From Steve Yegge, 2023: Cheating is all you need:
All you crazy MFs are completely overlooking the fact that software engineering exists as a discipline because you cannot EVER under any circumstances TRUST CODE. That’s why we have reviewers. And linters. And debuggers. And unit tests. And integration tests. And staging environments. And runbooks. And all of goddamned Operational Excellence. And security checkers, and compliance scanners, and on, and on and on!
So the next one of you to complain that “you can’t trust LLM code” gets a little badge that says “Welcome to engineering motherfucker”. You’ve finally learned the secret of the trade: Don’t. Trust. Anything!
I linked to it back then.
Mistrust is core to my own approach to code as well. Mistrust-Based Technology Choices (from a year ago).
Mulling how I think about code, computers, and… everything today.
I was listening to Dan Hon interviewed the other day. Rethinking Civic and Public Interest Technology with Dan Hon
Dan Hon (newsletter.danhon.com, company verylittlegravitas.com) is an expert on government and tec.
Mistrust looms large right now when I think about computers in government because for those of us in software the shenanigans of DOGE look very much like a software attack and strongly imply data exfiltration.
I’ll quote Dan, slightly out of context, please watch or listen to the whole video:
I saw someone describe this in a pretty good way is that if you know enough about the technology that runs government what what’s been happening over the last 3 days is a bit like the January 6th storming of the Capital–it is people going in and they are breaking shit, they are pulling plugs out, they are going through all the filing cabinets but these aren’t filing cabinets these are the Personnel records and the Social Security numbers and everything of 2 million federal employees.
And another quote:
I think the way to look at it at least from the technology point of view is — if the imagery of the 6th of January in the US — provokes a visceral response then the same thing is happening to government technology systems — and for the people who are doing that they know how to mess with other people’s technology systems as well.
So I don’t think it is too out of the ordinary to say — “What would you do if one of Musk’s people turned up–right–at your office?” or someone emboldened by what Musk is doing turned up at your office– or turned up at your data center– (for example if you’re significantly bigger) –or what if for example in this shitty environment that you’re in what if you get snitched on you know by someone in the community or by someone in the neighborhood?
I think one of the massive bright points is the ability to distribute information among the populace as easily and quickly as possible … I read reports about the head of ICE being very frustrated that people in Chicago all happen to know their rights right now and I’m like “oh right yeah that seemed to be distributed fairly well and fairly efficiently and fairly quickly–that’s great”
Later Dan summarizes this all as:
“What if there’s a bad actor inside the system?”
Whether that’s an LLM which created software to help you do things, or politically malevolent forces have hardware access to your systems, how do we create software we can trust?
