How a law becomes the law
You've probably heard that our government is bad at using technology. And, depending on your political worldview, you probably think that means one of two things: it should outsource more to technological specialists, or it should spend more to modernize its own technology.
Jennifer Pahlka, the founder of Code for America and a member of the Obama administration's digital team, agrees that our government is bad at using technology. But in Recoding America, she says neither of those solutions is enough. The government already outsources most of its digital work, which hasn't solved the problem (for example, the failed healthcare.gov effort was contracted to the government's primary outsourcing partner). And while it's true that government systems are older than most private-sector peers, modernization efforts take forever and don't really get results (for example, when Congress expanded unemployment insurance at the start of the pandemic, the half of states that had modernized their UI systems handled it just as poorly as the half that hadn't).
The fundamental problem, she says, is that digital efforts are hamstrung by the government's organizational structure: who does what work and how they do it. And this problem persists because of misplaced priorities that go way beyond technology—citizens, and in turn lawmakers, don't care enough about making sure that laws work in practice.
Why hierarchy fails
Government technology projects are run by a strict hierarchy: Congress passes laws telling an agency to do something; the agency's civil servants parse that high-level direction into a set of project requirements; and then they get bids for contractors to make something that follows those requirements.1 In tech companies, this is called a "waterfall" process: each phase is done one at a time (ideation, planning out requirements, making the product, testing if it works) and there's no going backward.
The problem: nobody is responsible for making sure the outcomes actually make sense. Lawmakers only want to think about "policy", the prestigious, big-picture stuff; they think implementation is beneath them and stay out of it unless something catastrophic happens. The contractors at the bottom don't have any power to question things, so they just fulfill requirements. And the people in the middle have agency lawyers yelling at them if they do anything the least bit creative; the safest route is to think of every possible requirement and include it.
So products become bloated and complex. That includes external products: a California web form to apply for SNAP benefits ballooned to 212 mandatory questions. And it includes internal products: the state's unemployment insurance claims are processed with a patchwork of overlapping systems that spans six decades of programming languages and require years of training to operate.
This is bad for users: not only are complex products more delayed and error-prone, but even when they work exactly as intended, they’re confusing and time-consuming to use. But in a waterfall process, you can’t go back and change the requirements as you learn from users—which is why nearly all private-sector software companies have abandoned waterfall processes in favor of more iterative workflows.2
Hierarchy and its downsides exist everywhere in government, but they're much costlier for digital projects. That's partly because poilcymakers are less likely to understand technological details, so they're more likely to require things that are dumb or impossible.3 But it's mostly because it kneecaps the exact feature that has made digital technology so powerful—the ability to learn and adapt quickly. It’s easy to gather lots of data on how the product is or isn’t used, and code can be changed much more quickly than a physical product can.
And digital projects increasingly aren’t just for laws about technology; they’re for any law that involves submitting or storing data, which is practically everything.
Organizational structure shows why "outsource more" or "modernize technology" don't work:
When you think of outsourcing, you probably think of giving it to one of the giant consumer-tech companies whose products dominate our lives. But by and large the government isn't outsourcing to those companies, in part because they don't want to work in a waterfall, and they wouldn’t be good at it anyway since learning and iteration is key to their success. Instead, the government chooses companies that specialize in checking off boxes. More outsourcing and oversight would just make the problems worse: more requirements would have to be formalized up-front, and more work would be done by companies whose core competency is winning government contracts instead of making products work well for users.
When you think of modernizing technology, you probably think of replacing old software languages with newer ones. That is indeed a problem on the margin,4 but the real issue is that ballooning requirements make systems so complex that they're impossible to replace, because nobody even understands how it all works. (To be fair, this is partly because the law itself is similarly complex, since new laws build on top of old laws and are being added all the time—the tax code changed an average of once per day from 2001-13.) Modernization not explicitly aimed at reducing this complexity won't help, and it may make things worse: a high-cost modernization project comes with its own up-front requirements, which take years to deliver, and might be out of date by the time they’re finished.
Praising product management
What's the solution? To over-simplify Pahlka’s answer, it's to bring product management into government. The job of a "product manager" is relatively new and, at least in my experience, poorly understood.5 It involves figuring out what users need (sometimes by observation, sometimes with data), defining how to change the product to address them, and helping resolve any questions that arise (especially between different departments). A lot of this work is invisible, so it sometimes gets a bad rap; but it’s also valuable, so basically every major tech company now has a large product management function.
In government, a product manager (or someone acting like one) would crucially have the power to set priorities—what should be built and published first, what can be followed up on later, what doesn't need to happen at all, and what new things should happen based on user insights. This is harder in government than it is in the private sector, because everything must always follow the letter and spirit of Congress' laws. But that doesn’t mean it can’t be done: many requirements aren't actually prescribed by law; they're just based on recommendations, precedent, or legal interpretations. In a hierarchical organization, these are still fulfilled even when they don’t make sense, because following orders is easier and safer for your career.
The government’s natural tendency of requirement creep makes prioritization all the more necessary. For example, a 2015 law changing Medicare reimbursement also instructed that small medical practices be allowed to file paperwork together, The overseeing agency's lawyers interpreted it to mean that it also had to create a new social network for doctors to find each other too. An enterprising member of the government's in-house digital team, putting on her product manager hat, surveyed doctors and found that none of them would ever use such a network. Then she asked the bill's original authors if they'd intended that part, and they had no idea what the hell she was talking about. Then she asked the industry groups that had lobbied for that clause, and they hadn't wanted anything of the sort, it was just mistranslated. Even after all that, the requirement was going to stand until she found a lawyer senior and flexible enough to kill it.
What else can be done to stop the waterfall? Hiring contractors that work iteratively, bringing tech-literate people into the lawmaking process, and providing more data on how products are being used by real people. One of Pahlka's most interesting ideas (though only noted in passing) is staged funding, like venture capital-backed startups have. Instead of appropriating hundreds of millions of dollars over years for a website that does everything, appropriate tens of million dollars over months for a website that does one thing well. If that gets good traction and there's demand for more features, go ahead and spend more only then.
Beyond Schoolhouse Rock
I’d be into a book that's just about government technology and org structure, but you probably wouldn’t be. Recoding America is really important as an example of a larger theme: the importance of engaging with what laws actually do.
Pahlka opens with a reference to the "how a bill becomes a law" video we all watched in elementary school, because that's the extent of most people's understanding of how policy works. That probably gives us too much credit; the dominant form of policy engagement today is often "shout louder on Twitter and hope that convinces people to get on your side."6 People certainly don't pay attention to who fills in the details of laws and makes them happen, which means lawmakers don't care about it either, which means processes remain broken.
This means that laws don't always accomplish what they're supposed to.
From the book: When California legalized marijuana, it also expunged prior marijuana convictions. But people had to file their own petitions for expungement, which involved requesting materials from several departments. This was so onerous that nobody really tried—in San Francisco, fewer than two dozen—so that aspect of the law, passed with good intentions to help people still suffering consequences for doing something that was no longer a crime, did nothing.
From the news: The 2021 infrastructure law included billions of dollars to install electric vehicle chargers around the country, one of several pro-climate provisions that progressives prioritized in the bipartisan bill. But more than two years later, zero chargers have been installed, in part because it's hard to follow the Federal Highway Administration's complex set of guidelines, and in part because implementation was delegated to state offices that had no experience with electric vehicles.
If you're on the left, you might complain that the real issue is conservatives deliberately starving the government of resources and capacity. (For example, Republicans intentionally make tax filing user-unfriendly, and they're at best indifferent to the burden that complex eligibility requirements place on people in need of welfare.) Pahlka brushes past this topic as quickly as possible, which is intellectually frustrating but strategically understandable; she's doing her best to build a broad coalition for a position that's otherwise fairly progressive.
She's not wrong, however, that liberals are also to blame—particularly their reliance on lawsuits and review processes that let small groups stop or slow down projects that would benefit many. This is the factor that makes government agencies defer to their lawyers, which means more requirements and less flexibility, with the regressive result that it's harder for needy people to actually get the benefits they’re entitled to. (From the book: one agency declared that online and paper forms had to be exactly identical lest they get equity complaints; that meant the widely used digital forms were annoying and hard to complete, such as by asking questions that were already invalidated by previous answers.) And it's why public programs are slow and costly, with the conservative result that less gets done. (From the news: "environmental review" processes are frequently used to stall green energy projects.)
The alternative, empowering agencies to act more autonomously and directly, involves uncomfortable trade-offs. But that's the whole point: you can ignore those trade-offs at the level of shouting, or even at the level of writing laws, but not at the level of implementation. To have a government that can serve citizens more effectively (through technology or not), policymakers have to work through those details—and we have to want them to.
Like most of the book, this characterization is specific to the US; I'm not yet sure how much of it translates to other countries.
Companies that build technology in the physical world still generally use waterfall systems, because it's much harder to iterate on atoms than on code. But even that may be changing: SpaceX develops its rockets iteratively, giving it a massive speed advantage over Boeing and other incumbents.
In their defense, the line between what's easy and what's impossible in technology is often inscrutable. Pahlka's example: automatically clearing marijuana possession convictions was easy, but automatically clearing petty larceny convictions would have required data that had never been collected.
From the book: at the Treasury department (which includes the IRS), 63% of IT employees are over 50 years old—key systems use an out-of-fashion assembly language, which limits their talent pool.
It took me like a year of working in tech to learn that a “product manager” is distinct from a "project manager" who makes sure timelines are set and met, and another year or so to get a handle on what it actually means.
I wonder how much of the "shout louder" ethos is fighting the last war from gay marriage, which was actually achieved by just winning over public opinion broadly. Ironically, the gay rights movement itself didn't realize this at first, instead seeking technocratic change through under-the-radar state legislatures—because that's how its own last war, interracial marriage, was won.