Chelsea Mauldin is the Public Policy Lab’s executive director and co-founder. 

Statement in Support of Claim

Imagine you’re a veteran of the U.S. military. During your past deployments, you had experiences with danger, violence, or death that have had a lasting effect on your wellbeing and your everyday activities. After years of encouragement from your family and buddies, you’ve decided to apply to the VA—the U.S. Department of Veterans Affairs—for disability compensation for the mental-health effects of those in-service traumatic events. Now you have to fill out a form to justify your claim by describing what you experienced. 

When the Public Policy Lab began a collaboration with the VA Center for Innovation (VACI) in 2015, that form (21-0781 Aug 2014) asked veterans to detail any “stressful incidents,” list any citations or medals they’d received because of the incidents, and also provide the name, rank, and date of injury or death of any servicepeople injured or killed during the incidents. A veteran told me it felt like he was being asked to capitalize on the death of his friends to get help. Other veterans and their family members told us how filling out the form triggered flashbacks and other mental-health crises. A PDF was a source of felt and moral harm.

Our team at PPL learned about Form 21-0781 while leading research on how the VA could best support veterans who were experiencing significant mental-health challenges. At the time, an average of 22 veterans per day were dying by suicide, and VACI leadership was committed to providing better help. Over the course of a year, we conducted in-person research with veterans and family members in five states; spoke with experts and frontline staff at VA hospitals, veteran-service organizations (VSOs), and academic institutions; and worked closely with a passionate, dedicated team at VA headquarters. We produced a set of recommendations across three timeframes: quick wins, pilot projects, and system transformations. (One of the ‘quick wins’ we proposed was to redesign Form 21-0781.) You can read the report here. 

Then the project was over. The veterans we spoke with had expressed appreciation for being heard. VA staff had engaged in a new kind of human-centered, design-driven inquiry. Our partners had received our work and promised to carry it forward. At that moment in 2016, what was the value of PPL’s work? And how would we demonstrate it?

Times, Values, Society, Bodies

“There is only one thing that seems discernible: we may say that radical evil has emerged in connection with a system in which all [people] have become equally superfluous.” ― Hannah Arendt, The Origins of Totalitarianism

This blog post is about measurement of value. This is not just an abstract question: Philanthropic funders, whom we rely on to cover a portion of our staff’s pay, sometimes ask that PPL make a quantitative case for a project’s future successful impact. We typically undersatisfy them. We can promise certain outputs (“deliverables” is the inelegant term of art), and we can (sometimes) estimate the potential number of users of those outputs. But I’m deeply reluctant for PPL to predict what outcome, what value those users interacting with those outputs will generate. 

What’s the problem? Why don’t we want to propose metrics, then measure them? 

First, the timelines. Neither philanthropies nor governments will generally tolerate the Public Policy Lab (or an external evaluator) spending five or ten years capturing the effects of our interventions before reporting outcomes. They want up-front statements about the expected scale of impact and timely reporting on completion. But the meaningful ripples from our work—or from any work to change a complex policy-delivery system—are unpredictable and take years to spread, a factor that econometricians refer to as ‘lag.’

Second, the preferred values. David Graeber reminded us that ‘value’ is slippery business—simultaneously meaning and measurement. I am not opposed to the collection of evidence—PPL’s teams dedicate significant resources to data collection. The request for metrics, however, is a mainly request for numbers about how much time or money will be saved or spent. I don’t believe that the utilitarian signifiers of market economics should be used to define public value in a democracy. But dignity, equality, and freedom do resist the easy quantification of minutes or dollars.  

Third, the attitude toward social reality—what legal scholar Megan Stevenson has called the “engineer’s view” of the structure of the social world: that social change can be technologically determined, tested, and measured. In fact, as Stevenson demonstrates, 50 years of randomized controlled trials of social-policy interventions shows this is largely a myth. Institutional processes resist change, effect cascades are rare, and context dependence limits replication. 

This is absolutely true of PPL’s work: Recall the personal, political, and material multiplicity of the public-service environments in which we operate, with hundreds of thousands or millions of staff and beneficiaries, each inhabiting their own circumstances. We conduct real-world pilots and formative evaluations of our work all the time; we see, recurrently, that the real social world resists any honest claims that PPL can engineer interventions that provably overcome reversion to status quo, perform identically in different environments, and generate predictable outcomes. Claims of that kind also run contrary to what, as human-centered practitioners, we observe and celebrate: the non-replicable particularity of each individual’s context-specific experience. 

Ultimately, standard approaches to measurement ask for aggregations of reality, abstracted from how that reality is lived in specific human bodies. But one veteran’s experience of Form 21-0781 is not an average veteran’s experience: It is that particular person’s regret or anger or grief. Those feelings are neither wrong nor superfluous—they’re accurate and meaningful, as well as variable, subjective, and temporary. That is the nature of human experience: We feel what we feel when we feel it. Until we’re replaced by robots, citizens of democracies must reasonably demand that definitions of successful policy delivery respect that the public is alive, in flesh. 

What 15 years of design practice in policy arenas has taught us is that the demand for stable, quantified, machine-readable outcomes is not a demand for truth—it is a values-based demand for a representation of reality that is legible to funders or administrators. This demand systematically distorts what happens when governance meets time and democratic values and social scale and a human body.

2,322,925 Unhappy Minutes Saved!

The win was not quick. In 2017 and 2021, the VA released new versions of Form 21-0781. Some design changes were made, but veterans were still asked to describe how they saw people killed or injured. Not until March 2024 did a further revision introduce new language and remove the request for descriptions of harm. The form also now acknowledges the difficulty of detailing trauma, saying, “While providing this information may be difficult, this information may help identify evidence to support your claim.”

I don’t know if PPL’s recommendations in 2016 influenced the changes made in 2024; certainly, we are not the only people who’ve worked with or at the VA who knew the old form was causing harm. And what no one knows—it is unknowable—is how much veterans suffered because of the form before 2024. Even how much the individual feelings of the veterans who’ve received the post-2024 form diverge from what they would have felt if given the old form, that also is not meaningfully quantifiable. 

I could craft a numbers story: the number of times that form has been submitted since it was redesigned, multiplied by the official estimate from the Office of Management and Budget of how many fewer minutes it takes to fill out the 2024 form (supposedly 25 minutes less than the 2014 version!), then tell you X millions of presumably unhappy minutes were saved. But that population-level measurement would not tell you anything about the effect of the design change on each person and their loved ones, which is what actually matters. 

There’s a further lagging impact of PPL’s engagement with the VA: how we helped change not just a form, but an agency. We’d previously worked with VACI as an advisor on the agency’s very first effort at applying human-centered design, a 2014 project that came to be known as Toward a Veteran-Centered VA. That led to the mental-health project, and as that work wrapped up in late 2015, we joined a start-up team of external consultants supporting the launch of the first dedicated ‘customer’ experience office in the federal government, the Veterans Experience Office (VEO), and the development of the VA Welcome Kit—efforts we captured in another public-access report.

Subsequently, the VEO’s efforts to improve service delivery have been tied to survey results showing increased veteran trust in the VA. In August 2025, almost exactly ten years after the VEO project began, the U.S. Congress passed the Improving Veterans’ Experience Act, which “requires that the VEO and a chief veterans’ experience officer be in place, that there are sufficient staff and resources necessary for CX initiatives, that the VA secretary be advised about the accuracy and helpfulness of consumer-facing VA materials… and about opportunities to improve the VA’s customer service,” as former VEO executive Lee Becker explains. 

I could pitch a story about how our pioneering nonprofit organization helped launch a revolution in the way the VA serves veterans, ultimately leading to federal legislation requiring an ongoing commitment to improving veterans’ experiences. But the real credit is much more distributed, and the real story is ongoing: Thousands of veterans have generously shared their experiences and insights. Thousands of professionals—both at the VA and outside organizations—have worked to create a more veteran-centered approach to policy delivery. That work is now felt, to an unknown and unknowable degree, by VA staff and millions of millions of veterans. That work continues and will never be done.

So too with PPL’s practice. Our job requires that we continuously develop our craft, and our craft is the design of policies and services that help people live more satisfying and meaningful lives. My hope is not that we become better at generating metrics over the next 15 years, but that we become ever more skillful at deploying thoughtful, human-centered attempts to deliver value.

As the Public Policy Lab marks its 15th year, we’re publishing a series of essays reflecting on the projects that shaped us and what they’ve taught us about public-interest innovation. Each post is paired with a related conversation, bringing practitioners together to explore the theme and its implications for the future of public systems.

Recommended Citation 

Chelsea Mauldin, “Measuring the Unmeasurable: Design, Democracy, and the Limits of Outcome Metrics,” New York, NY: Public Policy Lab, May 22, 2026, https://www.publicpolicylab.org/resources/measuring-the-unmeasurable/