The RCT Agenda

An excerpt from *Unbeatable*

Randomized Controlled Trials: Could you be any more scientific? The book I’m now writing, Unbeatable: The Brutally Honest Case for Free Markets, insists that the randomistas of the economics profession actually have a thinly-veiled political agenda. Namely: To get economists to humbly serve the demagogues that rule the world instead of bluntly challenging their unabated demagoguery.

Paranoid? I don’t think so. Show me a single published economics RCT that concludes, “Government should stop trying to fix this problem and just allow a free market.” No, no, no. They’ll always either find that (a) the treatment “worked” — so government should scale it up; or (b) the treatment “failed” — so government should keep running RCTs until they find a treatment that works. “Freedom” never counts as a “treatment.”

Still skeptical? Here’s an excerpt from “Chapter 1: The Hyperbole of Laissez-Faire: What Economists Really Think About Markets and Government.”

---

Cutting-Edge Research Versus Free-Market Economics

After explaining the theoretical trade-offs, textbooks admonish us to “find the right balance” between markets and government.  How are we supposed to do so?  With high-quality empirical research.  Where are we supposed to find such research?  In top economics journals, naturally.  When you read today’s top economics journals, however, the research seems largely divorced from this “find the right balance” mission.  Few of the articles even try to weigh the merits of markets versus government.  Instead, cutting-edge papers’ goal is to definitively measure the causal effect of one variable on another using Randomized Controlled Trials (RCTs).  If a randomly-selected person gets an extra year of education, how much does their income rise?[i]  If you randomly give someone in China uncensored internet access, how much do their political opinions shift?[ii]  If a randomly-selected neighborhood gets 10% more police, how much does crime fall?[iii]  If you offer a random family in Kenya an energy-efficient cookstove, how much will they pay for it?[iv]  Earlier generations of economists identified as Keynesian or monetarist, saltwater or freshwater, free-market or radical.  Today’s star researchers are more likely to brand themselves as “randomistas.”[v] 

Randomization is the heart of the experimental method.  If you get a pool of subjects and randomly split them into treatment and control groups, then the groups’ average outcome difference estimates the causal effect of the treatment.[vi]  That’s how the best medical studies work.  Vitamin use may correlate with better health — but is that genuinely causal?  Perhaps the health-conscious falsely believe that vitamins work, creating a spurious correlation.  Only experiments can resolve the issue. 

The randomistas of economics embrace this logic: Social experiments yield social knowledge; the rest is speculation.  Sometimes this means laboratory experiments, where you test your theories on students on campus.  Sometimes this means field experiments, where you test your theories on non-students off-campus.  And sometimes this means natural experiments, where you scavenge your data from plausibly random events outside your control.

Abhijit Banerjee and Esther Duflo, the husband and wife who won the 2019 Nobel prize, personify this RCT revolution in empirical economics.[vii]  Though their work focuses on Third World poverty, they don’t pretend to know any transformative policy reforms for the Third World or anywhere else.  Instead, they devote themselves to what Duflo calls “plumbing problems.”  As she explains in her Nobel address:

In plumbing problems, the government is not asking itself whether it should invest in health or education, or even in any particular intervention. Rather, it is asking a question of the form: “We are running this particular program and there are issues with it. What can we do to address these issues and achieve our objectives?”[viii]

You could interpret this as, “Some economists specialize in big questions; some economists specialize in small questions.  We do the latter, but both approaches are worthwhile.”  Yet in Banerjee’s Nobel address, he strongly implies otherwise:

[W]hile it is true that RCTs do not answer most of these kinds of “big questions,” most other methods do not either, except by assertion or by ignoring the many frailties of the answers they offer. In particular, cross-country comparisons that often purport to answer these kinds of questions tend to be grossly unreliable, to the point of being nearly worthless for policy purposes.[ix]

You call this “science”?

Why don’t randomistas try harder to make headway on the “markets versus government” question?  Part of the problem is practical: Researchers can’t treat countries like guinea pigs.  There is no way to get a sample of forty poor countries, randomly assign some to laissez-faire and others to socialism, then measure their prosperity gap ten years later.  You can’t even get a sample of two countries to submit to such experimentation.  True, you could call West Germany versus East Germany, North Korea versus South Korea, or mainland China versus Hong Kong “natural experiments on markets versus government.”  But top journals these days will never let you get away with that.  The sample is too small, the randomization too questionable, and the question too big.  Way too big.

Still, the fundamental reason why randomistas make so little headway on “markets versus government” is that they quietly reject the question!  Instead of weighing markets versus government, they almost invariably weigh government versus government.  As a result, government can never really lose. 

To see what I mean, suppose an RCT tests whether smaller class sizes increase adult earnings.  If researchers discover a sizable positive effect, the heralded “policy implication” will be: Spend more taxpayer money to reduce class sizes.  If researchers find a small or negative effect, however, the policy implication is never: Cut school budgets and refund the savings to taxpayers.  Instead, the lesson the researchers will draw is: We desperately need experiments on other ways schools might increase adult earnings.  Maybe we should do an experiment with more school lunches.  Maybe we should randomly raise teachers’ pay.  Or perhaps do an RCT on afterschool programs.  No matter how many experiments fail, the Executive Summary will never read, “Let’s defund public education.”

If an experimenter did write such an Executive Summary, his peers would almost surely fault him for being “ideological.”  To use a colleague’s catchphrase: “This is not science.”  Yet how is asking, “Should government spend money on this goal?” any more ideological than asking, “How should government spend money on this goal?”  In effect, randomistas call fellow economists “ideological” for making inquiries with potentially ugly answers, instead of sticking to crowd-pleasing loaded questions like, “What’s the best way for government to help?”

One of the most pressing of these questions with potentially ugly answers, to repeat, is “What is the right balance between free markets versus government”?  On the surface, randomistas just stop engaging this classic issue — and pressure other economists to do the same.  Yet given laymen’s resentment of free markets, expert silence is effectively a forfeit in favor of government.  If the public disdains free markets, and economists never mention that the public’s position is debatable, disdain wins. 

Deep down, though, the randomistas are happy to forfeit in favor of government.  Most don’t push silence on “markets versus government” because economists lack the tools to answer.  They push silence because they think the question is settled.  Once, like Duflo, you reduce empirical economics to “plumbing” for the government, free-market economics is tantamount to a plumber saying, “Maybe your house doesn’t need water.” 

If you oppose free-market policies, you’re might be pleased that cutting-edge researchers have deleted them from the academia agenda.  Why waste Ivy League brainpower on intellectual dead ends?  While the rest of the book tries to reverse opposition to free-market policies, my focus here remains the current state of the economics profession.  For better or worse, cutting-edge economic research is definitely not pro-market.  Indeed, a total victory for RCTs would be the end of free-market economics.  If you can’t have a serious discussion of free-market policies in an economics department, where else is there to go?

---

[i] See The Case Against Education, pp.76-9.

[ii] https://www.jstor.org/stable/26737888

[iii] https://www.journals.uchicago.edu/doi/full/10.1086/426877

[iv] https://www.aeaweb.org/articles?id=10.1257/aer.20210766

[v] https://www.amazon.com/Randomistas-Radical-Researchers-Changing-World/dp/0300236123

[vi] Cite

[vii] They shared their prize with Michael Kremer, but he is definitely less vocal in his promotion of this methodological shift.

[viii] https://www.nobelprize.org/uploads/2019/10/duflo-lecture.pdf, p.455.

[ix] https://www.nobelprize.org/uploads/2019/10/banerjee-lecture.pdf, pp.419-20.  For a thoughtful critique of Bannerjee and Duflo, see Pritchett https://www.cgdev.org/blog/homage-randomistas-occasion-j-pal-10th-anniversary-development-faith-based-activity