Financial Times printed letter to the editor from EES board member Mr. Robert Picciotto Randomised trials present problems

Randomised trials present problems

Sir, Rigorous and independent evaluation can help ensure that politicians unlike physicians are prevented from burying their mistakes. But Tim Harford errs when he argues that this is best done by subjecting political ideas to randomised control trials ("Political ideas need proper testing", March 18). Not only are such experimental methods expensive but their validity has proved problematic even in clinical trials. According to a Tufts University professor most published medical research findings are wrong and the greater the financial interests involved the less likely a research finding is to be true. When applied to a social program randomised control trials raise a host of substantive ethical and technical issues. They provide the illusion of certainty to decision makers but severe challenges must be faced in the selection of control and treatment groups. Spillover effects as well as unintended impacts on individual and group behaviour ('Hawthorne effect") are frequent and the very same policy is apt to yield very different results depending on how the program is implemented and adapted to the context. The instances where experimental research methods are serviceable are few and far between. Given their complexity they are readily manipulated by vested interests. Fortunately, the evaluators's tool kit is well stocked: despite what many economists believe, a variety of techniques that do not involve randomisation are available to determine whether, how and why a social program works or not.

Robert Picciotto
visiting professor

King's College, London

CZECH-IN s.r.o., Professional Event & Congress Organiser, Prague Congress Centre, 5. kvetna 65, 140 21 Prague 4, CZE, tel.: +420 261 174 301, info@ees2010prague.org