Comments on: Development that Works: RCTs (Methodology) http://www.criticaltwenties.in/economicsocialpolicy/development-that-works-rcts-methodology Powered by Webshore Sun, 29 Mar 2015 02:07:36 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Rohit http://www.criticaltwenties.in/economicsocialpolicy/development-that-works-rcts-methodology#comment-10091 Rohit Sun, 18 Dec 2011 12:19:25 +0000 http://www.criticaltwenties.in/?p=4049#comment-10091 Thanks Arghya, those are all very important points. I absolutely agree that RCTs have their niche and that they cannot answer every question. Indeed, one of the objections leveled at RCTs is that ultimately, the questions that they CAN answer are far too narrow when one adds in the various caveats. Ultimately, they help us answer a specific policy question in a specific context, that is all. However, in so far as the development project is a large, heterogeneous one, well-conducted RCTs can form a very useful part of the overall program. RCTs currently are either motivated by specific questions that researchers/academics have, or policies that NGOs and aid groups would like to get some rigorous evidence on. Based on my experience, while government-assistance and compliance is a crucial part of facilitating RCTs, they are not usually the originators of the research. However, they certainly form a large part of the target audience, so it is essential that we realize the uses and limitations in presenting our data. In future posts, I hope to work through these issues a bit more, so keep the questions coming! Thanks Arghya, those are all very important points. I absolutely agree that RCTs have their niche and that they cannot answer every question. Indeed, one of the objections leveled at RCTs is that ultimately, the questions that they CAN answer are far too narrow when one adds in the various caveats. Ultimately, they help us answer a specific policy question in a specific context, that is all. However, in so far as the development project is a large, heterogeneous one, well-conducted RCTs can form a very useful part of the overall program.

RCTs currently are either motivated by specific questions that researchers/academics have, or policies that NGOs and aid groups would like to get some rigorous evidence on. Based on my experience, while government-assistance and compliance is a crucial part of facilitating RCTs, they are not usually the originators of the research. However, they certainly form a large part of the target audience, so it is essential that we realize the uses and limitations in presenting our data.

In future posts, I hope to work through these issues a bit more, so keep the questions coming!

]]> By: Arghya http://www.criticaltwenties.in/economicsocialpolicy/development-that-works-rcts-methodology#comment-10086 Arghya Sat, 17 Dec 2011 11:58:07 +0000 http://www.criticaltwenties.in/?p=4049#comment-10086 Hi Rohit, Great article and thanks for demystifying the idea of a randomised control trial for types like me who keep hearing the words without knowing what they mean. I have an intuitive problem though with some such trials and the backing that they provide for decisions to be made. I think trials which are the foundation of what is now known as evidence-based policy-making can often act as shrouds to push through certain ideological points of view, backed by seemingly 'objective' data. Now this may not always be the case but looking at the way governments keep bandying data around, as if they are gospel truths and seeing that they do enjoy a fair degree of unquestioning credibility, makes me worried. I think if a political judgment is taken, we should know it is a political judgment and be able to question it on that basis rather than coming up against a brick wall of inaccessible data. Perhaps, and this is my question after this long ranting detour, we should limit such trials and by extension this type of data collection to aid decision-making, to certain kinds of decisions and not others? Not sure where the distinction can be drawn currently and what a reasonable classification would be, but I intuitively feel that in certain situations where RCTs are used currently, intuition may be better than argument. Thanks, Arghya. Hi Rohit,

Great article and thanks for demystifying the idea of a randomised control trial for types like me who keep hearing the words without knowing what they mean. I have an intuitive problem though with some such trials and the backing that they provide for decisions to be made. I think trials which are the foundation of what is now known as evidence-based policy-making can often act as shrouds to push through certain ideological points of view, backed by seemingly ‘objective’ data. Now this may not always be the case but looking at the way governments keep bandying data around, as if they are gospel truths and seeing that they do enjoy a fair degree of unquestioning credibility, makes me worried. I think if a political judgment is taken, we should know it is a political judgment and be able to question it on that basis rather than coming up against a brick wall of inaccessible data. Perhaps, and this is my question after this long ranting detour, we should limit such trials and by extension this type of data collection to aid decision-making, to certain kinds of decisions and not others? Not sure where the distinction can be drawn currently and what a reasonable classification would be, but I intuitively feel that in certain situations where RCTs are used currently, intuition may be better than argument.
Thanks, Arghya.

]]>