This video was easily my favorite part of Jameel Poverty Action Lab 's ten year anniversary celebration last weekend in Cambridge. And I wasn't alone; I'm pretty sure these J-PAL research assistants -cum- directors garnered the most applause of the entire event (even more than Bono... albeit on video).

Because I am PhD student conducting my own RCT, I have the pleasure as serving as my own principle investigator and as my own research assistant. The joys of reviewing piles of surveys, getting intimate with Stata and traveling in the back of vans on terrible East African roads rocking out to some American pop.... yes.

Although the prime publications, important policy implications and fancy awards get all the press, the real brilliance on an RCT happens in the field. Cheers, RAs.

Woah, exciting things happening in the world of conditional cash transfers.

A fascinating This American Life/ Planet Money dream team podcast on the effectiveness and absurdity of of dropping all pretense of developing education, institutions or capabilities in poor countries and just giving out straight cash.  The follow-up response has been awesome, with posts in the NYT, NYT Magazine and of course, a clever summary with provocative highlights from Chris Blattman.

According to David Kestenbaum of Planet Money, these cash hand-outs are sponsored by nerds:

Until pretty recently, the charity world has been about doing stuff that helps, without thinking really about how much does it help exactly and how much does it cost? But there does seem to be this shift that's happening. Philanthropy is getting nerdier and more data driven.

Is this because economists are trying to take over the world? Maybe. GiveDirectly is an organization, not surprisingly run by two Harvard-trained J-PAL affiliated economists. Of course, GiveDirectly is doing Randomized Control Trials (RCT) to evaluate the impact of these cash handouts on all sorts of outcomes (family dynamics, height, nutrition, housing, asset ownership and education). Kestenbaum again:

This study is not just about how well giving money works. Its also a challenge to other charities....Providing training, flying in experts, paying staff- all that costs money. If you think all your doing is better than just giving money, prove it.

As a friend said, this seems a lot like when academic economists jump into a very complicated real-life situation and claim to know how to do everything better (perhaps unsurprisingly, she dates an economist). But when you think about the impact of development in terms of dollar spent on organization websites, a states-side office and director salaries versus a dollar spent in Meatu by Joe the Tanzanian Plumber, it is hard to make a strong case for all that costly stuff without data.  It's no surprise that the economists at GiveDirectly despise glossy aid brochures and are rigorously and empirically evaluating their own program. So what do other aid organizations think about this challenge to prove effectiveness? From Heifer International:

As an African woman, that sounds to me like a terrible idea... We're not about experiments. These are lives of real people... Data cannot capture everything. I am a proud woman and I have a voice. You can't measure that.

Denial of experimentation? CBlatts nails it on the head:

This is the way the Heifers of the world fool themselves. When you give stuff to some people and not to others, you are still experimenting in the world. You are still flipping a coin to decide who you help and who you don’t, it’s just an imaginary one.

For the amount of money that goes into aid organizations, yes, they should absolutely be evaluated with rigor and data. But it also seems likely that Heifer does bring more than a cow. They bring business and agriculture training, share milk production and technology knowledge, heighten business curiosity and even possibly, include this elusive idea of empowerment. Or, they like to think they do anyway. Do we, as economists, think we can measure all that?

Hat tips: COCO & DW

My fantastic field assistant and his wife had their first baby two months after the start of the (year-long) intervention. I'm proud that we both work for an organization that gave him paternity leave. Now, four months after the start of the intervention, he has a bad case of malaria.

I'm worried about the size of the detectable effect on outcome variables, but health and babies are more important. Too bad good health doesn't get you three stars on a regression output table.

In a previous post, I responded to an article in Econometrica that had been highlighted in Freakonomics about the externalities of experimental research in developing countries. One of the authors of the paper, Ken Leonard, responds:

In Tanzania, there has been a strong ethic to participate in research for the benefit of the country and that implies a reciprocal responsibility for researchers. Therefore, in the research we did with the Maasai, we explicitly avoided the use of the word research (utafiti) so as not to give the impression that we did not respect that implicit arrangement. The two villages where we did research were rural villages populated with Maasai people, but they were not unaccustomed to outside research. In one of the villages, the chief had taken to visiting women the day before a research team arrived to prime them as to what they were to say when the research team arrived. In our case, the women had all been primed to say that water was the most important issue. Many of the respondents to the exit surveys were concerned at the end that we had not given them the opportunity to say water (since of course our research was not about their development needs).

I was and continue to be very concerned about the impact of research on the villages I work in, but I can safely say that these villages were not in some sort of pristine pre-research state. That said, our research was so different from what anyone expected, that I am very confident that our results were not tainted by expectations.

Some people earned significant sums of money and many of these were the poorer and older members of the village. I cannot say with assurance that the money was put to good use, but I can testify that, as I handed people the money, the looks on their faces were very satisfying. People were genuinely surprised that I kept my word and gave them money and they were very pleased with the amounts they earned. Their surprise at seeing a promise fulfilled is a testament to the dishonesty of the average researcher in this area.

A friend of a friend is embarking on a research internship in Uganda this summer and posed the following questions:

In focus group discussions, did you compensate people (monetary or non-monetary) for their time? How long did each discussion last? What strategies did you follow when you were designing focus group questionnaires? What is the size of your focus group? And in terms of sampling, how did you get a random sample? Did you have village leaders to help you in the sampling process?

I was working with a well-known Tanzanian NGO and affiliated field assistant, which had experience conducting research and working in my study community.  We met first with the chairman in each village and asked them to organize groups of mothers and fathers for focus groups. We gave vague age restrictions (older than twenty) and asked for about ten to thirty people for each. This introduction worked fine because we did not need a random sample for focus groups, and it certainly helped the this NGO had a very good reputation for community development (trying to avoid those externalities again).  I wrote up about fifteen questions for each focus group (though these changed slightly for each additional village) and the discussions lasted about two hours. Because my main methodologies are quantitative, the focus group questions are more about informing the household survey and having a better understanding of cultural and institutional norms. We tried to play it safe with single-sex focus groups, but that did present some challenges. Participants were "thanked" (not, according to the NGO, compensated) for their participation with sodas bought at the local shops.

For the sampling procedure, we again benefited from the helpful village administration. Because Tanzania has a long socialist history, village and subvillage leaders keep track of families in their communities. So after another formal village introduction, and officially asking each village to participate in the study, we asked the leaders to make lists of every household in the village. The first day that we intended to start sampling, we took the lists (separated by subvillage) and numbered them (~200-800 households). We then pulled out the handy iPhone random number generator and picked households. We were sampling 60 households per village and sampled proportionately across subvillages (~2-9 per village). The subvillage leaders often came along with us in the truck to help find these randomly selected households. And then the fun began.

In a fun and puzzling Freakonomics radio podcast, Dubner and Levitt explore the ways in which women participate differently in society, or in other words, women are not men. One of the main studies they cite is an interesting piece from Econometrica by Uri Gneezy, Kenneth Leonard, and John List which examines how culture effects competition between men and women. They attempt to understand if its nature or nurture that effects the stark difference between men and women opting to compete.

To do this, they immerse the competition studies in two different tribes: the Khasi in India (a matrilineal tribe) and the Masai in Tanzania (very patriarchal). They set up a simple game where participants were paid to throw balls into a bucket, and were paid up to $3 (4500 TSH) to compete. The game certainly gave insight into the long arm effect of patriarchal culture on reducing women's appetite for competition, but I can't read this paper without thinking about the externalities of participation. The authors admit that they weren't exactly perceived as participating in a partnership for local development:

Gneezy says it wasn’t long before word got around that these “ridiculous Americans” were paying villagers to throw tennis balls into buckets. People came from all over the village to play. 

Because I struggled with whether or not to pay people in my own Tanzanian research study, the consequences of participation in this study, without any noted explanation of the fuller purpose of the gender research, seem inappropriate and detrimental. Most Masai villages that I saw were a thunderstorm of land rights debacles, domestic violence, sexually transmitted disease, poor education and dangerous wildlife conflicts. These areas are certainly worth examination and understanding, but what will happen to future researchers in these Masai villages?

It seems to me that these sorts of externalities of paid participation are much more severe when the participants do not see any benefit or meaning in the purpose of the study. Did this article in Econometrica effect gender equity in the Masai study villages? Seems unlikely.

I wouldn't go so far as to pretend that I am not perceived as a white walking pile of money ("white person! give me gift!"), but gaining IRB approval is not a replacement for critical judgement about the consequences of human subject research.

Hat tip: DR

In an effort to incentivize household visits and family planning consultations throughout the entire village (i.e. make sure this intervention actually happens), my project is giving "work materials." While a bicycle, backpack, raincoat and gumboots are certainly helpful work materials for tromping around distant subvillages and filling out forms, they are also particularly awesome gifts.

And I got to be part of distributing these gifts. It was a lot of fun.

Further evidence of the importance of humility in doing research abroad:

One thing I’m finding in the field is that the struggle to do regular, non-research things often leaves me feeling helpless and generally incapable. ...At what point will I transition from “high-functioning kindergarten student” to “acclimated field researcher who can walk the walk in Brazil”? (Judging from my mentorship team, 8-10 years is the place to start.)

From CS O'Connell

Working with a great group of enumerators, dedicated family planning distributors and intelligent NGO staff, I’ve found that I’m interacting with folks who want my research to turn out well. I’ve also found that Tanzanians in particular are non-confrontational and eager to please. This combination, as I learned in my last field experience, lends itself to the possibility that instead of learning the truth in this research, I’ll get handed the “right” answer.

Last week, my field assistant, a family planning trainer, three distributors and me traveled to a Meatu district village to practice household visits and family planning consultations. Two of the trainers are absolutely fantastic, dynamic and encouraging. The trainer who traveled with us last week was not one of those two.

Once we got out to Sapa village, one of the distributors and I visited a small household with a 35-ish year old woman, who had a baby on her back, and was sitting over a pot of simmering rice.  I sat listening to the conversation, which dabbled between Swahili and Kisukuma (the local tribal language), trying not to be too invasive in this personal discussion. There was some mention of the clinic and being sick.  She pulled out the family planning questionnaire from her bag and began asking the questions. Meanwhile, the trainer came over to supervise the conversation and sat between the distributor and myself.  When the distributor got to the last question about whether the woman wanted to take any contraceptives, which is meant to facilitate discussion about which method may be the most appropriate, the trainer swooped in and began to mansplain to her why yes, in fact she does want them. [A small apology to JAJ for the condescending vocabulary choice, but mansplain is, in fact, entirely the most accurate word for this situation.] I began to interrupt and he spoke to me in English, which the distributor and interviewee both cannot understand: “Does she want to take family planning? Yes is the answer.” He took the questionnaire from the distributor and marked the “ndiyo” (yes) box.  I explained in meager Swahili that I wasn’t sure she had actually said yes since she indicated that she had been sick. “Yes,” he said, “the hormones gave her psychological problems.” Well, hello, then it is understandable that she might say no to this question. I tried to explain that if she actually says that she does not want contraceptives, it is not a problem for the form, regardless of the reason. If she says no, we should mark the box no. “Ah,” he said, “family planning, it is a process.” To him, I think this means that “no” is only acceptable for the first few visits, but then, of course “yes” is the right answer.  This seems to be his interpretation of the objective of my research, which he is very happy to hand to me on a silver platter.

I later explained this to my field assistant, who is excellent at translating my worries into clear directions in Swahili. But I still have a lingering concern that this trainer may have imparted the notion that we are only looking for the right answer, which is yes, everyone wants contraceptives.

With widespread intimate partner violence in Tanzania (39% according to DHS) and in Meatu district (35% according to my sample), there is likely a   relationship between contraceptive use and violence. It's hard to say what exactly that relationship is, especially with the difficulty of establishing causality when these things are all related to the personal intricacies of an intimate relationship. This study in the Journal of Biosocial Science, however, finds a strong relationship between covert contraceptive use and violence.

But again, does that mean using the pill in secret increases violence, or is it the case that women who experience violence are more likely to hide the pill? Researchers wanted.