Why is appropriate probing important?

A question on the labor status of an individual seems pretty easy to explain when looked at from the outside. But as you delve deeper into it, you discover the multiple layers. In a training session that we conducted, while giving examples on how to categorize an individual's current labor status, one field enumerator asked, "What is the kind of employment of a priest?"

Another enumerator answered, "Self-employed".

This led to a long drawn discussion among the field enumerators on how it is neither of the above and the ambiguity had to be resolved with the following explanation, given the context of the assignment. A priest employed by a temple trust and paid a monthly salary will be categorized as a "salaried employee". On the other hand, a priest who conducts ceremonies as and when required at houses of people or other places will be marked as "other" in the options list and be specifically listed as a "daily wage earner”. This simple question of a field enumerator led to a very important probe being incorporated in implementing the survey instrument as and when such a response is received.

Lesson learned - Even the simplest of questions come with its own set of connotations and may require in-depth probing to arrive at the desired answer.

Under Suspicion

The village was, as is often the case, remote – 3 to 5 kilometers from the nearest road. Coming in we were met with stares instantly, our guide, a local ASHA, told us that outsiders were rare. This wasn’t unique but odd to us. The villagers followed us around, tense, listening carefully to what we were saying in our unfamiliar accents. Our study – on the topic of sexual and reproductive health – was a sensitive one and we were to talk to young adolescents in groups by themselves. The village members were clearly not happy with this, and although the village’s ASHA and Mukhiya supported us, we were not welcome. We ended up having a small group discussion with the few adolescents whose parents were comfortable. We left quickly, the villagers followed us to ensure that we were gone.

“Why were we met with so much hostility?” We asked the ASHA, the Mukhiya, and the few friendly respondents.  Soon the story came out – there were rumored cases of outsiders luring children away from a neighboring village, and harvesting their organs. Us, strangers coming in and wanting to speak privately with children had unknowingly triggered the villagers’ fears.

For institutional review boards, the ethics are clear – you go to a village, you get informed consent, tell the respondent the risks and benefits, talk to the respondent privately so that they are not ostracized for their views and then you leave. We had followed these best practices and more, going ahead and talking to the ASHA and Mukhiya, explaining our study and gaining their support. But the backlash still happened. How do you get data from a place that doesn’t trust you?

'Purush' versus 'Aadmi'

“Aapke ghar mein kitne purush rehte hain?”

“Purush?”

“Kitne aadmi rehte hain?”

She glared at us.

Did we make a mistake? Yes, we did. In Rajasthan, the word aadmi denotes “husbands”, not men. For our respondent, the question translated to “How many husbands do you have?” No wonder she was offended.

We wanted to know the number of male members living in the households. Finally, after apologies, some subtle probing we got the number.

Lesson learned: There are more linguistic variations than what we might be aware of. So be wary while translating!

The legacy of an RCT

She had been following us for hours. Survey after survey, a steady shadow that dogged our steps despite the mid-day heat, at the height of the Delhi summer. “Didi, why are you here?” we finally asked. “Survey me as well” she answered.

She wasn’t one of our randomly selected respondents. The survey was long, almost two hours, and we normally had to beg respondents to take it, not fend them off. “Didi, why do you want to be surveyed?” we asked. Her answer was garbled with her passion but with some help from her neighbors, we finally got the story.

A couple of years ago, another set of researchers had visited and administered surveys. Like us, they were randomly choosing respondents. But unlike us, it was for the baseline of Randomized Control Trial where the selected respondents were given monetary and technical help to construct houses. She hadn’t been selected, but her neighbors had. And she had watched over the years, as they built their fancy homes while she was forced to live in her shack. Determined not to be omitted from a survey again, she now makes sure that surveyors include her.