February 28, 2025. TORONTO, CANADA.
Kweku Opoku-Agyemang
In this research impact highlight, I discuss my role doing research on a social media messaging platform, end-to-end research and close with what happened as a result of the research. This research highlight summarizes the following paper:
In this paper, we analyzed small-scale Indian farmer intuitions about agriculture and their environments and algorithmic effectiveness on an educational social media platform.
I was a postdoctoral research associate at Cornell Tech in human-computer interaction while doing this research.
We asked: how can an education social media platform be made to successfully achieve scale, when the platform is entirely voice-driven?
Reaching out with voice messaging: What do we know about engagement on social media?
We know that users prefer quality over quantity, although both are important. Scissors, Burke and Wengrovitz (2016) find that Facebook users care more about who likes their posts than how many likes they receive. In our paper, we ask: can this insight help audio education scale? If so, how?
We faced an important constraint from our education context that is distinct from most social media platforms. Educational messages are not as entertaining as the average social media post and getting a call/voice message on your phone is not going to be as pleasant as a visual user interface.
Our study was based in Gujarat, India and focused on small-scale farmers, although it later expanded. At the start, farmers could receive educational messages about planting, crop protection and harvesting either (1) directly or (2) indirectly.
Direct: A user may receive an educational message from the platform; they may be algorithmically targeted based on their sociodemographics
Indirect: A user may receive an educational message indirectly from another user; users may use their intuition to forward messages to up to 5 phone numbers for free.
We wanted to know: Which mechanism do listeners prefer? Do farmers prefer messages assigned to them from an algorithm, or messages curated by other farmers?
There were algorithmic fairness issues to think hard about, however. A message received from the platform may favor certain subgroups. Even if not, a forwarded message is sure to be non-randomly assigned: Users who received messages directly from the platform may be systematically different from receivers of forwarded messages. We wanted to avoid a scenario where people were only listening to messages because a certain person had sent them, but to do so because they believed in the content itself. This was an identification problem (in the true sense of the phrase) that we had to overcome in our research.
Our partial solution was blind users from knowing who sent a message. To a receiving user, messages from the platform and from other users appear identical. If users cannot tell the difference, they are arguably as good as randomly assigned as far as the source is concerned.
The Khedut Saathi Message Forwarding System
Khedut Saathi “Farmer’s friend” was a homegrown innovation by an Indian award-winning startup. Users of the Khedut Saathi platform could subscribe by leaving a missed call to a local number and pressing 9. Subscribers receive 3 voice messages each week about best agricultural practices. An episode plays as an audio recording.
We began with a focus on farmers in Gujarat. After listening to the content, farmers can record their questions and comments. At the start of the study, 12,000 farmers had subscribed to KS and only 1% had unsubscribed. The pickup rate was 75% on average, a highly engaged audience. Farmers may forward a message to up to 5 other numbers for free. A day (24 hours) after a message is sent out, we collect new data on forwarding and receiving forwarded messages.
In our experiments, we look at the impact of message source on the effective duration or listening rate (listening time/message time). The features are socio-demographics (age, gender, income), crop fixed effects (five commonly grown crops), month-fixed effects, and whether or not the farmer has Facebook or WhatsApp on their phone (a proxy for whether the phone is a smartphone).
We find that message source significantly affects listening duration and listening rates (the proportion of a message that is listened to). Our main result is that messages received from other farmers are listened to more frequently and more thoroughly than data-driven messaging.
We later removed the blinded aspect of the content and tracked messages as they were forwarded across social networks. We were surprised to find that the educational and scientific content went viral across India, which we recorded by identifying new calls into the Khedut Saathi system from around the country. The experiment resulted in significant interest beyond the agricultural sector, and there seems to be a significant interest in educational content by farmers and non-farmers alike. Although our information on non-users were limited, we were in a position to submit surveys and understand the social dynamics of such listeners. Finally, we did qualitative interviews of a small subset to understand why the content seemed to resonate so strongly. Some users seemed surprised at the question and thought it self-evident that the material was important. Overall, we found the material to be strongly appreciated.
The study suggests that the intuitions of users have an important role to play in technological design. Development economics and human-computer interaction remain underexplored relative to the scale of relevant problems, but the synergies are clearly powerful. Although much work remains, there is a need for innovative research that further empowers small-scale farmers in India and beyond.
Although we focus on agricultural education in this paper, it also seems clear that education is not just useful for its own sake, but is also a powerful platform that other aspects of development can piggy-back on for the economic and social benefits of education to scale. Collaborations across different parts of the world are another way that we can make the world a better place.
