Research Highlight: Scaling up a social media educational platform in India
August 8, 2021
In this research impact highlight, I discuss my role doing research on a social media messaging platform, end-to-end research and close with what happened as a result of the research. This research highlight summarizes the following paper:
In this paper, we analyzed small-scale Indian farmer intuitions about agriculture and their environments and algorithmic effectiveness on an educational social media platform.
I was a postdoctoral research associate at Cornell Tech in human-computer interaction while doing this research.
We asked: how can an education social media platform be made to successfully achieve scale, when the platform is entirely voice-driven?
Reaching out with voice messaging: What do we know about engagement on social media?
We know that users prefer quality over quantity, although both are important. Scissors, Burke and Wengrovitz (2016) find that Facebook users care more about who likes their posts than how many likes they receive. In our paper, we ask: can this insight help audio education scale? If so, how?
We faced an important constraint from our education context that is distinct from most social media platforms. Educational messages are not as entertaining as the average social media post and getting a call/voice message on your phone is not going to be as pleasant as a visual user interface.
Our study was based in Gujarat, India and focused on small-scale farmers, although it later expanded. At the start, farmers could receive educational messages about planting, crop protection and harvesting either (1) directly or (2) indirectly.
Direct: A user may receive an educational message from the platform; they may be algorithmically targeted based on their sociodemographics
Indirect: A user may receive an educational message indirectly from another user; users may use their intuition to forward messages to up to 5 phone numbers for free.
We wanted to know: Which mechanism do listeners prefer? Do farmers prefer messages assigned to them from an algorithm, or messages curated by other farmers?
There were algorithmic fairness issues to think hard about, however. A message received from the platform may favor certain subgroups. Even if not, a forwarded message is sure to be non-randomly assigned: Users who received messages directly from the platform may be systematically different from receivers of forwarded messages. We wanted to avoid a scenario where people were only listening to messages because a certain person had sent them, but to do so because they believed in the content itself. This was an identification problem (in the true sense of the phrase) that we had to overcome in our research.
Our partial solution was blind users from knowing who sent a message. To a receiving user, messages from the platform and from other users appear identical. If users cannot tell the difference, they are arguably as good as randomly assigned as far as the source is concerned.
The Khedut Saathi Message Forwarding System
Khedut Saathi “Farmer’s friend” was a homegrown innovation by an Indian award-winning startup. Users of the Khedut Saathi platform could subscribe by leaving a missed call to a local number and pressing 9. Subscribers receive 3 voice messages each week about best agricultural practices. An episode plays as an audio recording.
We began with a focus on farmers in Gujarat. After listening to the content, farmers can record their questions and comments. At the start of the study, 12,000 farmers had subscribed to KS and only 1% had unsubscribed. The pickup rate was 75% on average, a highly engaged audience. Farmers may forward a message to up to 5 other numbers for free. A day (24 hours) after a message is sent out, we collect new data on forwarding and receiving forwarded messages.
In our experiments, we look at the impact of message source on the effective duration or listening rate (listening time/message time). The features are socio-demographics (age, gender, income), crop fixed effects (five commonly grown crops), month-fixed effects, and whether or not the farmer has Facebook or WhatsApp on their phone (a proxy for whether the phone is a smartphone).
We find that message source significantly affects listening duration and listening rates (the proportion of a message that is listened to). Our main result is that messages received from other farmers are listened to more frequently and more thoroughly than data-driven messaging.
We later removed the blinded aspect of the content and tracked messages as they were forwarded across social networks. We were surprised to find that the educational and scientific content went viral across India, which we recorded by identifying new calls into the Khedut Saathi system from around the country. The experiment resulted in significant interest beyond the agricultural sector, and there seems to be a significant interest in educational content by farmers and non-farmers alike. Although our information on non-users were limited, we were in a position to submit surveys and understand the social dynamics of such listeners. Finally, we did qualitative interviews of a small subset to understand why the content seemed to resonate so strongly. Some users seemed surprised at the question and thought it self-evident that the material was important. Overall, we found the material to be strongly appreciated.
The study suggests that the intuitions of users have an important role to play in technological design. Development economics and human-computer interaction remain underexplored relative to the scale of relevant problems, but the synergies are clearly powerful. Although much work remains, there is a need for innovative research that further empowers small-scale farmers in India and beyond.
Although we focus on agricultural education in this paper, it also seems clear that education is not just useful for its own sake, but is also a powerful platform that other aspects of development can piggy-back on for the economic and social benefits of education to scale. Collaborations across different parts of the world are another way that we can make the world a better place.
A New Era for Development Economics
As countries around the world continue to build and rebuild their economies in this new era, business as usual is not going to enough for people, families, communities and nations. There is a vital need for economic transformation that is based on next-generation development economics, informed by economic history as well as political economy, inclusive, and ready to evolve with the digital age.
Although there are glimmers of hope from the Africa Rising narrative to the mobile phone revolution and a nascent startup and machine learning ecosystem, much work still remains in the realms of policy, infrastructure and transformation. All of these and other movements–and this goes back to the independence era of the 1960s and the second democratic wave in the 1990s–have tended to happen outside the purvey of development economics, suggesting that ours is a profession that needs a better connection with history in the making. We see this with the growing interest of the global technology sector in the developing world, social movements across the board, and a new wave of young people making change in their worlds.
Productive industrialization as well as economic and political development remain viable, but it is clear that it must look quite different from what has come before if it is to succeed. Being tied into global chains in the traditional sense, while important, also risks entanglement and even stagnancy, requiring new approaches to globalization and social impact. Success itself is prime for a redefinition, being a moving target. The assumed limitations of developing country governments and their partners actually have much in common with small startups that can punch above their weight with data-driven and scientific mindsets.
DEVELOPMENT ECONOMICS X is development as the human condition.
Most development economists joined the profession with game-changing transformation in mind, before life got complicated and they had to lower their ambitions and accept the limited status quo as a culture. However, it’s not too late to improve as we are all learning.
Although many young people are and will be the next-generation of consumers for firms and beneficiaries of policy originating in developed countries, we have a chance to chart an entirely new path. What are their dreams and how can they come true? From physical capital, human capital, technological change and global competitiveness to name a few, needs abound. New environments are emerging, from digital infrastructure to innovative legal and socioeconomic institutions that put the interests of developing country citizens first, and not as a constraint to the interests of parties in developed nations.
The next generation will decide what development economics is, does, and means for the future of the developing world, the planet in general, and the development economics profession in particular.Kweku Opoku-Agyemang
Development in the twenty-first century
Development economics in the twenty-first century is occurring with the backdrop of a new political economy and economic history in the making.
Development Economics X is dedicated to ensuring that technological change fundamentally improves the human condition in the developing world. By taking action to ensure that innovation is a true global phenomenon and harnessing the next generation, development economics can and will become a culture around the world.