Saturday 11:00 AM–11:30 AM in C11

Difficulties in working with the government data and how can tech help us resolve it.

Ankita Mathur

Audience level:


Government of India posses a lot of crucial data but in its current shape, it is not very useful. Technology plays a major role to transform this data which can be used to empower decision making process. At SocialCops data goes through a perfect blend of technology and research which helps data not to loose out on its solution and impact yielding quality.


This is Anjali. Her village doesn't have a school, so although she is 11 years old, she has never seen the benefits of the government schemes that aim to provide her with free education until she is 14. In order for her to reach a classroom, she would have to walk nearly 12 km each direction, a distance that would not only be difficult for a child to cover, but puts a strain on her family, as she is needed to help at home.

The way that data is collected and stored in the developing world makes it easy for children like Anjali to slip through the cracks. Although India is one of the technological hubs of the modern world, data about its inner-workings is the single biggest obstacle to making meaningful decisions. And, in many cases, it isn't because the data doesn't exist. In the village Putrela, a government employee named Sita is responsible for tracking a government nutrition scheme for pregnant women. Every time she advises a woman, she records the woman's information and details about the interaction in a paper register. Sita is just one of six employees responsible for monitoring this one scheme in her one village. There are over 640,000 villages in India, and numerous schemes spanning all sectors. This means there are 1.6 million field staff just like Sita who generate more than 10 billion rows of data per month, all on paper. But the vast majority of this information will never be converted to a digital format. Even data that does eventually go digital is usually stored in local languages and formats that are impossible to interpret. And if the data somehow makes its way into a consumable format, it is often useless because it is stored in siloed, disconnected systems that cannot talk to each other. For example, the Indian district of Nagpur alone has 32 separate health data systems that are completely incapable of working together. And just when you think it can't get worse, it does.

Traditionally, collecting this amount of data and using it to solve a problem of this size would have been difficult, if not impossible. SocialCops data intelligence platform, however, removes many of these challenges by flipping the decision-making model. Rather than looking at the data that already exists and letting it lead us to a solution, SocialCops identifies the problem and allows it to take us to the data we need to solve it. We knew that there were still students underserved by India's education system, regardless of the schemes in place to level the playing field. Faced with this problem, we set out to tell their stories. We mapped each of the 640,000 villages in India and used existing data sources to find information about each school. From there, we were able to learn where exactly each one was, and what needed to be changed to meet RTE requirements and provide all students with better opportunities.

This is where our platform's intelligence comes in. It knows that Bombay changed its name to Mumbai in 1995, and people tend to misspell Pomburna as Pompurna. Through these algorithms, even the messiest, most mismatched data could come together into a useful format like the interactive map we created. This dashboard allowed officials to see just how their district's schools were doing and where they needed to make changes.

This is just one example of how our use of data is making it easier for decision makers to make accurate, informed decisions. How do we do this? It’s called data intelligence. Now, big question — what’s data intelligence? It’s a new model of using data to drive big decisions. Data intelligence is important because today’s model of engaging with data is broken. Since much of today’s data is so problematic, so ugly, even the most important decisions only rely on a small subset of the world's data — the data you already have or data that's easy to get. But this isn't enough for big decisions in the social sector. You need more. That’s where data intelligence comes in. It’s about finding and making sense of all the possible data you need to make the best decision. It's about looking for data in places where you may never have looked before.

Subscribe to Receive PyData Updates


Get Now