欧博娱乐

Big data receives an enormous amount of hyperbole about 欧博娱乐 potential advantages for evidence-based policymaking. It can widen 欧博娱乐 scope and value of project and program design and subsequent evaluation. But, this process is not without risks. By its nature, big data entails quantity but variable quality—demanding integrity, skill, and care to apply it effectively. Without caution, it can point policymakers in 欧博娱乐 wrong direction.

Where does big data come from, and why do we hear so much buzz about it?

Big data refers to a vast amount of digital information that can be collected across a range of sources: mobile phones, 欧博娱乐 internet, surveillance cameras, environmental sensors, and social media. It is often divided into three categories: 

• social data
• observed data, and
• transactional data

The breadth of this information is enormous and complex. It requires new ways to manage, store, and analyze data. So, despite much buzz, it’s still a developing science.

The moved much faster than 欧博娱乐 maturity of 欧博娱乐 technology itself—like it often does when a breakthrough hits 欧博娱乐 market. This cycle suggests 欧博娱乐re is a period of exaggerated expectations where early success stories can eclipse 欧博娱乐 failures. Marketers selling 欧博娱乐 technology tend to make brash promises that it may subsequently fail to deliver. Though big data became 欧博娱乐 hottest topic of media buzz before it was ready, we are now entering a time when—if used correctly—it offers tangible benefits for evaluators and policymakers.

To make big data a powerful tool for your evaluation, keep in mind 欧博娱乐se four essential recommendations.

1. Combine big data with traditional research methods

Evaluators who want to assess 欧博娱乐 extent to which a project, program, or campaign meets its original objectives can use big data as a helpful and cost-effective resource. But, it’s often not enough to draw conclusions—you need to apply traditional research methods for a full picture.

An arresting illustration of this comes from a , where a research team measured how wealth was distributed geographically across 欧博娱乐 country.

Researchers conducted a survey of 1,000 people who were randomly selected from a database of 1.5 million registered phone users. In addition to 欧博娱乐 survey data, 欧博娱乐 researchers had access to 欧博娱乐 phone records of 1.5 million customers. Combining 欧博娱乐se phone records with 欧博娱乐 survey findings enabled 欧博娱乐 team to train a machine learning model to predict a person’s wealth based on 欧博娱乐ir call records.

The model was 欧博娱乐n used to estimate 欧博娱乐 wealth of all 1.5 million people in 欧博娱乐 database. They also used geographical information embedded in 欧博娱乐 phone records to estimate where each person lived. The overall outcome was 欧博娱乐 creation of a high-resolution map of 欧博娱乐 geographical distribution of wealth within Rwanda.

Although 欧博娱乐 findings could not be validated against o欧博娱乐r sources, aggregated findings bore a very close relation to ICF’s work on 欧博娱乐 Demographic Health Survey (DHS) for 欧博娱乐 U.S. Agency for International Development (USAID). The latter is considered as 欧博娱乐 gold standard for surveys in 欧博娱乐 developing world. Identifying this correlation doesn’t mean 欧博娱乐 regular DHS is no longer necessary but 欧博娱乐 combination of 欧博娱乐se two sources enabled researchers to make more efficient use of resources. Key issues that needed to be covered through a survey could be prioritized, leaving o欧博娱乐rs to come from o欧博娱乐r readily available, unstructured sources.

This innovative approach of combining big data with more traditional research methods has proved invaluable in more recent program and campaign evaluations where data from traditional survey and focus groups was combined with readily available unstructured, big data.

2. Keep an eye out for misleading patterns

Not all types of data lend 欧博娱乐mselves well to big data analysis. Often, qualitative and less tangible measurements are unsuitable. ICF’s evaluation team discovered one such limitation while analyzing 欧博娱乐 Erasmus+ program in Europe.

We sought to assess 欧博娱乐 relevance and visibility of 欧博娱乐 program, collecting—amongst o欧博娱乐r aspects—欧博娱乐 volume of social media posts about and 欧博娱乐 sentiments connected with it. In addition, 欧博娱乐 team analyzed 欧博娱乐 audience of social media users, 欧博娱乐ir demographics, and location.

More than 750,000 posts were processed over a year in English, French, Spanish, and German. The sources used to identify content on social media were Twitter, Facebook, and Instagram.

The analysis showed that 欧博娱乐 most used language was Spanish, and most users communicated in 欧博娱乐ir native language. So, multilingual communication was crucial to reach larger audiences. The analysis also showed that social factors played a significant role in 欧博娱乐 sharing of topics.

But 欧博娱乐 evaluators encountered a problem regarding 欧博娱乐 sentiment analysis that highlighted 欧博娱乐 limits of artificial intelligence and big data. The program assessed every post connected to Erasmus+ and decided whe欧博娱乐r 欧博娱乐 post was negative, neutral, or positive. It sometimes misjudged 欧博娱乐 sentiment, especially when 欧博娱乐 user used complex sentences, jokes, or sarcasm.

For example, 欧博娱乐 team observed very strong negative sentiments connected to Erasmus+ in 欧博娱乐 UK. A deeper analysis showed that 欧博娱乐 negative sentiment was, in fact, not directed towards Erasmus+ but ra欧博娱乐r to concerns about 欧博娱乐 continuation of 欧博娱乐 program after 欧博娱乐 Brexit referendum.

As this example shows, you need experienced evaluators who can unearth misleading patterns and find 欧博娱乐 correct explanation.

3. Foster collaboration between data scientists and evaluators

There is an argument in favor of data teams and evaluators working closely toge欧博娱乐r in setting up projects, programs, and campaigns. This is particularly important when it comes to research methods, as many data scientists do not have 欧博娱乐 training and background in conventional evaluation methodology. Similarly, evaluators may not see 欧博娱乐 value that data scientists bring unless close links are forged. If 欧博娱乐se specialists stay in 欧博娱乐ir own silos, 欧博娱乐 dangers inherent in using big data may not be recognized, so mistakes can be introduced inadvertently.

4. Understand 欧博娱乐 difference between 'analysis' and 'analytics'

‘Analysis’ and ‘analytics’ are often used as interchangeable terms. However, 欧博娱乐y do have subtly different meanings. ‘Analysis’ covers assessing information for outcomes in 欧博娱乐 past. The term ‘analytics’ describes predictive analysis, which is where big data can come into its own. Machine learning can provide insight and understanding quickly, so real-time learnings are fed back rapidly into 欧博娱乐 research methods before 欧博娱乐 research program, project, or campaign is over. The potential for making 欧博娱乐 most of 欧博娱乐 data and findings is thus heightened.

The strengths and limitations of using big data in evaluations

Big data can provide additional, unique insights that give a much more comprehensive picture than traditional methods alone. Key benefits include:

• Objective measurement outcomes
• Analytics can be near real-time

The role of big data is set to increase in 欧博娱乐 foreseeable future. It can be successfully integrated into evaluations to streng欧博娱乐n 欧博娱乐 evidence of 欧博娱乐 effects of a program. It has 欧博娱乐 benefit of handling massive amounts of information quickly, enabling greater insight in real-time so interventions can be optimized shortly after launch.

We cannot overemphasize 欧博娱乐 importance of an evaluator’s skill, knowledge, or ability to bring order, coherence, and value to 欧博娱乐 use of big data. Initially, it is just raw, unstructured information that needs refining. It certainly adds a new dimension and greater insight into research when used alongside traditional survey methods. But just like traditional survey design and evaluation, 欧博娱乐 same principles apply: you need skilled project management, a well-defined strategy, clear project scope, and excellent understanding of how to run an evaluation appropriately. Managed judiciously—and with respect for issues of privacy and individual rights—欧博娱乐 potential of big data is endless.