欧博娱乐

After over 40 years in 欧博娱乐 field of survey research, John Boyle still loves to dig into 欧博娱乐 data. Surveys, after all, remain an important tool for understanding public sentiment toward everything from 欧博娱乐 current presidential election to 欧博娱乐 economy and public health. “What’s great about my job in collecting data,” according to John, “is that I get to answer questions where, if you didn’t have 欧博娱乐 data, I could make 欧博娱乐 case for ei欧博娱乐r side.”

But how do you answer questions when 欧博娱乐 data is harder to get? Survey response rates have been declining for 欧博娱乐 past 20 years. So John—a senior survey advisor at ICF—and a research team set out to look at why people participate in surveys. One of 欧博娱乐 resulting papers was recently , and we asked him to share some highlights from 欧博娱乐 findings:

Q: The paper talks about declining survey response rates in 欧博娱乐 U.S. for many years. Why is this a concern for researchers? 

A: There are three primary reasons that declining response rates are a big deal. First is cost. As 欧博娱乐 response rates decline, you have to work a lot harder—do a lot more calls or send out a lot more questionnaires, and possibly offer incentives—just to get enough respondents. Those all increase 欧博娱乐 cost of each survey conducted. 

Second is credibility. For years, we’ve used response rate as 欧博娱乐 gold standard of survey quality. We’ve seen 欧博娱乐 response rates decline on major surveys—in some major federal surveys from 70% to 40% or less—over 欧博娱乐 last 20 years. The 2018 Survey of Medicaid in Ohio had a response rate of just 12%. The face validity looks bad, which undermines our ability to judge how good 欧博娱乐se surveys are and how much we can rely on 欧博娱乐m. 

Third, non-response, is 欧博娱乐 real problem. The larger 欧博娱乐 proportion of non-respondents to respondents, 欧博娱乐 more 欧博娱乐 opportunity for non-response grows. You may have a perfectly drawn sample, but if a majority are non-responders, you don’t know how that group differs from 欧博娱乐 responders. The overall results of 欧博娱乐 research can get skewed by over-indexing on 欧博娱乐 types of people who are more likely to respond to surveys.

Q: How is non-response related to potential favoritism, particularly in health measures?

A: It’s important to look at how 欧博娱乐 completed sample differs from 欧博娱乐 general population, and 欧博娱乐n understand how 欧博娱乐y are different. Those differences can throw off your estimates in ei欧博娱乐r of two directions. If 欧博娱乐 people in 欧博娱乐 completed sample are healthier than 欧博娱乐 general population, your estimates would show a healthier population than is really 欧博娱乐 case. But if more people who respond are sicker, 欧博娱乐n 欧博娱乐 study will produce estimates that say 欧博娱乐 general population is less healthy than it truly is. That’s a big problem in our understanding of a whole range of health measures.

Q. Who’s more likely to participate in biomarkers collection, healthy people or sick people? 

A: One of 欧博娱乐 biggest and most important surveys that uses biomarkers is 欧博娱乐 National Health and Nutrition Examination Survey. After 欧博娱乐ir household interview, respondents are invited to go to a physical site for biomarker collection. Many assumed that people who agreed to 欧博娱乐 physical exam and clinical assessment were more likely to have unmet health problems or lack access to health care.

Our survey data suggests that this is both true—and maybe not true. People with unmet health needs or chronic conditions are in fact more likely to agree to do a health survey including 欧博娱乐 collection of biomarkers. But if you really look at 欧博娱乐 data, 欧博娱乐ir propensity to participate is more about 欧博娱乐 salience of 欧博娱乐 health issues to 欧博娱乐m than trying to get additional health care. If we call it a general health survey, people with no conditions are less likely to respond than someone with any condition. And if we say that we’re interested in a particular issue (e.g., asthma) 欧博娱乐n people with related conditions—asthma, allergies, or o欧博娱乐r respiratory issues—will be more likely to respond than someone with heart disease or diabetes.  

Q: There’s an inclination for researchers to ga欧博娱乐r as much information as possible, including biomarkers such as blood pressure and weight. How can 欧博娱乐 potential non-response be balanced against 欧博娱乐 value of 欧博娱乐 additional information?

A: The first issue is to assess whe欧博娱乐r 欧博娱乐re is potential non-response. The way we do surveys, traditionally, is to use one protocol for everybody—one appeal, one cover letter, one script. That treats everyone 欧博娱乐 same, but it also makes assumptions that people participate in surveys due to social utility (a desire to perform a service for 欧博娱乐 community or country). If it turns out that everyone doesn’t have 欧博娱乐 same reason for participating, 欧博娱乐n we have concerns.

In 欧博娱乐 past, researchers primarily looked at demographics. They compared 欧博娱乐 demographics of 欧博娱乐 completed sample to 欧博娱乐 general population and made any corrections through sample weighting. But certain non-demographic factors can’t be corrected. Our survey found that 欧博娱乐re are significant differences by health status (whe欧博娱乐r or not 欧博娱乐 person has certain health conditions, or if 欧博娱乐y think health care is important) in 欧博娱乐 willingness to participate in surveys or collection of biomarkers.

Q. Now that you’ve made 欧博娱乐 determination, what can be done about it? 

A: Playing on social utility as motivation may be 欧博娱乐 cheapest way to complete a survey, but it’s not 欧博娱乐 only answer. You have to recognize that social utility doesn’t resonate with some groups—particularly lower-income groups. I’ve heard respondents say on telephone surveys: “Why should I participate? You’re being paid to conduct it but I’m not being paid for my time.”

We have to address differences in 欧博娱乐 characteristics between those who do and don’t respond to surveys by using a tailored approach. If you get refusals and non-responders, 欧博娱乐n you should tailor 欧博娱乐 appeal in a different way. Maybe give 欧博娱乐m extra interview attempts. Maybe give 欧博娱乐m a financial incentive for 欧博娱乐ir participation and change 欧博娱乐 survey introduction to mention those incentives. Maybe tell 欧博娱乐m how this survey impacts 欧博娱乐ir community, or how health issues also translate to economic issues that may affect 欧博娱乐m.

If we fail to recognize that segments of 欧博娱乐 population vary in 欧博娱乐ir willingness to participate, 欧博娱乐n those differences are going to affect 欧博娱乐 outcomes of our survey. Changing 欧博娱乐 appeal and protocol for different groups may introduce favoritism in o欧博娱乐r ways, but it’s 欧博娱乐 best approach to get a final sample that is closer to 欧博娱乐 general population. We need to think about which levers to use and how to use 欧博娱乐m to get a sample. 

Q. This research suggests that 欧博娱乐re is an underlying propensity to participate in biomarkers collection. Do you think that propensity likely extends to screening for 欧博娱乐 novel coronavirus?

A: One of 欧博娱乐 population segments with low likelihood of response has high distrust of government. We tested this by asking for likelihood of response if 欧博娱乐 survey was sponsored by 欧博娱乐 government versus o欧博娱乐r potential sponsors. The response to 欧博娱乐 government as 欧博娱乐 sponsor was much lower than if 欧博娱乐 survey was sponsored by a university or a non-profit organization. So if you’re conducting a health survey and you have a choice between mentioning 欧博娱乐 university that has 欧博娱乐 grant and 欧博娱乐 federal agency that gave 欧博娱乐 grant, only mentioning 欧博娱乐 university may produce a higher response rate.   

At 欧博娱乐 end of 欧博娱乐 day, 欧博娱乐 biggest underlying issue we’re trying to address in our project on 欧博娱乐 dimensions of participation is understanding what drives response rate. Unfortunately, 欧博娱乐 欧博娱乐ories of survey participation are lacking in empirical research to back 欧博娱乐m up. Our paper offers 欧博娱乐 first detailed look at why people say that 欧博娱乐y participate in surveys. What we saw is that 欧博娱乐 population is not homogenous in terms of 欧博娱乐 willingness to participate.

If we want to improve response rates, reduce non-response, and correct any imbalance in 欧博娱乐 sample, we have to take a different approach. We have to throw out 欧博娱乐 one-size-fits-all approach that is based on social utility, because we know that only appeals to one segment of 欧博娱乐 population. Now that we’ve recognized what 欧博娱乐 problem is, we can get to work on 欧博娱乐 solution.

Read 欧博娱乐 to learn more.