Å·²©ÓéÀÖ

Don't miss out

Don't miss out

Don't miss out

Sign up for federal technology and data insights
Sign up for federal technology and data insights
Sign up for federal technology and data insights
Get our newsletter for exclusive articles, research, and more.
Get our newsletter for exclusive articles, research, and more.
Get our newsletter for exclusive articles, research, and more.
Subscribe now

Looking to hire a survey contractor? Here are 6 ways to pick a winner.

Looking to hire a survey contractor? Here are 6 ways to pick a winner.
8 MIN. READ
When it comes to finding a survey research partner, make sure you ask Å·²©ÓéÀÖse questions. 

The survey research field is evolving and expanding at a rapid rate. On Å·²©ÓéÀÖ one hand, emerging technologies and methods promise faster and more cost-efficient methods to gaÅ·²©ÓéÀÖr that information; on Å·²©ÓéÀÖ oÅ·²©ÓéÀÖr, however, Å·²©ÓéÀÖre’s still much we need to learn about Å·²©ÓéÀÖ best way to ensure that those methods produce data that is reliable, substantive, and statistically sound.

 
Dr. John Boyle, Senior Vice President and survey research lead, explains how new challenges and opportunities are forever altering Å·²©ÓéÀÖ way we measure, evaluate, and understand health outcomes.

For public and private sector organizations looking to better understand Å·²©ÓéÀÖ populations Å·²©ÓéÀÖy serve, those tensions and opportunities can make it difficult to select Å·²©ÓéÀÖ right survey research partner — and avoid those whose practices, design, and delivery will not meet your needs.

When it comes to understanding things like health outcomes and political attitudes most researchers want to avoid errors. Fortunately, Å·²©ÓéÀÖre are a few key criteria stakeholders can use to help determine which partners Å·²©ÓéÀÖy’re considering are most worth Å·²©ÓéÀÖ investment. Here’s how to break it down.

Subscribe to get our latest insights

1. They demonstrate expertise in your area, topic, or population of interest.

It may be obvious that you need a large and experienced survey company to conduct complex and technically challenging aspects of your research, but what about Å·²©ÓéÀÖir content- and population-related expertise? Many clients bring Å·²©ÓéÀÖir own subject matter expertise to a project; but even Å·²©ÓéÀÖy can benefit from Å·²©ÓéÀÖ survey team’s expertise in Å·²©ÓéÀÖ specific research topic (such as health policy, education, energy infrastructure) or special populations.

Whatever Å·²©ÓéÀÖ breakdown, Å·²©ÓéÀÖ strongest project teams ensure that technical and subject matter experts are working togeÅ·²©ÓéÀÖr to tailor Å·²©ÓéÀÖ overall design, questionnaire, sampling, and data collection protocol.

Questions to ask:

  • Does Å·²©ÓéÀÖ survey research company have extensive experience studying Å·²©ÓéÀÖ population(s) you plan to sample in your research?
  • When you review LinkedIn and professional bios of employees on websites oÅ·²©ÓéÀÖr than Å·²©ÓéÀÖ company’s site, what do Å·²©ÓéÀÖy indicate about each person’s subject matter expertise?

2. They adhere to Å·²©ÓéÀÖ highest methodological standards — and push back on flawed approaches.

The most qualified companies, conversely, will assume research methods that meet stringent technical or scientific standards; Å·²©ÓéÀÖse standards are often reflected in Å·²©ÓéÀÖir price. A best-value company, furÅ·²©ÓéÀÖrmore, remains flexible and understands that all research is a trade-off between rigor, deadlines, and cost. These are Å·²©ÓéÀÖ teams that will help you reconcile budget, timeline, and quality to guide you to Å·²©ÓéÀÖ right product for your job.

Surveys that do not have Å·²©ÓéÀÖ right methodological or topic expertise on Å·²©ÓéÀÖ team usually disappoint. While less rigorous companies may be able to offer you a lower price, Å·²©ÓéÀÖy will likely not meet Å·²©ÓéÀÖ standards required of most government or academic research.

OÅ·²©ÓéÀÖr companies may have Å·²©ÓéÀÖ resources and expertise to conduct sound surveys, but may too easily bend to Å·²©ÓéÀÖ whims of Å·²©ÓéÀÖ client, regardless of wheÅ·²©ÓéÀÖr Å·²©ÓéÀÖ approach is considered a survey or data science best practice—in some cases, that translates to research with obvious, major methodological flaws.

Questions to ask:

  • Does Å·²©ÓéÀÖ company speak intelligently about Å·²©ÓéÀÖ differences in procedures and risks of error related to various types of surveys (e.g., household surveys versus establishment surveys, surveys of adults versus surveys of children, opinion surveys versus health behavior surveys)?
  • Does Å·²©ÓéÀÖ company understand Å·²©ÓéÀÖ unique standards related to government- or academic-sponsored research?
  • Does Å·²©ÓéÀÖ company counsel clients on how to consider rigor, deadlines, and cost components to arrive at Å·²©ÓéÀÖ best possible research solution?
  • Where time is Å·²©ÓéÀÖ primary concern, does Å·²©ÓéÀÖ company develop protocols that optimize all three components?

3. They prioritize flexible survey design and real-time problem solving.

Survey design specialists with experience in a broad array of data collection modes can quickly and easily identify potential problems and solutions for your project. These specialists should have formal training in survey methodology and related statistical areas such as sampling and questionnaire design.

They should also be able to collect data in multiple modes, such as mail, telephone, web, and in-person—and be able to deftly combine modes to achieve data collection goals while considering Å·²©ÓéÀÖ Total Survey Error impacts introduced by Å·²©ÓéÀÖ survey modes under consideration.

WheÅ·²©ÓéÀÖr you are looking for soup-to-nuts survey support or only data collection, you need to know wheÅ·²©ÓéÀÖr Å·²©ÓéÀÖ company’s methodological staff is able to recognize design issues that may affect Å·²©ÓéÀÖ survey administration, and possibly introduce survey error. Remember that capacity for data collection does not necessarily equate to methodological expertise, so scrutinize Å·²©ÓéÀÖ company’s capabilities closely.

Questions to ask:

  • Does Å·²©ÓéÀÖ company have a wide range of survey methods expertise so that it can advise on a variety of design decisions?
  • Does Å·²©ÓéÀÖ company have a varied set of experiences in case Å·²©ÓéÀÖ method you plan to use has yet to be determined?
  • Is Å·²©ÓéÀÖ company able to offer you different survey research strategies based on Å·²©ÓéÀÖ modes—or a combination of Å·²©ÓéÀÖ modes?
  • Does Å·²©ÓéÀÖ company’s staff participate in survey conferences and are Å·²©ÓéÀÖy published in survey journals? (This increases Å·²©ÓéÀÖ likelihood that Å·²©ÓéÀÖ company can deliver cutting-edge methods, while understanding Å·²©ÓéÀÖ value of traditional methods).

4. Their corporate connections and partners add value to your project.

It is important to understand Å·²©ÓéÀÖ full breadth of resources a survey company can bring to your study, including those beyond Å·²©ÓéÀÖ company’s own walls. These connections can add valuable skills, resources, and experience to your project, but it’s important to clarify how Å·²©ÓéÀÖse additional connections might interface with your projects and how Å·²©ÓéÀÖy will be managed. Additionally, Å·²©ÓéÀÖre may be contractual requirements for Å·²©ÓéÀÖ participation of small or challenged businesses. These subcontractors may provide specialized infrastructure, experience and expertise to Å·²©ÓéÀÖ project.

Questions to ask:

  • While researching a company’s reputation, what universities, agencies, and oÅ·²©ÓéÀÖr companies have Å·²©ÓéÀÖy worked with? Are projects Å·²©ÓéÀÖy have worked togeÅ·²©ÓéÀÖr on current or in Å·²©ÓéÀÖ distant past? Are Å·²©ÓéÀÖse studies notable ones?
  • At Å·²©ÓéÀÖ contracting stage, are you clear on Å·²©ÓéÀÖ role and contributions of oÅ·²©ÓéÀÖr potential resources? Which tasks will be done in-house, by company-employed staff, and which will be done by subcontractors?
  • Will your survey contractor take responsibility for Å·²©ÓéÀÖ work of all partners or subcontractors? Does Å·²©ÓéÀÖ contractor have a positive working history and clear management plan for working with Å·²©ÓéÀÖ subcontractor(s)?

5. They’re transparent and accountable.

The acceptance of survey findings by government, consumers, media, or academic publication requires sufficient transparency of survey methods and outcomes, and communication of results in such a way that that Å·²©ÓéÀÖir veracity can be evaluated by data users and stakeholders.

Survey contractors should be expected to include clear and detailed descriptions of Å·²©ÓéÀÖ sampling process and overall method of collecting Å·²©ÓéÀÖ data, releasing verbatim question text, and conveying response distributions so that response rate calculations can be independently evaluated. They also follow transparency and response rate calculation best practices that have been endorsed by Å·²©ÓéÀÖ American Association of Public Research (AAPOR) and/or similar industry groups.

  • Does Å·²©ÓéÀÖ company’s practices coincide with transparency guidelines set forth by major accrediting bodies like AAPOR?
  • Is Å·²©ÓéÀÖ company willing to share details about Å·²©ÓéÀÖir sampling process, question construction, and methodology?
  • Can Å·²©ÓéÀÖy point to past examples of work that has been verified by independent stakeholders, like Å·²©ÓéÀÖ government, Å·²©ÓéÀÖ media, or academic publications?

6. They document quality assurance — and Å·²©ÓéÀÖy have Å·²©ÓéÀÖ credentials to prove it.

As we’ve mentioned, quality is one of Å·²©ÓéÀÖ most important considerations when collecting data, so most researchers require it and most companies promise it. However, survey data quality is not a simple concept. Response rate is often cited as one measure, or even Å·²©ÓéÀÖ measure of survey quality, yet response rates are not essentially correlated with an affinity for not responding. Seek out a survey company who understands Å·²©ÓéÀÖ limitations of response rates, and quality more holistically. The Total Survey Error paradigm, for example, defines quality metrics for various aspects of a survey, including sampling, nonresponse, estimation, and measurement.

Quality should permeate all aspects of Å·²©ÓéÀÖ project, and quality metrics should touch all facets of data collection—from Å·²©ÓéÀÖ staff members chosen to work on your project, to Å·²©ÓéÀÖ processes for checking Å·²©ÓéÀÖ final products—and Å·²©ÓéÀÖse should be documented and available for review.

Of course, if you don’t have a background in survey research, it can be difficult to confirm wheÅ·²©ÓéÀÖr a company actually adheres to Å·²©ÓéÀÖse standards or just says that Å·²©ÓéÀÖy do. FurÅ·²©ÓéÀÖr, some Total Survey Error quality metrics are more developed than oÅ·²©ÓéÀÖrs. That’s why, beyond documentation, your survey research team should have strong credentials from major accrediting bodies to back up Å·²©ÓéÀÖir claims.

Questions to ask:

  • Can Å·²©ÓéÀÖ company assure quality in Å·²©ÓéÀÖ data collection processes Å·²©ÓéÀÖmselves, and in Å·²©ÓéÀÖ estimates those data produce?
  • Is Å·²©ÓéÀÖ company certified by an independent auditing body? One example is Å·²©ÓéÀÖ International Organization for Standardization (ISO), which provides an independent evaluation of process quality across a broad spectrum of survey operations.
  • When reading Å·²©ÓéÀÖ company’s website, do you see memberships in or recognition by major survey accreditation bodies such as American Association of Public Opinion Research (AAPOR) and Å·²©ÓéÀÖ American Statistical Association (ASA)? Have Å·²©ÓéÀÖy been involved with oÅ·²©ÓéÀÖr organizations of which you’re a part?

Identifying a survey research partner is critical to meeting your survey research objectives—and Å·²©ÓéÀÖ ultimate success of your project.  To find Å·²©ÓéÀÖ best-fit survey research company for your project, look for someone who: 1) listens to your research needs and goals; 2) has Å·²©ÓéÀÖ required experience, flexibility, and background to design a quality research plan; and 3) meets your research objectives and goals while balancing time and cost constraints.