Choosing an online sample provider is a research-design decision, not merely a procurement decision. The provider affects who can enter the study, which respondents complete it, how quickly quotas fill, and how confidently the resulting data can be interpreted.
The right questions therefore focus on evidence and operating controls. A credible provider should be able to explain its process in specific terms, distinguish owned sources from partner sources, and show how its approach changes for different audiences and markets.
Start with source transparency
Begin by mapping where respondents come from. Ask whether the provider uses proprietary communities, affiliate traffic, publisher networks, social recruitment, partner panels, or a blended model. Blended sourcing is not inherently a problem; undisclosed sourcing is.
A useful source description should explain:
- whether recruitment is active or passive;
- how participants give consent;
- what profile information is collected and refreshed;
- whether a respondent may appear in multiple sources;
- when third-party suppliers are introduced; and
- how source-level performance is monitored.
This matters because two respondents who appear demographically identical can have very different recruitment histories and survey-taking behaviour. Source transparency lets the research team interpret those differences and set appropriate controls.
For a structured supplier disclosure, compare the provider's answers with the ESOMAR 37 Questions. The value of that document is not the badge; it is the operational detail behind each answer.
Test feasibility before accepting the quote
Feasibility should connect the target definition to observable supply. A reliable estimate considers incidence, interview length, device requirements, geographic precision, quota structure, field period, language, and exclusion rules.
Ask the provider to state the assumptions behind its estimate. If an audience is difficult to reach, the response should identify the constraint and propose a defensible adjustment—not simply promise a larger number.
Also determine whether the provider checks for quota interaction. Several individually achievable quotas may become impractical when combined. Reviewing this before launch prevents rushed changes that weaken the design midway through fieldwork.
Review quality controls as a system
Quality is not one fraud score. It is a sequence of controls applied at appropriate points in the respondent journey. Review the provider's controls in three stages.
Before entry: identity signals, duplicate prevention, geographic checks, source risk, device assessment, and eligibility screening.
During the interview: speeding checks, attention measures, open-end review, inconsistent answers, suspicious patterns, and technical telemetry appropriate to the study.
After completion: cross-response analysis, source-level review, final disposition, replacement policy, and documented exclusions.
The provider should explain thresholds, escalation paths, and false-positive handling. A control that cannot be interpreted or challenged is difficult to govern. AIM's research quality approach describes how layered checks can be incorporated into the wider fieldwork workflow.
Check privacy, consent, and market coverage
Ask how participant information is collected, used, retained, transferred, and deleted. The answer should distinguish profile data, project data, survey responses, and technical data. It should also explain which parties act as controllers or processors where relevant.
Market coverage requires similar precision. “Global” may mean direct access in some countries and partner access in others. Confirm:
- which markets are served directly;
- where local partners are required;
- which languages and devices are supported;
- how local consent requirements are handled; and
- whether market-level quality reporting is available.
The objective is not to avoid partners. It is to understand the chain of responsibility and ensure the study's requirements remain enforceable throughout it.
Require fieldwork visibility
A provider should make fieldwork understandable while it is happening. Agree the reporting cadence, escalation contacts, quota views, rejection reasons, source labels, and incident process before launch.
Useful operational reporting includes starts, completes, terminations, quota-full outcomes, quality removals, average interview time, source mix, and material changes made during fieldwork. The exact dashboard is less important than consistent definitions and timely access.
For complex international work, assign decision rights in advance. Identify who may pause a source, relax a quota, extend the field period, replace completes, or approve a methodological change. This keeps commercial urgency from silently overriding the research design.
Use a repeatable scorecard
Compare providers using the same evidence categories rather than an overall impression.
| Area | Evidence to request | Warning sign |
|---|---|---|
| Sourcing | Source types, partner disclosure, recruitment method | Vague or changing source descriptions |
| Feasibility | Stated assumptions and quota interaction | Guaranteed volume without assumptions |
| Quality | Controls, thresholds, review and replacement process | One unexplained score presented as the full system |
| Privacy | Consent, roles, retention and deletion process | Generic policy with no project-level explanation |
| Operations | Reporting cadence, incident ownership, change log | No clear escalation route |
| Delivery | Final dispositions and source-level fieldwork report | Only a file of completed interviews |
A scorecard does not remove judgement. It makes that judgement explicit, comparable, and auditable. Weight the categories according to the study: a low-incidence medical audience, a national consumer tracker, and a rapid concept test do not carry the same risks.
The final decision should answer a simple question: Can this provider show, in a way your team can verify, how the requested audience becomes a defensible research dataset?