Methodology – QRC Survey Sampling, Validation & Acceptance Framework

Framework Overview

Three systems. Five documented subsystems.

Our methodology divides across two upstream systems — Sampling and Validation — that feed into a third: the Quality Acceptance System. Each operates on documented logic, not heuristics.

Sampling System

Governs who gets invited, from which recruitment channels, and under what quota constraints.

Sampling logic

Deduplication logic

Validation System

Runs multi-layered quality checks across entry, in-survey, and post-completion stages to evaluate respondent integrity.

Identity verification

Behavioral validation

Quality Acceptance System

Applies final acceptance criteria against the validated dataset before delivery. No response enters your data without passing this stage.

Acceptance criteria

200+

Quality checks

5

Documented subsystems

7

Recruitment channels

98.4%

Acceptance rate on delivery

Subsystem 1 — Sampling System

Sampling Logic

Who gets invited into a study, from which channels, and under what quota constraints. These decisions happen before any respondent sees your survey.

Probabilistic Quota Management

Quotas are applied against verified profile segments — not raw panel counts. A "senior decision-maker" quota fills only from profiles with corroborated seniority signals, not self-reported titles. This adds 8–14 hours to feasibility assessment but eliminates a major source of segment contamination.

Multi-Channel Recruitment

We maintain seven distinct recruitment channels and disclose which contributed to your final sample on request. Panels assembled from a single recruitment channel carry a homogeneity risk that blended-source panels don't — we document channel distribution in every quality report.

Incidence Rate Pre-Qualification

Before you finalize your screener, we run a feasibility pass against current panel segments to estimate realistic incidence rates. Most providers withhold this until after launch. We give you the estimate upfront because it changes study design decisions in ways that matter for your timeline and budget.

Survey Frequency Capping

Panel members cannot participate in more than three studies per rolling 90-day window. For specialist segments — healthcare, C-suite, regulated industries — this cap drops to two. Over-surveyed respondents develop habitual response patterns that are difficult to screen out after the fact.

Subsystem 2 — Sampling System

Deduplication Logic

The same respondent attempting to complete a study twice is one of the most common quality failures in panel research — and one of the easiest to overlook.

Our deduplication layer operates on five distinct signals, each weighted differently based on study type. No single signal is treated as definitive — the system scores across the full signal set.

Device fingerprint matching

Browser environment hash compared against all prior completions in the same study window. Partial matches trigger review rather than automatic rejection.

Profile-level panel ID reconciliation

Cross-referencing against our respondent registry to identify accounts that share attributes with previously terminated participants.

Network-level deduplication

Multiple completions originating from the same IP subnet within a short time window are flagged, with household exceptions applied for consumer studies.

Open-end similarity across completions

Verbatim responses with high cosine similarity to existing completions indicate the same respondent re-entering under a different identity — caught at post-completion stage.

Full Validation Framework →

Data deduplication and identity reconciliation interface

Subsystem 3 — Validation System

Identity Verification

The eleven checkpoints that run before a respondent answers your first question. This is where the most consequential filtering happens.

Deterministic identity matching

Self-reported demographic data is the weakest link in any panel. Our verification layer cross-references profile data against four external registries — professional licensing bodies, company registration databases, firmographic APIs, and corroborated professional signals — before a profile is cleared for any study.

We don't verify every attribute for every respondent. We verify the attributes that matter for your specific quota cells. A study targeting hospital pharmacists requires pharmacy board verification. A study targeting CFOs at $50M+ companies requires revenue-band corroboration. The verification scope is set at study configuration, not applied uniformly.

Pre-admission checkpoints (11 total)

Geo-IP vs. profile address match Always on

VPN / proxy / datacenter IP detection Always on

Device fingerprint deduplication Always on

Known bad-actor ID registry match Always on

Profile freshness check (180-day maximum) Always on

Study frequency cap verification Always on

Firmographic attribute corroboration Study-specific

View Quality Framework →

Subsystem 4 — Validation System

Behavioral Validation

Seven checkpoints that run during active survey completion. These catch respondents who passed pre-admission but are not engaging genuinely with your questions.

01

Per-Question Timing Analysis

Each question's completion time is compared against a distribution built from verified-quality completions of that question type. Responses more than 2.5 standard deviations below median are flagged for review — not automatically rejected. Some legitimate respondents are fast readers; the timing signal is one input, not a verdict.

02

Response Pattern Detection

Straight-lining (selecting the same scale position repeatedly) and seesaw patterns (alternating between extremes) are scored on a 0–100 suspicion index. Responses above 72 enter manual review. We publish threshold calibration methodology on request.

03

Topic-Specific Attention Items

Embedded validation questions use known-answer items specific to the study topic — not generic "select the third option" instructions. A C-suite technology study might include a question with a verifiable industry fact as the correct response. Generic attention checks are easier to game; topic-specific ones require genuine engagement.

04

Real-Time Inconsistency Flagging

Responses to screener attributes are compared against responses to substantive questions mid-survey. A respondent who qualified as a CFO in the screener but answers operational-level questions inconsistently with that role generates a consistency flag that carries forward to the acceptance stage.

Subsystem 5 — Quality Acceptance System

Acceptance Criteria

The final gate before data delivery. Every response that passed the previous four subsystems is evaluated against acceptance criteria before entering your dataset.

What "accepted" actually means

Most quality frameworks stop at detection — flagging suspicious responses and leaving clients to decide what to do with them. Our acceptance criteria define a positive standard: what a response must demonstrate to enter the delivered dataset, not just what it must avoid.

Acceptance operates on a composite score, not a pass/fail for each individual checkpoint. A response that fails one behavioral check may still be accepted if the remaining signals are strong. The composite threshold is calibrated by study type and reviewed quarterly.

Open-end semantic scoring

Verbatim responses are scored for semantic relevance, minimum meaningful length, and within-batch similarity. AI-generated text is assessed separately and disclosed, not automatically excluded.

Cross-response consistency check

Demographic and behavioral responses are cross-checked for internal consistency. Contradictions between screener-qualified attributes and substantive answers trigger a final review.

Composite score threshold

A weighted aggregate across all validation signals determines final acceptance. The threshold and weight distribution are included in the quality audit report delivered with your data.

Manual review for borderline cases

Responses that score within 5 points of the acceptance threshold go to a human reviewer before the dataset closes. The reviewer's decision, with rationale, is logged in the audit trail.

Quality Control Detail →

Data acceptance criteria and quality scoring dashboard

Audit Report

What ships with your data

Every delivered dataset includes a quality audit report documenting each subsystem's output. You don't need to ask for it — it arrives with the data file.

Checkpoint-level output

Count filtered at each of the 23 checkpoints, by stage

Quota performance vs. target

Achieved vs. specified by segment, with any deviation flagged

Composite score distribution

Score histogram for accepted responses; threshold noted

Flagged-but-retained cases

Scoring rationale for responses that triggered a flag and were accepted anyway

Channel composition

Recruitment channel breakdown for your delivered sample

Open-end scoring detail

Per-response semantic scores with AI-text assessment where applicable

Request a Sample Audit Report

Common Questions

What researchers ask about the framework

Yes. We provide a formal methodology citation document on request, formatted for APA, AMA, and Vancouver styles. It describes recruitment sources, verification protocols, and acceptance criteria in language that peer reviewers accept. Clients have cited this document in submissions to Nature, JAMA, and several management journals.

Mid-survey terminations are recorded with a reason code. Respondents are notified of the termination but not given the specific reason (to prevent pattern-learning). Their profile is flagged for the relevant study type, which affects future invitation eligibility. This creates a behavioral feedback loop that financial incentive structures alone don't produce.

All EU panel members have provided explicit, granular consent under GDPR Article 6(1)(a). Consent records are maintained with timestamp, IP address, and specific consent language version. Data Processing Agreements (DPAs) are available for every client study. We are registered with the relevant supervisory authorities and maintain a data protection officer on staff.

Based on our manual review sample — we manually review 8% of all flagged responses — our false positive rate is approximately 4.2%. About 4 in every 100 filtered responses are legitimate respondents that triggered a filter incorrectly. This is a known trade-off. We err toward exclusion and always field slightly above target sample size to account for it. A 0% false positive rate would require loosening thresholds that currently catch real quality issues.

Yes, with a caveat. You can access a live data stream through the client portal, but we apply a 15-minute processing delay to ensure preliminary validation has run before data is visible. Exporting unprocessed raw data during fielding is technically possible — but we'll tell you directly if you request it that we advise against it.

Network-level deduplication applies household exceptions automatically for consumer studies — multiple completions from the same residential IP are permitted up to the household size cap (typically two). For B2B studies, the threshold is lower and requires firmographic differentiation between participants sharing a network address.

Five subsystems. One documented framework.