Quality Control for Document Review: Understanding QC, QA, and Quality Testing

Effective quality control catches discovery failures before they become problems — but only if it's designed into the review process rather than treated as an afterthought.

This article covers the three disciplines that together constitute a complete quality program — quality control, quality assurance, and quality testing — and explains how to apply each one across the full document review lifecycle to protect against privilege waiver and costly rework.

👉Looking for worry-free solution to document review? Learn more about DISCO’s Managed Review Services.

Defining quality control for document review

The terms quality control, quality assurance, and quality testing are often used interchangeably, yet they represent fundamentally distinct activities with different timing, scope, and operational purposes. Mistaking one for the other creates blind spots that compromise the integrity of a review. Establishing clear definitions and boundaries across all three is the essential first step toward building a review program that is efficient, consistent, and genuinely defensible.

Quality control (QC) is reactive and continuous

Quality control is the active, ongoing process of identifying and correcting errors in reviewer coding decisions. In document review, QC typically involves a senior reviewer or review manager sampling documents that have already been tagged — for relevance, privilege, responsiveness, or issue coding — and checking whether those decisions align with the review protocol.

It catches problems that have already occurred. What distinguishes it from a final audit is that it happens continuously throughout the review, not just at the end. The goal is to surface inconsistency early enough to correct it before errors compound across thousands of documents.

⚙️Technology that makes a difference: DISCO’s quality control feature ensures accuracy and efficiency with the results of your review process, keeping your productions reliable and consistent.

Quality assurance (QA) is proactive

Quality assurance operates at the process level rather than the document level. Where QC asks, Did the reviewer tag this document correctly, QA asks, Is the process likely to produce correct tags consistently?

QA activities include protocol design, reviewer training, workflow audits, and ongoing calibration sessions that keep the team aligned as the review evolves. 

It prevents errors rather than catching them after the fact. A team with strong QA processes produces fewer QC errors to catch.

Quality testing is evaluative rather than corrective

Quality testing validates that the review as a whole has met its goals — typically by statistically sampling completed work to measure error rates, recall, and precision across the full document set. 

In Technology-Assisted Review (TAR) workflows, quality testing is a defined, formal step. The review isn't considered complete until testing confirms the model has reached an acceptable performance threshold.

Quality testing answers the question, Did we do a good enough job? and provides the documented evidence needed to defend that conclusion.

Building QC into your document review strategy

Quality control starts before the first document is opened. Here’s how:

Review protocol design

The first step of quality control is to design a review protocol that defines: 

• Explicit Decision Trees:
  
  • What's relevant and what isn't
  • How privilege is determined
  • What the issue tags mean and when they apply
• How QC will be conducted: who samples, how frequently, and what happens when errors exceed an acceptable threshold
• A documented mechanism for updating the protocol as the review evolves, ensuring all reviewers operate under updated instructions.

A weak protocol produces weak reviews. When reviewers are left to interpret ambiguous criteria individually, inconsistency is the predictable result. 

A strong protocol is specific enough that two reviewers, working independently, would reach the same coding decision on the same document. Achieving this standard requires more upfront investment in protocol design, but it dramatically reduces the volume of downstream QC remediation and cost.

Relevance and privilege criteria

Once criteria are defined in the protocol, teams should conduct calibration exercises that walk reviewers through edge cases and borderline documents to align reviewer judgement.

Privilege deserves particular attention. It carries the highest stakes, and errors in either direction are costly: Over-designation can delay production or invite challenge while under-designation can risk disclosure. Clear, specific privilege criteria — with examples — reduces the judgment variation that creates exposure.

ESI agreements

It’s also important to negotiate the right ESI protections before review begins. 

For example, a clawback agreement under Federal Rule of Evidence 502(d) provides a safety net against inadvertent privilege waiver, which means teams can move through review at the pace the matter requires without treating every privilege determination as irreversible. Without those guardrails in place, the cost of a QC miss is significantly higher.

QC across the document review lifecycle

Quality control looks different at each stage of discovery, and neglecting any one stage may create risk that downstream QC checks can't fully recover.

Collection

QC at the collection stage focuses on completeness and defensibility. 

• Did the collection capture all relevant custodians? 
• Were the date ranges and search parameters applied correctly? 
• Were any sources overlooked or improperly excluded? 

Errors at collection are the hardest to detect later because the missing data simply isn't there to find. A collection verification process — confirming that all identified sources were preserved and collected — is the most effective way to prevent downstream gaps.

Processing

Processing QC involves confirming that data was ingested correctly: 

• File counts are consistent
• De-duplication was applied appropriately
• Metadata was preserved
• Document families are intact 

A processing error, such as misapplied de-NIST filters, dropped attachments, or corrupted native files, can affect thousands of documents and may not be visible until production. Auditing the processing report before review begins is a standard and necessary step.

First-level review

First-level review is where the volume is highest and QC is most operationally demanding. Reviewers are making rapid decisions across large document sets, which means error rates are a real and expected variable. 

QC at this stage typically involves sampling a percentage of documents from each reviewer's work — both tagged responsive and non-responsive — and comparing those decisions against the protocol. Sustained inter-rater disagreement is a signal that the protocol needs clarification.

🔑Get the guide: How to Set Up and QC Your First GenAI-Powered Document Review

Second-level review

Second-level review catches what first-level missed and resolves documents that were flagged as uncertain or potentially privileged. 

QC at this stage focuses on confirming that elevated documents have received appropriate senior review and that privilege determinations are well-documented. This is also the right moment to run a privilege log quality check, since privilege log accuracy is frequently a source of dispute.

📚Additional reading: ESI Review Protocols Are Evolving with GenAI — So Should You

Production

Production QC is the last line of defense. Before any data leaves the organization, teams should confirm that:

• Production specifications match what was agreed upon
• No privileged documents are included in the production set
• The production is complete relative to what was requested

A pre-production quality check — often called a "QC of the QC" — is standard practice in high-stakes matters for exactly this reason.

QC technology for document review

The volume and complexity of modern document review have made technology an essential part of any quality control program. Manual QC processes remain necessary, but they can only cover so much ground. 

The right technology — AI in particular — enables teams to identify coding inconsistencies, validate review decisions, and document the QC process itself at a scale that manual oversight alone cannot reach.

🔍Dive deeper: Access the webinar, Using GenAI for Doc Review: Skills and Best Practices, for best practices on quality control and what to do when you and the AI don’t agree.

AI-powered conflict detection

AI can evaluate an entire document population for coding consistency — something no manual sampling process can match. It continuously compares its predictions against reviewer decisions, concentrating on the documents where human judgment and model predictions diverge. 

When those conflicts surface, they are flagged and prioritized for supervisory review, shifting QC from a sampling exercise, where errors are caught by chance, to a targeted one, where attention goes exactly where it's needed.

⚙️DISCO’s platform does exactly this. It surfaces these conflicts through visual breakdowns of AI predictions versus actual tagging choices, giving review managers a precise, real-time view of where coding consistency may be breaking down.

AI-assisted review validation

Beyond conflict detection, AI can play a direct role in validating the accuracy of review work at both the individual document level and across large document sets. 

Cecilia Auto Review, for example, is designed not only to accelerate first-pass review but to check the accuracy of work performed by human reviewers, providing a written justification for each tagging decision that can be audited and defended. 

For more granular validation, Cecilia's Single-Doc Q&A allows review managers to interrogate individual documents and verify that coding decisions hold up against the review protocol.

Built-in QC workflows

Most ediscovery platforms today include native quality control functionality that formalizes the sampling process without requiring teams to build their own workflow from scratch. 

These tools allow review managers to define a search query, select the review decisions to evaluate, and set a statistically meaningful sample size. Overturns are tracked automatically as trusted reviewers work through the sample, creating a documented record of where coding decisions were changed and why. 

The most sophisticated technology goes a step further, allowing teams to extrapolate sample results to the full document population. This allows them to turn a targeted QC exercise into a statistically grounded assessment of overall review quality.

Human oversight, auditing, and accountability

The most defensible quality control programs combine automated tools with clearly defined human accountability structures.

Multi-level review structures

Multi-level review — where first-level reviewers make initial coding decisions that are then checked by more senior reviewers — is the foundational architecture of QC in document review. The structure creates systematic oversight without requiring every document to receive senior attention. 

What makes multi-level review work is clear criteria for escalation. Reviewers need to know what kinds of uncertainty or complexity should move a document up the chain.

Reviewer performance monitoring

Real-time monitoring of reviewer metrics provides an early warning system for QC problems, such as throughput, inconsistency rates, and privilege designation percentages. 

A reviewer whose privilege rate is an outlier, or whose decisions systematically diverge from the model's predictions, warrants targeted coaching and additional sampling before the problem spreads across a large volume of work.

Quality testing vs. process auditing

Two important QC checks are quality testing and process auditing. 

• Quality testing validates outcomes, whether the review produced accurate results. 
• Process auditing validates execution, whether the review followed its defined procedures. 

In high-stakes or high-scrutiny matters, documentation of both types of review provides the most complete basis for defending the process if it's challenged.

Training and SOPs

Even well-designed protocols can fail if they’re applied incorrectly. 

• Pre-review training should walk reviewers through the specific criteria that govern this matter, with worked examples covering common edge cases. 
• Standard operating procedures (SOPs) should be written and accessible.

Documentation and defensibility

"We had good reviewers" is not a substitute for documented quality control procedures. Courts and opposing counsel expect to see evidence that the review was conducted with reasonable care. 

The QC process paper trail should include: 

• What was sampled
• What errors were found
• What remediation was taken
• Who made those decisions

📖Get the ebook: How to Use Generative AI for Document Review

Managing risk and cost

The case for investing in quality control comes down to risk and cost — and they move in opposite directions. Teams that build QC into their review architecture from the start catch errors before they compound into remediation, re-review, or privilege disputes. 

Implementing structured quality control delivers clear benefits across every stage of the review process:

• Consistent coding reduces the inter-reviewer variation that creates exposure on privilege determinations. 
• Documented QC procedures provide the audit trail that makes a review defensible if it's challenged. 
• Early detection of misclassifications minimizes expensive re-review cycles and prevents non-responsive data leaks.

Furthermore, AI-assisted review changes the economics of the whole process — teams using AI-powered tools have seen review time cut by 50% or more. That means QC no longer has to compete with speed for priority. At up to 32,000 documents per hour, Cecilia Auto Review can validate human review work at a scale that would have been unimaginable in a manual process.

See how DISCO puts quality control into practice

DISCO's ediscovery platform is built for the kind of systematic, defensible review quality control. It equips  review teams with the infrastructure to move quickly without sacrificing accuracy.

⚙️Explore DISCO Ediscovery to see how the platform supports quality control at every stage of the review lifecycle.

⚙️See Auto Review in action to learn how AI-powered document review can reduce review time and strengthen your QC documentation.

⚙️Schedule a demo to talk through your review challenges with the DISCO team.