Matter Pods: Why Specialized Review Teams Beat Generic Review Pools

Ask a litigator what makes a document review go wrong and you will hear about deadlines, budgets, and platforms. Ask what made the last good review go well and the answer is usually a person. Someone on the team understood the case, caught the odd document early, and made sure every reviewer was making the same call on the same kind of document.

That observation sits at the center of how we think about review at Roshwin. Review is expensive, it is repetitive, and it is where most of the risk in a production hides. This piece looks at the evidence on why review quality varies, where subject matter expertise genuinely helps and where it does not, and a staffing model we call the matter pod.

Review Is Still Where the Money Goes

The most cited study of eDiscovery costs is still the 2012 report from the RAND Institute for Civil Justice. It found that review took about 73 percent of production spend, with processing at 19 percent and collection at 8 percent. Review has since lost some share as collection costs have grown. One industry reconciliation puts review at about 62 percent of task spend in 2025 and projects 52 percent by 2030.

Even at 52 percent, review remains the largest line. It is also the line that depends most on human judgment. When the biggest cost is also the most judgment-driven, a small gain in consistency is usually worth more than a small gain in speed. A review that is fast and inconsistent has to be partly done again.

Reviewers Disagree More Than Anyone Admits

The research on reviewer agreement is older than most people expect, and it has not been kind to the idea that trained reviewers produce identical results. Ellen Voorhees studied assessors who were trained experts and found that two of them overlapped on only about 45 percent of the documents they judged relevant. With three assessors, the overlap fell to about 30 percent. A well known 2010 study by Roitblat, Kershaw, and Oot put agreement between human review teams in roughly the 70 percent range.

The useful question is why. In 2012, Maura Grossman and Gordon Cormack examined disagreements from the TREC 2009 Legal Track and concluded that much of the inconsistency came from human error, not from documents that were truly borderline. That is good news in a practical sense. If disagreement were mostly ambiguity, nothing could be done about it. Because it is largely error, a better process can reduce it.

Error tends to come from predictable places. Reviewers learn the case from a written protocol alone. Instructions change mid-review and not everyone hears about it. A newer reviewer reads a request literally, while an experienced one reads it for what the requesting side actually needs. None of these are talent problems. They are setup problems.

What Expertise Actually Buys

It would be easy to overstate the case here, so it is worth being precise. Some of the research points the other way. A 2012 study by information scientist William Webber, widely discussed in the legal press, suggested that legally trained reviewers were not clearly more accurate than non-lawyers on first-pass responsiveness calls. The sample was very small, and practitioners were quick to point out that real reviews involve iterative instructions that a controlled study does not capture. Still, the result is a fair warning against assuming that a law degree alone makes a reviewer more accurate.

Our reading is that expertise pays off in four specific places. Routine responsiveness calls on clear instructions are not one of them.

So the case for specialization is not that specialists read faster. It is that a team which already understands the subject needs less instruction, asks better questions, and notices problems sooner.

AI Raises the Value of Specialists

It is tempting to assume that technology-assisted review and GenAI make this debate irrelevant. They do the opposite. In active learning, the reviewers' coding decisions train the model. If the reviewers are inconsistent, the model learns inconsistent lessons and ranks documents accordingly. Cormack and Grossman made a related point in 2017, arguing that human relevance labels are imperfect observations of a property that is sometimes ambiguous, and that this affects training, evaluation, and quality control.

GenAI shifts the problem rather than removing it. A model that makes the first pass still needs instructions, and those instructions are only as good as the understanding behind them. Someone who knows the subject has to write the prompt, test it against a sample, read the disagreements, and decide whether the model or the instruction is wrong. That is expert work. The reviewer's role moves from calling every document to validating the calls that matter, which makes domain knowledge more valuable, not less.

This is why we design pods to work across workflows. The same team that would run a linear review can run active learning or a GenAI first pass, because the part that determines quality, understanding the matter, does not change.

When the Stakes Spike: Second Requests

The argument is strongest in matters where timing and experience cannot be improvised. Consider the Second Request, the government's demand for additional information during a merger review under the Hart-Scott-Rodino Act. These are uncommon. The FTC and DOJ issued 41 in fiscal 2025, about 2.1 percent of notified transactions, and 59 in fiscal 2024, about 3.0 percent. Of the 2025 requests, 78 percent went to deals worth more than 500 million dollars.

That creates an awkward staffing problem. Volume is lumpy, the deal is high value, and the deadline is set by a regulator, not by the review team. Practitioner guidance is consistent that everyone on the team should have worked on multiple Second Requests before, and that the team should document its decisions in case the agencies later ask how the response was prepared. A generic pool assembled for the occasion cannot supply that. Experience of this kind has to exist before the request arrives.

The Roshwin Matter Pod

A matter pod is a standing group of reviewers and a lead attorney organized around two things: the type of litigation and the industry. An antitrust pod and an employment pod need different instincts. A financial services pod and a pharma pod need different vocabularies. Where the two meet is where a team becomes useful quickly. Our launch pods cover financial services litigation, pharma, antitrust, and Second Requests, and more will follow the same logic.

Pods are not tied to a platform. Roshwin does not sell a review platform. The pod works on the client's platform or the client's vendor's platform, and can advise on platform selection when asked. What the pod brings is people who already know the subject and a workflow built around them. An engagement runs in four steps.

1. 1

   Scope. The pod joins before the first document is reviewed. It aligns with counsel on the review protocol, the scope, and the definitions. Questions raised here cost almost nothing to fix. The same questions raised in week three cost a re-review.

2. 2

   Design. The pod recommends and deploys the workflow that fits the matter: linear review, active learning, a GenAI first pass, or a hybrid. AI is used where the client wants it and where it can be defended.

3. 3

   Calibrate. Reviewers are baselined on demonstrated accuracy before the main volume starts. Sampling is set by the size, scale, and complexity of the matter, then adjusted for subject matter, document type, and each reviewer's own performance. A small first batch is delivered within 24 to 48 hours of review starting, and the team ramps up only after the client gives a green light.

4. 4

   Ramp and report. The team scales to the deadline. Because the pod understands the subject, it can share trends with counsel while the review runs, including emerging themes, unusual documents, and instruction questions, instead of handing over a coding export at the end.

This is what we mean by going slow to go fast. The first days feel slower than simply starting. They are also the days that protect the deadline.

Do You Need a Pod? A Short Diagnostic

Before staffing the next matter, it helps to answer a few questions honestly.

Select each question your answer is yes to.

What This Means for Litigation Counsel and Legal Departments

Conclusion

Document review will keep getting more automated. The judgment around it will not. The teams that deliver consistent, defensible results are rarely the ones with the largest headcount. They are the ones that understood the case early and kept every reviewer making the same call. Roshwin built the matter pod around that idea: a team that learns the matter before it reviews, so that speed, once it arrives, is speed that holds up.