Playbooks That Learn: Contract Review With Falling Escalation Rates

Every legal department knows the feeling. The contracts arrive faster than the team can read them. Sales wants an answer by Friday. Procurement wants the vendor onboarded this month. Somewhere in the pile is the one clause that really matters, and it looks almost exactly like the forty clauses that did not.

Most of what a reviewer does on a typical contract is not new thinking. It is the same handful of questions, asked again, against a standard that lives partly in a document and partly in a senior lawyer's head. This piece looks at what the evidence says about how repetitive contract review really is, a finding about reviewer consistency that surprised us, and a model we use at Roshwin for building a playbook that gets better with every contract it sees.

The Same Few Terms, Year After Year

World Commerce and Contracting (WorldCC) has surveyed contract professionals for years about which terms get negotiated most. The answer barely moves. In the 2024 report, limitation of liability, price, and indemnification kept their places at the top, followed by termination, payment, scope, warranty, intellectual property, delivery, and liquidated damages.

WorldCC has made the same observation before: the list stays stable, and negotiation time is dominated by risk allocation rather than value creation. For anyone designing a review process, that stability is good news. It means a large share of review is a finite set of recurring questions, and recurring questions can be answered in advance.

What One Contract Costs to Review

The numbers on review effort are worth looking at, with some care. In 2024, researchers at legal technology company Onit compared junior lawyers, senior lawyers, outsourced reviewers, and large language models on the same ten procurement contracts, all checked against a playbook-style list of standards. Senior lawyers averaged about 43 minutes per contract and junior lawyers about 56. Outsourced reviewers took about 200. The best-performing model took under five minutes.

The cost gap was larger still. Human review ran from roughly 37 to 76 dollars per contract, while model costs came to cents. Three cautions apply. The study was written by a vendor, the sample was small, and the model costs leave out setup and oversight. The authors report about sixteen hours to build and validate the model instructions, which they note is similar to the time spent briefing human reviewers. Treat the figures as direction, not as a quote. The direction is still clear: first-pass review against a clear standard is slow when done by hand and fast when it is not.

On accuracy, the picture was closer. At determining whether a contract met each standard, the leading model scored about the same as outsourced reviewers and slightly ahead of junior lawyers. At pointing to the exact clause, outsourced reviewers led, and the best model came close behind. Across all of the groups, errors leaned toward missing an issue rather than inventing one, often by overlooking a small wording difference, such as a standard that required notice to be in writing.

The Finding That Surprised Us

The study also measured how often reviewers agreed with each other. Agreement among the senior lawyers was the lowest of any group, at 0.72 on a scale where 1.0 is perfect. Junior lawyers scored 0.77. The outsourced reviewers, working from the same written checklist, agreed with each other perfectly.

It would be a mistake to read that as saying outsourced reviewers are better lawyers. The senior lawyers set the answer key by majority view, and nothing in the study shows that perfect agreement means perfectly right. What it shows is consistency. Experienced lawyers applying judgment from experience diverge. People applying a written standard converge. A playbook is how you turn the first kind of review into the second kind without losing the judgment, because the judgment goes into writing the standard.

There is a catch, and it matters for how a playbook should be run. A group that agrees perfectly can also share a blind spot. If the playbook misses a nuance, everyone who follows it will miss that nuance together. A playbook needs owners, sampling, and a way to learn.

What a Playbook Actually Is

A playbook records, for each clause that matters, what the business wants, what it will accept instead, and what it must not accept without asking. Every clause in a contract then lands in one of three places: accept, fallback, or escalate.

Two design choices make the difference. First, the playbook is built from past negotiations, not from policy documents. Past deals show what the business actually accepted under real pressure, which is often different from what the policy says. Second, the client's legal team approves it. The playbook is the legal team's judgment, made in advance. Roshwin does not give legal advice. We work as the execution arm of the legal team, applying the standard the team has set and sending back anything it does not cover.

Playbooks That Learn

Most playbooks fail slowly. The first version never covers everything, so exceptions go to a lawyer, the lawyer decides, and the decision disappears into an email thread. The next time the same issue arrives, it is escalated again. The playbook has not changed. Only the lawyer's workload has.

A playbook that learns closes that loop. Every escalation is a question the playbook could not answer. The client's legal team answers it once. We then record the answer as a new fallback or a new trigger, with the legal team's approval, so the next contract with the same issue is handled without anyone being asked.

The result is a curve that falls quickly at first and then flattens. It never reaches zero, and it should not. New deal types and new law keep producing real exceptions that deserve a lawyer's attention. The point is that the lawyer's attention goes to the contracts that need it, and the number of times the team is asked the same question trends toward none.

The loop produces a second benefit that is easy to overlook. Every escalation and every counterparty pushback is recorded, so the playbook becomes a source of data. Which clauses do counterparties resist most? Where do the business's own templates invite negotiation? That information is what lets a legal team standardize its paper upstream instead of fixing the same problem downstream, contract after contract.

Where AI Fits and Where It Does Not

AI belongs in the first pass against the playbook, which is the part of review that is repetitive and rule-shaped. The evidence for that is reasonably consistent. In the 2025 Vals Legal AI Report, which compared legal AI tools with a lawyer control group, the tools beat the lawyers on tasks such as document analysis and data extraction. The lawyers stayed ahead on redlining, the work of proposing contract changes, with a baseline score of about 80 percent.

That split maps well onto a playbook workflow. Let AI read the contract against the standard and flag each clause as accept, fallback, or escalate. Keep people on the redline and on every escalation. Where we use AI, reviewers check its calls and a sample is audited, because the Onit study shows that both people and models miss small wording differences. The goal is not to remove the reviewer. It is to put the reviewer's attention on the clauses where it changes the outcome.

This works for both sides of the table. A sales team reviewing customer paper wants speed and a clear sense of which concessions are safe to give. A procurement team reviewing vendor paper wants protection and a clear sense of which terms are not negotiable. The positions differ, but the loop is the same.

How to Tell Whether It Is Working

A playbook that learns should be measured like a process, not filed like a document. Three measures tell most of the story.

When escalation falls and standardization rises together, the playbook is doing its job. When escalation falls but standardization does not, something else may be happening. The team could be accepting terms quietly that deserve a second look, or the playbook may have drifted away from what the legal team would actually approve. That is why a regular review of sampled contracts belongs in the routine, not only in the setup.

Does Your Review Process Need a Playbook That Learns?

A few honest answers will tell you where the effort is going.

Select each question your answer is yes to.

What This Means for Legal and Commercial Leaders

Conclusion

Contract review will always need judgment. It does not need the same judgment applied again for every contract that looks like the last one. A playbook that learns lets the legal team make each decision once and then spend its time on the contracts that are genuinely new. Roshwin builds and runs these playbooks for sales and procurement teams. The standard comes from the client's lawyers, the first pass is fast, and the escalations get rarer, which is how you can tell it is working.