Citing TAR Research, Court OKs Production Using Random Sampling

Catalyst_Court_OKs_Production_Using_Random_SamplingCiting research on the efficacy of technology assisted review over human review, a federal court has approved a party’s request to respond to discovery using random sampling.

Despite a tight discovery timeline in the case, the plaintiff had sought to compel the defendant hospital to manually review nearly 16,000 patient records.

But the hospital presented evidence that manual review and redaction would require nearly 9,000 hours of work and cost in excess of $230,000. It asked the court to allow it to instead produce a random sampling of 252 patient records.

Granting the defendant’s request, U.S. Magistrate Judge Teresa J. James of the U.S. District Court in Kansas concluded that the request was justified by the time and expense that a full production would require.

Judge James further found that a random sampling could produce a more accurate result.

If Defendant were to employ contract workers to manually search for and review the records, and if the individuals were under time constraints to meet a court deadline, the element of human error would be a realistic factor.

In making that finding, the judge cited 2011 research by Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, 17 Rich. J.L. & Tech. 11, 61 (2011). The judge quoted this passage:

Overall, the myth that exhaustive manual review is the most effective — and therefore, the most defensible — approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort. Of course, not all technology-assisted reviews (and not all manual reviews) are created equal. The particular processes found to be superior in this study are both interactive, employing a combination of computer and human input. While these processes require the review of orders of magnitude fewer documents than exhaustive manual review, neither entails the naïve application of technology absent human judgment. Future work may address which technology-assisted review process(es) will improve most on manual review, not whether technology-assisted review can improve on manual review.

Two other points from this decision are worthy of note.

First, to select the sample size, the defendant sought permission of the court to RAT-STATS, statistical software used by the Office of Inspector General of the U.S. Department of Health & Human Services to select random samples and estimate improper payments.

Plaintiff offers no objection to or criticism of RAT-STATS. Instead, Plaintiff contends that allowing Defendant to conduct random sampling is akin to giving her a raffle ticket. She does not explain the analogy, nor is the similarity immediately apparent. Absent a reasoned objection and considering the source and reliability of the statistical tool, the Court is satisfied that the methodology Defendant proposes does not prejudice Plaintiff.

Also notable is the procedural posture of this case, which came before the magistrate judge as a request for a protective order. The court had previously granted plaintiff’s motion to compel and ordered defendant to produce documents within 14 days.

Here, defendant sought a protective order or, alternatively, an extension of time to respond. Plaintiff objected, arguing that defendant could have raised its undue burden argument in response to the initial motion to compel, but did not.

The court held that defendant had not waived its right to seek a protective order. At the time of the initial motion to compel, defendant was not aware of the “enormity of the task” of producing the requested records. Once that became apparent, defendant was entitled to seek relief from the court.

The case is Duffy v. Lawrence Memorial Hospital, Case No. 2:14-cv-2256-SAC-TJJ (D. Kansas, March 31, 2017).

mm

About Bob Ambrogi

A lawyer and veteran legal journalist, Bob advises Catalyst on strategic communications and marketing matters. He is also a practicing lawyer in Massachusetts and is the former editor-in-chief of The National Law Journal, Lawyers USA and Massachusetts Lawyers Weekly. A fellow of the College of Law Practice Management, he also writes the blog LawSites.

One thought on “Citing TAR Research, Court OKs Production Using Random Sampling

  1. Ethan

    Interesting that the sample size calculations relied on a 10 document (presumably non-random) sampling drawn for the unrelated purpose of estimating time. That related sampling suggested a 10% richness, but there’s nothing in the motion, order, or exhibit about how that came to be, or if it was sound. It’s what I’d use too in the absence of anything else, but its also worth noting that to get the same 10% confidence interval (@99% Confidence Level) under the worst case 50% richness, they’d have to sample over 400 files.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *