Citing TAR Research, Court OKs Production Using Random Sampling

Catalyst_Court_OKs_Production_Using_Random_SamplingCiting research on the efficacy of technology assisted review over human review, a federal court has approved a party’s request to respond to discovery using random sampling.

Despite a tight discovery timeline in the case, the plaintiff had sought to compel the defendant hospital to manually review nearly 16,000 patient records.

But the hospital presented evidence that manual review and redaction would require nearly 9,000 hours of work and cost in excess of $230,000. It asked the court to allow it to instead produce a random sampling of 252 patient records.

Granting the defendant’s request, U.S. Magistrate Judge Teresa J. James of the U.S. District Court in Kansas concluded that the request was justified by the time and expense that a full production would require.

Judge James further found that a random sampling could produce a more accurate result.

If Defendant were to employ contract workers to manually search for and review the records, and if the individuals were under time constraints to meet a court deadline, the element of human error would be a realistic factor.

In making that finding, the judge cited 2011 research by Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, 17 Rich. J.L. & Tech. 11, 61 (2011). The judge quoted this passage:

Overall, the myth that exhaustive manual review is the most effective — and therefore, the most defensible — approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort. Of course, not all technology-assisted reviews (and not all manual reviews) are created equal. The particular processes found to be superior in this study are both interactive, employing a combination of computer and human input. While these processes require the review of orders of magnitude fewer documents than exhaustive manual review, neither entails the naïve application of technology absent human judgment. Future work may address which technology-assisted review process(es) will improve most on manual review, not whether technology-assisted review can improve on manual review.

Two other points from this decision are worthy of note.

First, to select the sample size, the defendant sought permission of the court to RAT-STATS, statistical software used by the Office of Inspector General of the U.S. Department of Health & Human Services to select random samples and estimate improper payments.

Plaintiff offers no objection to or criticism of RAT-STATS. Instead, Plaintiff contends that allowing Defendant to conduct random sampling is akin to giving her a raffle ticket. She does not explain the analogy, nor is the similarity immediately apparent. Absent a reasoned objection and considering the source and reliability of the statistical tool, the Court is satisfied that the methodology Defendant proposes does not prejudice Plaintiff.

Also notable is the procedural posture of this case, which came before the magistrate judge as a request for a protective order. The court had previously granted plaintiff’s motion to compel and ordered defendant to produce documents within 14 days.

Here, defendant sought a protective order or, alternatively, an extension of time to respond. Plaintiff objected, arguing that defendant could have raised its undue burden argument in response to the initial motion to compel, but did not.

The court held that defendant had not waived its right to seek a protective order. At the time of the initial motion to compel, defendant was not aware of the “enormity of the task” of producing the requested records. Once that became apparent, defendant was entitled to seek relief from the court.

The case is Duffy v. Lawrence Memorial Hospital, Case No. 2:14-cv-2256-SAC-TJJ (D. Kansas, March 31, 2017).


About Bob Ambrogi

Bob is known internationally for his expertise in the Internet and legal technology. He held the top editorial positions at the two leading national U.S. legal newspapers, the National Law Journal and Lawyers USA. A long-time advisor to Catalyst, Bob now divides his time between law practice and media consulting. He writes two blogs, LawSites and MediaLaw, co-authors's Legal Blog Watch, and co-hosts the weekly legal-affairs podcast Lawyer2Lawyer. A 1980 graduate of Boston College Law School, Bob is a life member of the Massachusetts Bar Foundation and an active member of the Massachusetts Bar Association, which honored him in 1994 with its President's Award.