Judge Peck’s Latest: TAR Now ‘Black Letter Law’; CAL Reduces Significance of Seed Set

It is has been three years since U.S. Magistrate Judge Andrew J. Peck issued the first court decision approving the use of technology assisted review in e-discovery, Da Silva Moore v. Publicis Groupe & MSL Grp., 287 F.R.D. 182 (S.D.N.Y. 2012) (Peck, M.J.), affd, 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012). “This Opinion appears to be the first in which a Court has approved of the use of computer-assisted review,” he wrote then.

Magistrate Judge Andrew Peck

Magistrate Judge Andrew Peck

Now, in an opinion released yesterday, Judge Peck says that, in the years since Da Silva Moore, “the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.”

The opinion was issued in the case Rio Tinto PLC v. Vale SA, Case 1:14-cv-03042-RMB-AJP (S.D. N.Y. March 3, 2015). The issue before Judge Peck was approval of the parties’ stipulated TAR protocol. But Judge Peck took the opportunity to address several issues of broader interest to the e-discovery community about TAR cases and protocols.

The key issue he addressed was that of how transparent and cooperative the parties need to be with respect to the seed or training sets used in TAR. After reviewing a number of cases that have considered the issue, Judge Peck concluded that they are split. “Thus, where the parties do not agree to transparency, the decisions are split and the debate in the discovery literature is robust,” he observed.

Notably, Judge Peck said that transparency becomes less of an issue when the TAR methodology uses continuous active learning. “If the TAR methodology uses ‘continuous active learning’ (CAL) (as opposed to simple passive learning (SPL) or simple active learning (SAL)), the contents of the seed set is much less significant,” he wrote.

In support of this statement, Judge Peck cited two studies done by Gordon V. Cormack and Maura R. Grossman: Evaluation of Machine Learning Protocols for Technology-Assisted Review in Electronic Discovery, in Proceedings of the 37th Int’l ACM SIGIR Conf. on Research & Dev. in Info. Retrieval (SIGIR ’14) (ACM New York, N.Y. 2014), and Comments on “The Implications of Rule 26(g) on the Use of Technology-Assisted Review,” 7 Fed. Cts. L. Rev. 285 (2014).

As we’ve explained in a number of posts to this blog and in our recent book, TAR for Smart People: How Technology Assisted Review Works and Why It Matters for Legal Professionals, CAL reduces or even eliminates the need for a seed set, because the TAR engine continually ranks all of the documents all of the time. Training success is based on ranking fluctuations across the entire set, rather than on a limited set of randomly selected documents.

(CAL is the method used by Catalyst’s TAR platform, Insight Predict.)

Judge Peck points out that there are other ways besides cooperation that parties can ensure that training was done properly.

In any event, while I generally believe in cooperation, requesting parties can insure that training and review was done appropriately by other means, such as statistical estimation of recall at the conclusion of the review as well as by whether there are gaps in the production, and quality control review of samples from the documents categorized as non-responsive.

Ultimately, Judge Peck concludes that he need not rule on the seed-set transparency in this case, because the parties agreed to a protocol that discloses all non-privileged documents in the control set. But he added this caveat:

One point must be stressed – it is inappropriate to hold TAR to a higher standard than keywords or manual review. Doing so discourages parties from using TAR for fear of spending more in motion practice than the savings from using TAR for review.

Judge Peck concludes his opinion by approving the parties’ TAR protocol. However, he notes that he does so because the parties agreed to it. Describing the protocol as “somewhat vague and generic,” he suggests it should not necessarily be considered a guide for future cases.


About Bob Ambrogi

Bob is known internationally for his expertise in the Internet and legal technology. He held the top editorial positions at the two leading national U.S. legal newspapers, the National Law Journal and Lawyers USA. A long-time advisor to Catalyst, Bob now divides his time between law practice and media consulting. He writes two blogs, LawSites and MediaLaw, co-authors Law.com's Legal Blog Watch, and co-hosts the weekly legal-affairs podcast Lawyer2Lawyer. A 1980 graduate of Boston College Law School, Bob is a life member of the Massachusetts Bar Foundation and an active member of the Massachusetts Bar Association, which honored him in 1994 with its President's Award.