TAR in the Courts: A Compendium of Case Law about Technology Assisted Review

Magistrate Judge Andrew Peck

Magistrate Judge Andrew Peck

It is less than three years since the first court decision approving the use of technology assisted review in e-discovery. “Counsel no longer have to worry about being the ‘first’ or ‘guinea pig’ for judicial acceptance of computer-assisted review,” U.S. Magistrate Judge Andrew J. Peck declared in his groundbreaking opinion in Da Silva Moore v. Publicis Groupe.

Judge Peck did not open a floodgate of judicial decisions on TAR. To date, there have been fewer than 20 such decisions and not one from an appellate court.

However, what he did do — just as he said — was to set the stage for judicial acceptance of TAR. Not a single court since has questioned the soundness of Judge Peck’s decision. To the contrary, courts uniformly cite his ruling with approval.

That does not mean that every court orders TAR in every case. The one overarching lesson of the TAR decisions to date is that each case stands on its own merits. Courts look not only to the efficiency and effectiveness of TAR, but also to issues of proportionality and cooperation.

What follows is a summary of the cases to date involving TAR. Each includes a link to the full-text decision, so that you can read for yourself what the court said.

Have I missed any? Let me know and I’ll add them in.


2012


Da Silva Moore, et al. v. Publicis Groupe, No. 11 Civ. 1279 (ALC)(AJP), 2012 WL 607412 (S.D.N.Y. Feb. 24, 2012).

Judge: U.S. Magistrate Judge Andrew J. Peck

Holding: The court formally approved the use of TAR to locate responsive documents. The court also held that Federal Rule of Evidence 702 and the Daubert standard for the admissibility of expert testimony do not apply to discovery search methods.

Significance:  This is the first judicial opinion approved the use of TAR in e-discovery.

Notable quote: “What the Bar should take away from this Opinion is that computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review. Counsel no longer have to worry about being the ‘first’ or ‘guinea pig’ for judicial acceptance of computer-assisted review.”

Full text: Da Silva Moore v. Publicis Groupe (Peck Opinion) Feb. 24, 2012.

__________________________

Global Aerospace, Inc. v. Landow Aviation, L.P., No. CL 61040 (Vir. Cir. Ct. Apr. 23, 2012).

Judge: Circuit Judge James H. Chamblin 

Holding: Despite plaintiffs’ objection, court ordered that defendants may use predictive coding for the purposes of processing and producing ESI, without prejudice to plaintiffs later raising issues as to the completeness of the production or the ongoing use of predictive coding.

Significance: This appears to be the first state court case expressly approving the use of TAR.

Notable quote: “Defendants shall be allowed to proceed with the use of predictive coding for purposes of the processing and production of electronically stored information.”

Full text: Global Aerospace v. Landow Aviation, 4-23-2012.

__________________________

Da Silva Moore, et al. v. Publicis Groupe, 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012).

Judge: U.S. District Judge Andrew L. Carter Jr. 

Holding: The court affirmed Magistrate Judge Peck’s order approving the use of TAR.

Significance: Insofar as Judge Peck’s order was the first judicial opinion approving the use of TAR, its affirmance by Judge Carter further cemented its significance.

Notable quote: “Judge Peck concluded that under the circumstances of this particular case, the use of the predictive coding software as specified in the ESI protocol is more appropriate than keyword searching. The court does not find a basis to hold that his conclusion is clearly erroneous or contrary to law.”

Full text: Da Silva Moore v. Publicis Groupe_(CarterOpinion)_04_26_2012 .

__________________________

Nat’l Day Laborer Org. Network v. U.S. Immigration & Customs Enforcement Agency, No. 10 Civ. 2488 (SAS), 2012 WL 2878130 (S.D.N.Y. July 13, 2012).

Judge: U.S. District Judge Shira Scheindlin

Holding: In an action under the federal Freedom of Information Act, the court held that the federal government’s searches for responsive documents were inadequate because of their failure to properly employ modern search technologies.

Significance: In a decision in which Judge Scheindlin urged the government to “learn to use twenty-first century technologies,” she discussed predictive coding as representative of “emerging best practices” in compensating for the shortcomings of simple keyword search.

Notable quote: “Beyond the use of keyword search, parties can (and frequently should) rely on latent semantic indexing, statistical probability models, and machine learning tools to find responsive documents. Through iterative learning, these methods (known as ‘computer-assisted’ or ‘predictive’ coding) allow humans to teach computers what documents are and are not responsive to a particular FOIA or discovery request and they can significantly increase the effectiveness and efficiency of searches.”

Full text: National Day Laborer v. US Immigration and Customs Enforcement Agency.

__________________________

In re Actos (Pioglitazone) Prods. Liab. Litig., MDL No. 6:11-MD-2299 (W.D. La. July 27, 2012).

Judge: Magistrate Judge Patrick J. Hanna

Holding: In a multi-district products liability matter, the magistrate judge approved the parties’ agreement to use TAR for the production of ESI.

Significance: This case was significant as one of the earliest in which a federal court explicitly endorsed the use of TAR.

Notable quote: None.

Full text: In re Actos, 07 27 2012.

__________________________

EORHB, Inc. v. HOA Holdings, LLC, No. 7409-VCL (Del. Ch. Oct. 15, 2012).

Judge: Vice Chancellor J. Travis Laster

Holding: Court on its own initiative ordered parties to use predictive coding or to show cause why they should not. 

Significance: This was another early case in which the judge acknowledged the efficacy of using TAR.

Notable quote: “This seems to me to be an ideal non-expedited case in which the parties would benefit from using predictive coding. I would like you all, if you do not want to use predictive coding, to show cause why this is not a case where predictive coding is the way to go.”

Full text: EORHB_v_HOA_Holdings, 10 15 2012.


2013


Gabriel Techs., Corp. v. Qualcomm, Inc., No. 08CV1992 AJB (MDD), 2013 WL 410103 (S.D. Cal. Feb. 1, 2013).

Judge: U.S. District Judge Anthony J. Battaglia

Holding: Following entry of judgment in their favor in a patent infringement case, defendants filed a motion seeking attorneys’ fees, including $2.8 million “attributable to computer-assisted, algorithm-driven document review.” The court found that amount to be reasonable and approved it.

Significance: The court found that the costs of TAR could be recovered as part of the costs and attorneys’ fees awarded to the prevailing party in patent litigation.

Notable quote: “[T]he Court finds [lead counsel] Cooley’s decision to undertake a more efficient and less time-consuming method of document review to be reasonable under the circumstances. In this case, the nature of the Plaintiffs’ claims resulted in significant discovery and document production, and Cooley seemingly reduced the overall fees and attorney hours required by performing electronic document review at the outset. Thus, the Court finds the requested amount of $2,829,349.10 to be reasonable.”

Full text: Gabriel Technologies v. Qualcomm Inc. 02 01 2013.

__________________________

In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., 2013 U.S. Dist. LEXIS 84440 (N.D. Ind. Apr. 18, 2013).

Judge:  U.S. District Judge Robert L. Miller Jr. 

Holding: Court held that defendant’s use of keyword searching to cull documents population prior to application of TAR was reasonable under the requirements of Federal Rules of Civil Procedure 26(b). It declined to require the defendant to go back and use TAR on the entire ESI population.

Significance: The court found that proportionality trumped purity, and that even if predictive coding might unearth additional relevant documents, the cost would far outweigh the likely benefits.

Notable quote: “It might well be that predictive coding, instead of a keyword search, at Stage Two of the process would unearth additional relevant documents. But it would cost Biomet a million, or millions, of dollars to test the Steering Committee’s theory that predictive coding would produce a significantly greater number of relevant documents. Even in light of the needs of the hundreds of plaintiffs in this case, the very large amount in controversy, the parties’ resources, the importance of the issues at stake, and the importance of this discovery in resolving the issues, I can’t find that the likely benefits of the discovery proposed by the Steering Committee equals or outweighs its additional burden on, and additional expense to, Biomet.”

Full text: In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig.,_04 18 2013.

__________________________

EORHB, Inc. v. HOA Holdings, LLC, No. 7409-VCL, 2013 WL 1960621 (Del. Ch. May 6, 2013).

Judge: Vice Chancellor J. Travis Laster.

Holding: In an earlier order, the court ordered the parties to “retain a single discovery vendor to be used by both sides” and to “conduct document review with the assistance of predictive coding.” In this new order, the court accepted the parties’ agreement that defendants could use TAR and retain their own vendor and that plaintiffs would not be required to use TAR because the cost would likely outweigh the benefit.

Significance: The court declined to require a party to use TAR when its cost would outweigh its anticipated benefit.

Notable quote: “[B]ased on the low volume of relevant documents expected to be produced in discovery by [plaintiffs], the cost of using predictive coding assistance would likely be outweighed by any practical benefit of its use.”

Full text: Not available.

__________________________

Gordon v. Kaleida Health, No. 08-CV-378S(F), 2013 WL 2250579 (W.D.N.Y. May 21, 2013).

Judge: U.S. Magistrate Judge Leslie G. Foschio.

Holding: Impatient with the parties’ year-long attempts to agree on how to achieve a cost-effective review of some 200,000-300,000 emails, the magistrate judge suggested they try predictive coding. That led to a dispute over the extent to which the parties should meet and confer in order to agree on a TAR protocol. Because the parties ultimately agreed to meet, the judge never decided any substantive TAR issue.

Significance: The significance of this case is that it was the judge, not the litigants, who suggested the use of predictive coding.

Notable quote:  “At the last of a series of ESI discovery status conferences with the court, … the court expressed dissatisfaction with the parties’ lack of progress toward resolving issues related to completion of review and production of Defendants’ e-mails using the key-word search method, and pointed to the availability of predictive coding, a computer assisted ESI reviewing and production method, directing the parties’ attention to the recent decision of Magistrate Judge Peck in Moore v. Publicis Groupe & MSL Group, 287 F.R.D. 182 (S.D.N.Y.2012), approving use of predictive coding in a case involving over 3 million e-mails.”

Full text: Gordon v Kaleida Health 5 21 2013.

__________________________

In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., 2013 U.S. Dist. LEXIS 172570 (N.D. Ind. Aug. 21, 2013).

Judge: U.S. District Judge Robert L. Miller Jr.

Holding: The court ruled that defendants need not identify which of the documents, from among those they had already produced, were used in the training of the defendants’ TAR algorithm.

Significance: Because defendants had already complied with their obligation under the FRCP to produce relevant documents, the court held that it had no authority to compel the defendants to identify the specific documents it had used as seeds. Even so, the court said that it was troubled by the defendants’ lack of cooperation.

Notable quote: “The Steering Committee knows of the existence and location of each discoverable document Biomet used in the seed set: those documents have been disclosed to the Steering Committee. The Steering Committee wants to know, not whether a document exists or where it is, but rather how Biomet used certain documents before disclosing them. Rule 26(b)(1) doesn’t make such information disclosable.”

Full text: In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., August 21 2013.


2014


Federal Housing Finance Agency v. HSBC North America Holdings, 2014 WL 584300 (S.D. N.Y. Feb. 14, 2014).

Judge: U.S. District Judge Denise Cote

Holding: In a memorandum opinion, the judge stated that, earlier in the discovery process, she had permitted one defendant, JPMorgan Chase, to use predictive coding over the plaintiff’s objection. She recounted this in making the point that discovery is not expected to be a perfect process, but one in which parties act with diligence and good faith.

Significance: The case is significant as another in which a federal court allowed the use of TAR. It is also significant for its recognition that discovery does not require perfection.

Notable quote: “Parties in litigation are required to be diligent and to act in good faith in producing documents in discovery. The production of documents in litigation such as this is a herculean undertaking, requiring an army of personnel and the production of an extraordinary volume of documents. Clients pay counsel vast sums of money in the course of this undertaking, both to produce documents and to review documents received from others. Despite the commitment of these resources, no one could or should expect perfection from this process. All that can be legitimately expected is a good faith, diligent commitment to produce all responsive documents uncovered when following the protocols to which the parties have agreed, or which a court has ordered.”

Full text: FHFA v HSBC North America Holdings.

Progressive Cas. Ins. Co. v. Delaney, No. 2:11-cv-00678-LRH-PAL, 2014 WL 2112927 (D. Nev. May 20, 2014).

Judge: U.S. Magistrate Judge Peggy A. Leen.

Holding: The court rejected a party’s unilateral decision to use TAR because the party had already demonstrated that it lacked the willingness to engage in the type of cooperation and transparency that is needed for a TAR protocol to be accepted by a court.

Significance: The case is a reminder that efficiency and cost-effectiveness are not the only factors a court will look at in evaluating the use of TAR. Cooperation and transparency are also important factors.

Notable quote: “The cases which have approved technology assisted review of ESI have required an unprecedented degree of transparency and cooperation among counsel in the review and production of ESI responsive to discovery requests.”

Full text: Progressive Casualty Insurance v Delaney.

__________________________

FDIC v. Bowden, No. CV413-245, 2014 WL 2548137 (S.D. Ga. June 6, 2014).

Judge: Magistrate Judge G.R. Smith

Holding: In case involving some 2.01 terabytes of data, or 153.6 million pages of documents, the court suggested that the parties consider using TAR.

Significance: The court recognized TAR is more accurate than human review or keyword searching.

Notable quote: “Predictive coding has emerged as a far more accurate means of producing responsive ESI in discovery. Studies show it is far more accurate than human review or keyword searches which have their own limitations.” (Quoting Progressive Cas. Ins. Co. v. Delaney, 2014 WL 2112927 at *8 (D. Nev. May 20, 2014)).

Full text: FDIC v Bowden June 6, 2014.

__________________________

Bridgestone Americas, Inc. v. Int. Bus. Machs. Corp., No. 3:13-1196 (M.D. Tenn. July 22, 2014).

Judge: U.S. Magistrate Judge Joe B. Brown.

Holding: The court approved the plaintiff’s request to use predictive coding to review over 2 million documents, over defendant’s objections that the request was an unwarranted change in the original case management order and that it would be unfair to use predictive coding after an initial screening has been done with search terms.

Significance: The opinion suggests that e-discovery should be a fluid and transparent process and that principles of efficiency and proportionality may justify a party to “switch horses in midstream,” as the magistrate judge wrote.

Notable quote: “In the final analysis, the use of predictive coding is a judgment call, hopefully keeping in mind the exhortation of Rule 26 that discovery be tailored by the court to be as efficient and cost-effective as possible. In this case, we are talking about millions of documents to be reviewed with costs likewise in the millions. There is no single, simple, correct solution possible under these circumstances.”

Full text: Bridgestone Americas v. International Business Machines.

__________________________

In re Bridgepoint Educ., No. 12cv1737 JM (JLB), 2014 WL 3867495 (S.D. Cal. Aug. 6, 2014).

Judge: Magistrate Judge Jill L. Burkhardt.

Holding: This brief order included two holdings pertaining to TAR. First, in declining plaintiffs’ request to expand the scope of discovery as unduly burdensome on defendants, the court rejected plaintiffs’ argument that the use of predictive coding would alleviate any added burden. Second, the court declined to order defendants to use predictive coding for documents they had already produced, reasoning that it had approved defendants’ method of “using linear screening with the aid of search terms.”

Significance: The court applied principles of proportionality to limit the scope of discovery and the use of TAR.

Notable quote: “Defendants argued that putting the Individual Defendant documents already screened through predictive coding is likely to negatively impact the reliability of the predictive coding process. Defendants suggested that they would be willing to run additional search terms for the documents already screened but were not amenable to running these documents through the predictive coding process.”

Full text: In re Bridgepoint Education.

__________________________

Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, Nos. 2685-11, 8393-12 (T.C. Sept. 17, 2014).

Judge: U.S. Tax Court Judge Ronald L. Buch

Holding: The Tax Court approved petitioner’s use of TAR to identify potentially responsive and privileged data contained on two backup tapes, despite respondent’s objection that the technology was unproven.

Significance: This is the first opinion to formally sanction the use of TAR in the Tax Court.

Notable quote: ” Although predictive coding is a relatively new technique, and a technique that has yet to be sanctioned (let alone mentioned) by this Court in a published Opinion, the understanding of e-discovery and electronic media has advanced significantly in the last few years, thus making predictive coding more acceptable in the technology industry than it may have previously been. In fact, we understand that the technology industry now considers predictive coding to be widely accepted for limiting e-discovery to relevant documents and effecting discovery of ESI without an undue burden.”

Full text: Dynamo Holdings v. Comm’r of Internal Revenue.

mm

About Bob Ambrogi

Bob is known internationally for his expertise in the Internet and legal technology. He held the top editorial positions at the two leading national U.S. legal newspapers, the National Law Journal and Lawyers USA. A long-time advisor to Catalyst, Bob now divides his time between law practice and media consulting. He writes two blogs, LawSites and MediaLaw, co-authors Law.com's Legal Blog Watch, and co-hosts the weekly legal-affairs podcast Lawyer2Lawyer. A 1980 graduate of Boston College Law School, Bob is a life member of the Massachusetts Bar Foundation and an active member of the Massachusetts Bar Association, which honored him in 1994 with its President's Award.