Judge Peck Declines to Force the Use of TAR

TC_Bob_AmbrogiBob Ambrogi, who serves as director of communications here at Catalyst, posted a detailed analysis yesterday at Bloomberg Law’s Big Law Business of Magistrate Judge Andrew J. Peck’s latest decision involving technology assisted review, Hyles v. New York City. It’s well worth a look.

Hyles is an employment case where the plaintiff wanted the court to force New York City to use TAR rather than its proposed search terms. Even though Judge Peck emphasized “that in general, TAR is cheaper, more efficient and superior to keyword searching,” he nevertheless declined to force the defendants to use TAR, finding that it hasn’t yet displaced other tools to the point where using something else is unreasonable. Further, the Sedona Principles state:

Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.

I also noticed an interesting point lurking in the decision about continuous active learning. Discovery had apparently been contentious and New York City declined to use TAR “both because of cost and concerns that the parties, based on their history of scope negotiations, would not be able to collaborate to develop the seed set for a TAR process.”

If a TAR system that uses CAL was being considered, this problem could have been easily resolved. For starters, CAL systems use every attorney decision for training, so the city would already be producing all responsive, non-privileged training documents anyway. Judge Peck noted this last year in his Rio Tinto decision with a citation to third-party research, and raised it again in the last paragraph of this most recent decision:

To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning (“CAL”), which eliminates issues about the seed set and stabilizing the TAR tool.

Indeed, we have worked with clients in similarly contentious cases to develop TAR protocols that were acceptable to both sides. And even without that sort of collaboration, parties on the receiving end of a CAL review should review Cormack and Grossman’s 2015 paper on Multi-Topic Recall and CAL, which shows that as long as the TAR system is using CAL as its training protocol and the review continues until high recall (~75%) is reached, then all subtopics or different categories of relevant documents all have high recall individually. In other words, CAL reviews to high recall tend to be complete naturally, regardless of what training documents the review starts with.

In Hyles, therefore, a TAR 2.0 system using CAL could have saved New York City time and money. It could also have given the plaintiff some ways to easily participate in the initial seeding without hindering the city’s review. And all parties could be reassured that as long as the final validation showed that high recall had been reached, no significant topics had been missed.