In Affirming Use of TAR, Tax Court Takes Down ‘Two Myths’ of Review

TC_CourtTwo years ago, U.S. Tax Court Judge Ronald L. Buch broke new ground when he became the first judge to formally sanction the use of technology assisted review in the Tax Court. In Dynamo Holdings Limited Partnership v. Commissioner of Internal Revenue, Judge Buch said that TAR “is an expedited and efficient form of computer-assisted review that allows parties in litigation to avoid the time and costs associated with the traditional, manual review of large volumes of documents.”

After receiving Judge Buch’s permission, the petitioners went on to use a TAR process and to use the results of that process to respond to the IRS commissioner’s discovery requests. Earlier this year, believing the response to be incomplete, the commissioner served a new set of discovery requests. When petitioners objected, the commissioner filed a motion to compel, thus bringing the TAR issue before Judge Buch a second time.

Now, in a July 13 ruling denying the motion to compel, Judge Buch not only strongly affirmed the use of TAR, but he also took down what he described as “two myths” about e-discovery review.

The TAR Process

At issue was a discovery request from the commissioner to the petitioners to produce electronic information stored on two back-up tapes. After Judge Buch’s 2014 order, the parties cooperated to jointly develop a TAR protocol.

At the commissioner’s request, the petitioners started by running Boolean searches based on a list of 76 search terms provided by the commissioner. The commissioner initially requested that the documents that resulted from these searches be used to create the seed sets. Instead, the parties agreed that petitioners would randomly select two sets of 1,000 documents from each of two back-up tapes.

After receiving the seed sets, the commissioner identified which documents were relevant or not, and this coding was used to train the TAR algorithm. The parties set a goal of returning 95 percent of the relevant documents. After testing indicated that the model was not performing well, additional seeds were coded to further train the system.

Ultimately, petitioners ran the TAR process against the initial set of two back-up tapes containing approximately 406,000 documents at a 95 percent recall rate. Petitioners also ran a second algorithm on the set to identify privileged materials. When the process was complete, petitioners produced to the commissioner some 180,000 documents. The commissioner reviewed these and retained 5,796 documents.

The Motion to Compel

The commissioner’s motion to compel was founded on the fact that some 1,300 of the documents identified in the initial Boolean search were not included in the final production set. These documents were “highly likely to be relevant,” the commissioner argued.

Petitioners objected to the motion, arguing that the actual number of omitted documents was much smaller and that, in any event, the TAR algorithm had done its job of finding relevant documents based on criteria set by the commissioner.

The Two Myths of Review

Giving the commissioner the benefit of the doubt, Judge Buch began his analysis by agreeing to assume that the TAR response was flawed. But even assuming it was flawed, he said, “the question remains whether any relief should be afforded.”

The commissioner’s motion to compel, Judge Buch said, “is predicated on two myths.”

The first myth, he said, is “the myth of human review.” This he described as the widely held belief that manual review by humans is the gold standard by which all searches should be measured.

“This myth of human review is exactly that: a myth,” he said. “Research shows that human review is far from perfect.”

The second myth, he wrote, is “the myth of a perfect response.”

The Commissioner is seeking a perfect response to his discovery request, but our Rules do not require a perfect response. Instead, the Tax Court Rules require that the responding party make a “reasonable inquiry” before submitting the response. … [W]hen the responding party is signing the response to a discovery demand, he is not certifying that he turned over everything, he is certifying that he made a reasonable inquiry and to the best of his knowledge, his response is complete.

Likewise, the Federal Rules of Civil Procedure require only a “reasonable inquiry” when making discovery responses, he noted.

The Sufficiency of TAR

These standards do not change when a party uses TAR, Judge Buch said. “The fact that a responding party uses predictive coding to respond to a request for production does not change the standard for measuring the completeness of the response.”

Given this standard, it was clear that petitioners satisfied the court’s rules with their response, Judge Buch concluded.

“Petitioners made a reasonable inquiry in responding to the Commissioner’s discovery demands when they used predictive coding to produce any documents that the algorithm determined was responsive, and petitioners’ response was complete when they produced those documents,” he explained.

mm

About Bob Ambrogi

Bob is known internationally for his expertise in the Internet and legal technology. He held the top editorial positions at the two leading national U.S. legal newspapers, the National Law Journal and Lawyers USA. A long-time advisor to Catalyst, Bob now divides his time between law practice and media consulting. He writes two blogs, LawSites and MediaLaw, co-authors Law.com's Legal Blog Watch, and co-hosts the weekly legal-affairs podcast Lawyer2Lawyer. A 1980 graduate of Boston College Law School, Bob is a life member of the Massachusetts Bar Foundation and an active member of the Massachusetts Bar Association, which honored him in 1994 with its President's Award.