How to Manage Your Total Cost of Review: Actively Understand Your Document Set

Published on May 27, 2021

Zef Deda

Zef Deda is a business development manager at Acorn Legal Solutions. An e-discovery thought leader working with Am Law 100 firms and corporate legal departments, Zef plays a key role in leveraging his knowledge of advanced technology and phased project plans in helping clients solve complex issues. Zef positions himself as a collaborative thought partner to ensure the best outcomes through his understanding of his client’s problems, their end goals, and working closely with them to identify and ensure the best possible outcomes.

We have previously discussed how targeting our collections and utilizing an ECA to better understand our data and how it resulted in significant cost reduction to our TCR. With budgetary concerns still prominent more than ever among legal teams in 2020—and continue to be halfway into this year, any opportunity we have to reduce the scope of the review—and, in turn, its costs—should be seized. But teams must first take stock of where these opportunities exist throughout the e-discovery process in order to have the most significant impact on that bottom line.

So let’s continue to revisit that all-important working question of: How do our choices prior to review affect our total cost?

Establishing the Numbers

Let’s recap the assumptions we will continue to work off of. In this scenario, we have 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour. Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte per month. We will also be using 4-month longevity for this case.

In my previous article, How to Manage Your total Cost of Review: Every Percent Matters, we took this same scenario and presented the choice of paying upfront costs to utilize an ECA to reduce our cull rate by 5%. We found that, through this, we were able to produce significant cost savings, upwards of $11,000.

In today’s exercise, let’s move on from the ECA stage. The last question we’ll visit here is: With your current data set, would you spend $3,000 on analytics expertise upfront to use Active Learning technology to create a responsive and non-responsive model?

$3,000 for Active Learning?

With this, there are no additional hosting fees, there’s no other technology fees, you basically have to pay for an expert to help you think through how you’re going to set up your workflow and then to execute on it with the software on the back end.

This is very much like the previous situation. The first and second month fees are essentially the same, because we’re talking about something that’s happening a little later in the process. In this scenario, you have this expertise that you’re spending money on in month three versus the standard workflow, where you’re not. So, you won’t see any cost saving initially until month 3. If your standard workflow ends up costing you about $77,000, what does the active learning workflow cost you?

It’s Not Black and White

This becomes a tough question to answer actually, because active learning is used a few different ways.

One way is that we take all the documents that you are going to look at, we split them into the documents we care about, which are the ones with the “thumbs up” and you also split them into the ones we don’t care about, which are the “thumbs down”. A reviewer will still need to look at every document. This may be the safest way to utilize active learning, but It’s also the least economically efficient. The benefit of this from this because you’re still going find the documents that you care about faster, because we are using the model to push the most relevant documents to the front of the line.

The second way we can use active learning is, we can take your pile of documents. There is the stuff we care about, the stuff we don’t, and we can put enough eyes on the document set. With this, we don’t care to train the model. We are reasonably confident the model is producing accurate predictions. We never need to look at the document set, avoiding things such as mass emails, fantasy football emails, or other junk, since the model is helping to sort it out for us.

And lastly, we can use active learning is you look at just enough documents to train the model, but this approach requires full trust in the model.

And so depending on which one of these workflows you use, our results will vary. At Acorn, we almost always recommend the second approach, as more than often it tends to be most applicable and beneficial to clients.

What Do the Numbers Say?

And so, in the chart below we illustrate how these approaches play out. As we can see you can end up with an active learning project that results in equal or significantly less costs overall compared to the standard model. If we conceptualized our active learning from the beginning, we more likely would see a reduction of about $20,000, or 30-35%. Due to active learning’s capabilities of differentiating the relevant documents from the noise. You will still need eyeballs on the relevant document, but active learning can be a tool to avoid having to put eyeballs on every document, especially those that are not relevant to your matter.

What’s the Right Answer?

So again, generally, active learning is a good investment, even without reducing the document set. And that’s because it’s pretty inexpensive to implement. And even in cases where you don’t save money, you gain time, which is in and of itself, its own form of value.

Stay tuned for next month as I continue this exercise and examine how we can leverage advanced technologies to further streamline our review and further reduce our total costs.


Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit