So let’s continue to revisit that all-important working question of: How do our choices prior to review affect our total cost?
Establishing the NumbersLet’s recap the assumptions we will continue to work off of. In this scenario, we have 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour. Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte per month. We will also be using 4-month longevity for this case.
In my previous article, How to Manage Your total Cost of Review: Every Percent Matters, we took this same scenario and presented the choice of paying upfront costs to utilize an ECA to reduce our cull rate by 5%. We found that, through this, we were able to produce significant cost savings, upwards of $11,000.
In today’s exercise, let’s move on from the ECA stage. The last question we’ll visit here is: With your current data set, would you spend $3,000 on analytics expertise upfront to use Active Learning technology to create a responsive and non-responsive model?
$3,000 for Active Learning?With this, there are no additional hosting fees, there’s no other technology fees, you basically have to pay for an expert to help you think through how you’re going to set up your workflow and then to execute on it with the software on the back end.
This is very much like the previous situation. The first and second month fees are essentially the same, because we’re talking about something that’s happening a little later in the process. In this scenario, you have this expertise that you’re spending money on in month three versus the standard workflow, where you’re not. So, you won’t see any cost saving initially until month 3. If your standard workflow ends up costing you about $77,000, what does the active learning workflow cost you?
It’s Not Black and WhiteThis becomes a tough question to answer actually, because active learning is used a few different ways.
One way is that we take all the documents that you are going to look at, we split them into the documents we care about, which are the ones with the “thumbs up” and you also split them into the ones we don’t care about, which are the “thumbs down”. A reviewer will still need to look at every document. This may be the safest way to utilize active learning, but It’s also the least economically efficient. The benefit of this from this because you’re still going find the documents that you care about faster, because we are using the model to push the most relevant documents to the front of the line.
The second way we can use active learning is, we can take your pile of documents. There is the stuff we care about, the stuff we don’t, and we can put enough eyes on the document set. With this, we don’t care to train the model. We are reasonably confident the model is producing accurate predictions. We never need to look at the document set, avoiding things such as mass emails, fantasy football emails, or other junk, since the model is helping to sort it out for us.
And lastly, we can use active learning is you look at just enough documents to train the model, but this approach requires full trust in the model.
And so depending on which one of these workflows you use, our results will vary. At Acorn, we almost always recommend the second approach, as more than often it tends to be most applicable and beneficial to clients.
What Do the Numbers Say?And so, in the chart below we illustrate how these approaches play out. As we can see you can end up with an active learning project that results in equal or significantly less costs overall compared to the standard model. If we conceptualized our active learning from the beginning, we more likely would see a reduction of about $20,000, or 30-35%. Due to active learning’s capabilities of differentiating the relevant documents from the noise. You will still need eyeballs on the relevant document, but active learning can be a tool to avoid having to put eyeballs on every document, especially those that are not relevant to your matter.
What’s the Right Answer?So again, generally, active learning is a good investment, even without reducing the document set. And that’s because it’s pretty inexpensive to implement. And even in cases where you don’t save money, you gain time, which is in and of itself, its own form of value.
Stay tuned for next month as I continue this exercise and examine how we can leverage advanced technologies to further streamline our review and further reduce our total costs. ____________ Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!