Post Author: Zef Deda

Zef Deda is a business development manager at Acorn Legal Solutions. An e-discovery thought leader working with Am Law 100 firms and corporate legal departments, Zef plays a key role in leveraging his knowledge of advanced technology and phased project plans in helping clients solve complex issues. Zef positions himself as a collaborative thought partner to ensure the best outcomes through his understanding of his client’s problems, their end goals, and working closely with them to identify and ensure the best possible outcomes.

How to Manage Your Total Cost of Review: Putting it All Together

As we discussed in my previous articles, we identified where opportunities existed throughout our eDiscovery process where we can directly impact our bottom line and contain our total cost of review.

To recap, we first identified how targeting and narrowing our scope of collections reduced our review costs by up to approximately 30%. Secondly, we identified how an early investment in an ECA to reduce our cull rate by a few percent led to cost reductions of up to approximately 15%. Lastly, we identified how investing in advanced analytics, such as active learning, led to significant cost reductions of up to approximately 30%.

Individually, we can see that each one of our choices prior to review choices led to notable savings over the life of the case. In today’s exercise, we will look at back at all the oppurtunies we previously identified and the choices we made. This leads us to our final question: What happens when we put it all together?

Establishing the Numbers

Let’s recap the assumptions we will continue to work off of. In this scenario, we have 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour.

Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte, per month. We will also be using a four-month lifecycle for this case.

Putting it all Together

So what happens when you put it all together? If we narrow our initial collection, investing in culling the data better and leverage active learning, What’s the net impact of that in economic terms?

First of all, we have to acknowledge that there will be diminishing returns. When looking at each choice individually, it will have a greater impact since it is the only thing affecting our data set. But when done simultaneously, the individual impact will be reduced since we are leveraging each choice on top of the previous one. Think of it as taking a percent of a percent.

The “Perfect” Workflow

So let’s start in Month 1 start. We will perform our targeted collection, collecting 4 devices instead of 6. This will result in our initial cost savings of $1,000 and some reduction to our hosting and processing costs to come.

In Month 2, we will invest $5,000 upfront for an ECA and take a closer look at our data set. As we previously discussed we expected to increase our cull rate by 5% to 85%. Our hosting and processing will still be slightly reduced compared to the “Standard Workflow”, but our Month 2 total costs are now slightly higher with our “Perfect” Workflow. Don’t be deterred just yet, remember, our real goal is to reduce our total costs.

Let’s look at Month 3. We now make our $3,000 investment into analytics expertise upfront to use active learning technology. Similar to Month 2, our recurring costs stay the same and are lower than the “Standard Workflow”. But as we can see in the chart below, our incurred costs are continuing to increase and it would appear as if the “Perfect” Workflow is driving more expenses than savings. This doesn’t seem like a very convincing exercise does it?

The End is in Sight

Now it’s time to reap the fruits of our labor. Month 4 is the finish line we have been working towards. So through each month, we initially reduced out costs by $1,000 with our targeted collection, incurred $5,000 of costs with an ECA, and then an additional $3,000 of costs utilizing active learning. Net additional costs of $7,000. This might seem high now but is small in comparison to our total costs.

So, let’s move into our review phase and look at how our initial investments played out. So in this example, we would have reduced our total cost by approximately $34,000. You may also notice that there a range present on the chart. Realistically, it’s hard to gauge exactly what each investment affect on our review costs will be, but even at the expected high-end of the range, we still see cost savings of approximately $20,000.   

The Key Take Away

And so, my message to everyone is that strategic workflows can provide a significant amount of budget efficiency beyond just line-item reviews, reductions. So certainly, if you want to reduce total cost of review, and you can get down to 90 cents per document reviewed instead of $1 per document reviewed. That’s going to reduce your total cost of review, certainly, but we can only squeeze that lemon so much, and there’s not a lot of juice left in it. For the most part the industry has gotten about as low as it can get on almost all of the line items to continue reducing TCR. So, by identifying strategic opportunities throughout the life of our project we can continue to seize all possible cost savings.

And so as we think about discovery in 2021. The key takeaways is, now is the time to evaluate your opportunities  to reduce the total cost of review, which ones you currently utilize and which ones you have the potential to implement. We identified 3 effective ways to do so, and they all don’t need to be utilized, but just by adopting one of these approaches you are going to see significant results pretty quickly.  

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.

How to Manage Your Total Cost of Review: Actively Understand Your Document Set

Continuing on the subject of reining in your total cost of review (TCR), it must be reiterated that budget concerns were more prominent than ever among legal teams in 2020—and continue to be halfway into this year.

When it comes to e-discovery, review is always the greatest cost driver, so any opportunity we have to reduce its scope—and, in turn, its costs—should be seized. But teams must first take stock of where these opportunities exist throughout the e-discovery process in order to have the most significant impact on that bottom line.

So, returning to this always timely subject, let’s revisit that all-important working question of: How do our choices prior to review affect our total cost?

Thus far, we’ve analyzed the usefulness of targeted collections and ECA in reducing TCR. Today, let’s home in on artificial intelligence.

Establishing the Numbers

Let’s recap the assumptions we will continue to work off of. In this scenario, we have 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour.

Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte, per month. We will also be using a four-month lifecycle for this case.

 

 

In my previous article, How to Manage Your total Cost of Review: Every Percent Matters, we took this same scenario and presented the choice of paying upfront costs to utilize an ECA to reduce our cull rate by 5 percent. We found that, through this, we were able to produce significant cost savings: upwards of $11,000.

In today’s exercise, let’s move on from the ECA stage. The last question we’ll visit here is: With your current data set, would you invest $3,000 on analytics expertise upfront to use active learning technology? This would mean leveraging artificial intelligence to create a responsive and non-responsive model that will assist your review.

$3,000 for Active Learning?

With this option, there are no additional hosting fees and no other technology fees—you basically have to pay for an expert to help you think through how you’re going to set up your workflow and then execute on it with the software you’re already using.

This is very much like the previous situation. The first and second month fees are essentially the same, because we’re talking about a workflow option that’s happening a little later in the process. In this scenario, you have this expertise you’re spending money on in Month 3, versus the standard workflow where you’re not adding that expense. So, you won’t see any cost savings until Month 3 or later.

If your standard workflow ends up costing you about $77,000, what does the active learning workflow cost you?

It’s Not Black and White

This becomes a tough question to answer actually, because active learning is used a few different ways.

One way is that we take all the documents that you are going to look at, split them into the documents we care about—the ones with the proverbial “thumbs up” for relevance—and the ones we don’t care about, which are the “thumbs down.” A reviewer will still need to look at every document. This may be the safest way to utilize active learning, but It’s also the least economically efficient. The benefit of this approach is that you’ll still find the documents that you care about faster, because we are using the model to push the most relevant documents to the front of the line.

The second way we can use active learning is less manual still. We can take your pile of documents, knowing there is the stuff we care about and stuff we don’t, and we can put enough eyes on the document set to evaluate what’s relevant. With this approach, we opt not to train the model fully, by manually coding all documents. Instead, we are reasonably confident the model is producing accurate predictions. We never need to look at portions of the document set, avoiding things such as mass emails, fantasy football emails, or other junk, since the model is helping to sort it out for us.

And lastly, we can look at just enough documents to train the active learning model, and go with its predictions from there. But this approach requires full trust in the model and minimizes quality control checks, so it certainly isn’t right for every case.

And so, depending on which one of these workflows you use, your results in terms of the impact on TCR will vary. At Acorn, we almost always recommend the second approach, as more than often it tends to be most applicable and beneficial to clients.

What Do the Numbers Say?

With all of this in mind, in the chart below we illustrate how these approaches play out. As we can see, you can end up with an active learning project that results in equal or significantly less costs overall compared to the standard model.

If we conceptualized our active learning from the beginning, we more likely would see a reduction of about $20,000, or 30-35 percent. This is largely thanks to active learning’s ability to quickly and automatically differentiate the relevant documents from the noise. You will still need eyeballs on the relevant documents, but active learning can be an incredibly useful tool to avoid having to put eyeballs on every document—especially those that are not relevant to your matter.

What’s the Right Answer?

So again, generally, active learning is a good investment, even without reducing the document set before applying it. And that’s because it’s pretty inexpensive to implement. Even in cases where you don’t save money, you gain time—which is, of course, its own form of value.

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.

How to Manage Your Total Cost of Review: Every Percent Matters

Continuing on the subject of reining in your total cost of review (TCR), it must be reiterated that budget concerns were more prominent than ever among legal teams in 2020—and continue to be halfway into this year.

When it comes to e-discovery, review is always the greatest cost driver, so any opportunity we have to reduce its scope—and, in turn, its costs—should be seized. But teams must first take stock of where these opportunities exist throughout the e-discovery process in order to have the most significant impact on that bottom line.

On that note, let’s revisit that all-important working question of: How do our choices prior to review affect our total cost?

Last time we focused on the usefulness of targeted collections in reducing TCR. Today, let’s home in on early case assessment.

Establishing the Numbers

Let’s recap the assumptions we will continue to work off of. In this scenario, we have 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour.

Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte, per month. We will also be using a four-month lifecycle for this case.

In my previous article, How to Manage Your Total Cost of Review: Keeping Collections on Target, we took this same scenario and presented the choice of collecting broadly or collecting narrowly. We found that, through targeting our collection, we were able to not only reduce monthly recurring fees, but significantly reduce our total costs of review by up to $25,000.

In today’s exercise, let’s move on from the collection stage. The second question we’ll visit here is: For the custodians you collected, would you spend $5,000 up front to increase the cull rate from 80 percent to 85 percent?

Is 5% Worth $5,000?

$5,000 may seem like a lot when the total cost throughout the matter is about $50,000. Thinking in this vein, you may ask yourself: Since we already have such a high cull rate at 80 percent (remember—this means that only 1 in 5 collected documents make their way to review), why would I spend 10 percent of my overall budget for what seems like a minuscule increase of an extra 5 percent?

Well, let’s look at what the numbers say. You’re collecting and processing the same amount of data, so those costs will remain unchanged. However, in Month 3, we will start to notice that the costs shift between a scenario in which we make this extra investment in early case assessment (ECA) to increase our cull rate to 85 percent, and one where we don’t.

In the graph below you’ll notice the hosting sizes might appear similar but I assure you, they’re actually different, and we’ve reduced our monthly hosting costs by $400. But let’s be honest: it’s nothing notable and we had to spend $5,000 for those results. If we stopped right here, it would take us about a year to break even. At this point, you’re probably feeling pretty good if you chose to not to go with conducting ECA.

Looking at the Big Picture

But e-discovery doesn’t stop there. When we start thinking about this not in terms of hosting sizes but in the number of documents that will be reviewed, the real costs savings start to show.

Examined through this lens, you’ll find that, even though you’ve spent that additional $5,000 in Month 3, we’ve actually saved $11,000 over the life of the matter.

These savings are coming from two sources.

First, as we mentioned, your hosting costs have been reduced; not significantly, but it does compound over the life of the case.

Second, you should note that your biggest expense by far is the number of documents being reviewed. And so, if $5,000 removes at least 5,000 documents from review, then that $5,000 has paid for itself. If that $5,000 saves you more than 5,000 documents from being reviewed, then we were able to come out ahead and reduce our total costs.

How You Can Apply This

Utilizing ECA can be an ambiguous concept, as it can involve many different strategies for culling irrelevant data from a collection. A variety of tools can assist with the process as well, all of which can be found in RelativityOne.

One example of a basic approach to ECA that you could use to improve your culling rate is to take one or two key custodians, and pull them into an ECA workflow that allows you to gain insights into their data before it’s all fully processed into your review workspace. In this staging area, you can leverage artificial intelligence to produce communication visualizations and run a search terms report. Using some of this information and what you already know about the case, you can put together a brief timeline utilizing Case Dynamics to help you lay out the matter and see where your collected data might fall into that timeline. The combination of these applications will help you limit the date range and other data characterizations of your matter with confidence.

Now, this is just one example of a workflow and the tools you could utilize in an early case assessment exercise. To really extract the full value of the applications you may have on hand, and build case strategies for this and every matter that will help you produce the best possible outcomes, consider finding a partner with the expertise to apply the tools on a granular level. Their expertise may be essential to discovering and applying your new secret sauce.

It’s Always Worth Taking a Look

At Acorn, this scenario is one that we encounter quite often—but frequently with an even smaller percentage increase in the cull rate (think something like 80 percent to 80.5 percent). Even at a difference that small, depending on how much data you’re working with in your matter, that 0.5 percent can pay for itself very quickly.

Still, I do have to acknowledge that these scenarios are not one-size-fits-all. There are scenarios in which these additional upfront costs are not justifiable, such as if you expect to settle before review or if you’re working with a very small set of data. There’s little incentive to invest when you don’t expect to realize the returns of this investment.

In most scenarios, however, this investment will produce significant total cost savings throughout the life of a case—and in all scenarios, it is beneficial to at least evaluate this investment at the onset of the matter.

Stay tuned for next month as I continue this exercise and examine how we can leverage advanced technologies to further streamline our review and further reduce our total costs.

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

How to Manage Your Total Cost of Review: Keeping Your Collection on Target

As each year progresses, one thing is continually top of mind for businesses of all kinds: the budget.



In e-discovery, no one wants to miss their budget targets, whether they’re set by their team or their client—but it is a possibility we all face from case to case. Fortunately, it is usually preventable and at the very least mitigable.



With budget concerns having been particularly prominent through 2020, and that trend continuing into this year, it’s crucial to look at why and where we are generating the most costs and how we can manage them.

Review, of course, has long been a huge expense in the e-discovery processes. However, it is now facing added pressures from new challenges with evolving data types and volumes, and considerations for managing remote review teams as well. Even with these new challenges, creating efficiency for your review can still be easily achievable and directly translates to reduced total costs.



Since budgets are so important to the industry, our team at Acorn has spent a lot of time trying to think about how to best optimize its e-discovery workflows to consistently stay within parameters. To dig into this question, we’ve come up with a standard workflow for a case that is routine—pretty much what you would see across the industry—and put it to the test to understand what workflow changes can have the greatest impact on bottom lines.



With that in mind, we can have a real-world discussion around how review decisions get made and how the total cost of review (TCR) can be better managed and understood.



The working question guiding this exercise is: How do our choices prior to review affect our total cost?



To answer this question, let’s walk through an example. For our standard workflow, we’ve simply come back to the good old EDRM: collection, processing, hosting, and then review. I assigned prices to each stage that are generally close to market averages, but also nice round numbers for the sake of our math. I intend this to be an illustrative exercise, and not get into the legal arguments for increasing or reducing the scope of discovery—so keep in mind that this is entirely intended to be focused on the economics.


Establishing the Numbers

So, let’s lay out the scenario. As a baseline, let’s say we have a collection of 6 devices with costs of $500 per device and each device has 50 gigabytes of data. Assume each gigabyte contains 1,000 documents, and our reviewers work at a rate of 50 documents per hour at $50 an hour. Lastly, our processing rate will be $35 per gigabyte, and hosting will be $25 per gigabyte per month. We will also be using 4-month longevity for this case. We will be aggregating these costs throughout the months in the chart to follow.





Now that the baseline is established, let’s see how a review scenario can play out. In this article, we will focus on the collection stage. So, first question: We have the option of doing a collection broadly or doing it targeted. Which do you choose?


Collect Narrowly or Collect Broadly?

With a broad collection, we end up collecting the whole of all our data—meaning every custodian and every gigabyte that possibly could be relevant to our matter. On the surface, this has some sensibility; after all, we want to be thorough and meet our legal obligations, and we’re facing a relatively small cost of $500 per device for collection. As a result of this decision, we would collect data from all 6 devices in this case.



The alternative is a targeted collection. In this course of action, we will be more precise with our collection, avoiding custodians that are not crucial to the matter at this time. (We determine that we can go back later and collect from them if they turn out to be more critical than initially thought.) With this approach, we decide that only 4 of the 6 are crucial to our matter, so we would collect data from 4 of these devices.


Looking at the Big Picture

As you can see in the figure below, the initial costs are only marginally lower with the targeted collection—nothing too extreme. But the real impact of this collection is seen in Month 4. That’s where the numbers start to expand.



On the surface, when looking at the collection stage itself, we are faced with a difference in expenses of only $1,000. Why take the time to target your collection and risk having to recollect for a custodian later for such a small sum?



Well, as it turns out, that $1,000 savings initial isn’t the end goal—and how we collect will exponentially affect our TCR downstream. Let’s look at this through the full course of discovery.





As we see with this figure, initially our upfront costs are lower, followed by lower processing and hosting fees. But let’s look further down the line. With our targeted collection, we not only reduced our fees for Months 1 through 3, but significantly reduced our total review costs during Month 4. This led to a cost savings of approximately $25,000.


How You Can Address Collections

A targeted collection isn’t always time feasible or as convenient in the moment, but there are solutions out there that can assist with this process. One of those would be RelativityOne Collect.



RelativityOne Collect allows you to collect data straight from the source, without that data ever leaving the cloud. It currently integrates with Office 365—so, most of our emails—as well as Slack, Enterprise Vault, Microsoft Teams, Google Workspace, Bloomberg Chat, and WebEx, with additional endpoints to come. The nice thing about Collect is that you can filter your data sources at the onset, such as by custodian, date, and so on, allowing you to easily collect exactly what you need and nothing more.



Not only does Collect, and parallel tools, provide the ability to make your collections proactive, but it’s also defensibly sound with detailed reporting and a complete audit trail. This benefit that can prevent mishaps, such as those that occurred last year in Equal Employment Opportunity Commission v. M1 5100 Corp. when the defendant performed a self-collection.


Managing TCR is a Marathon, Not a Sprint

I do acknowledge this isn’t a perfect example and that we could quite possibly have to collect from another custodian as the case progresses, which would ultimately add to our costs—but in the end, even just collecting from 5 versus 6 devices is still impactful and would still reduce total review costs by at least $10,000.



Of course, this is only one consideration we should keep in mind from the onset of any matter. With this perspective, you can not only perform precise collections, but collections that are defensibly sound and create real budgetary impacts. But collections are only one step in the process, so don’t limit your focus.

____________

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

A New Year: Security Review and How RelativityOne Could Be Right for You

Going into 2021, some things remain uncertain, such as your data security, while others remain the same, such as the huge price tags on review. Many corporations and law firms have taken this pandemic as an opportunity to strengthen their security and as a result, hold their service partners to even higher standards. With RelativityOne’s industry-leading security standards, coupled with the speed and efficiency of their new Aero UI; RelativityOne allows us to now provide our clients with the personal service of a small vendor with all the highest security processes.

Security Considerations for How & Where You Host Data

Security has also been at the forefront of Discovery recently. The implementation of strict privacy regulations like the GDPR and CCPA add a very important driver for eDiscovery. According to Analyst at Gartner, more than 80% of data stored by companies is “dark data” in the form of unstructured, distributed data, which can become a significant legal and operational risks. With the global workforce now shifting to working remotely, this is of special concern as nearly all the company data maintained and used by remote employees is now in the form of “dark” or unstructured data. Are your providers adapting to today’s changing security environment? To find out, you should be asking your team, vendor, or other partners some simple questions, such as:
  • Do they have any training or education programs in place to prevent unauthorized access to your data through phishing?
  • Do they perform vulnerability assessments and penetration testing, and what were the results?
  • Do they have a Data Loss Prevention solution to proactively monitor whether your data is being exfiltrated in an unauthorized manner?
  • Is the data sandboxed, giving access only to authorized personnel?
  • Is user access secure with additional features such as two-factor authentication?
  • Have they developed a disaster recovery plan in the event that data theft does occur?
Security is a daunting project to get in front of. As you can see from these questions it’s not this black box and requires a lot of expertise to vet the strength of one solution versus another. Fortunately, there are standard security certificates you can use to avoid getting in the weeds. For example, FedRAMP is the highest data security standard required by the US government. In addition, organizations with ISO 27001 Type 2 certificates have reports generated and audited by third-party security experts which will explain a company’s security controls and should be available for your review. In eDiscovery, RelativityOne was built on Microsoft Azure, which meets more than 90 international and industry-specific compliance standards, to provide the industry’s most secure platform. Since it is available to us, you no longer need to choose between good service and data security.

Final Thoughts

RelativityOne has really allowed for not only stronger security, total cost savings and peace of mind to clients, but it has also allowed for providers like us to engage in more consulting and collaboration with our clients. Doing more with less is something I really believe will be the theme moving forward. Our world is changing and has required us all to adapt quickly, this should not be anything new to our industry. Being able to adapt as effectively and efficiently as possible, is now even more attainable with the right partner. If you haven’t found one already, there is never a better time than now to reach out and see if and how a trusted partner can help you start your matters on the right foot for 2021. In addition to being the most secure platform, you can also save up to 10% of your review costs just by switching. Stayed tuned for my next article to see how. ____________ Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

How to Identify Your Organization’s eDiscovery Architect

I have always thought of the eDiscovery process much like building a home and wondered where is our eDiscovery Architect.

Often times we get caught up on material price (or line item pricing) and projects become penny-wise and pound-foolish. As Doug Austin cited in this article, budgetary constraints are now one of the most impactful factors in this industry. When I read Doug’s article and saw the increase in concern with establishing and meeting budget in eDiscovery, I thought how other industries effectively manage this factor.

In order to successfully accomplish building a home on-time, on-budget and to the homeowners’ needs, you would need to have an Architect consult on general needs, draw up blueprints and dimensions, spec materials needed, etc.

Architect is not a defined role within our industry – but the skills are a tremendous avenue for career growth. Anyone can take on the mantle: lit support, attorneys or service providers. So, who is your project’s eDiscovery Architect? I would like to share with you a few simple ways that will help you become your organization’s Architect.

Put Your Foundation in Place

Take responsibility for creating the “blueprints” (workflows), build-out budgets, and defining key milestones and deliverables. To accomplish this, make sure you are:

  • Asking for thorough budgets with concrete assumptions and expectations
  • Understanding the foundational data and custodians that need to be collected from
  • Thinking of workflows that are not only well designed but that best suit the specific matter at hand
  • Identifying key vendor partners, internal stakeholders and client stakeholders that will be needed to execute on your plan

    • As you begin laying out your blueprint, don’t worry too much about not having all the information. It’s impossible to know everything at the beginning of the project and things will change. But, establishing a plan at the outset with what you do know will get everyone on the same page to avoid the project becoming a fire drill and you become reactive instead of proactive due to requests, etc. With your blueprint in hand, you can strategically and proactively address the unexpected, keep the project on course and keep everyone in the loop about the changes.

    Find the Right Tool for the Trade

    Now that you have laid the foundation (or have a game plan in place) it is time to spec out materials and begin executing.

    You should start evaluating the tools and weigh the tradeoff of different tools. There are a lot of tools to choose from. With your workflow ideas and the foundation laid out properly, will be best suited to understand what tools are necessary moving forward and where costs can be saved.

    Complex matters may call for more complex tools. These tools tend to be a little more expensive and intricate but are also very powerful and effective, reducing your total cost of review. For your more basic matters involving simpler data, you may want to utilize tools or a review platform that is simpler and more streamlined for your case team to navigate through. These tends to have lower price points but again will not have some high-end features like advanced AI and Analytics.

    It is critical to understand what tool is best suited to your situation and needs. A screwdriver might seem like a cheaper option, but you’ll probably want a power drill if you’re assembling an entire house.

    Contract the Right People

    Just like an expert carpenter costs more on an hourly basis, but saves money in the long-run, so too do your eDiscovery Project Managers. Understanding the trade-offs between price, expertise and reliability relative to your project-needs is where an eDiscovery Architect shines. In home-building, Architects keep a deep bench of service providers and eDiscovery Architects should, too.

    If you are working a case that needs heavy use of AI or analytics, then you would go to a vendor or team member that you know has expertise in this area and you can rely on to help with those needs. They can advise you about the technical and legal pitfalls, saving you time and money. Even though having your blueprint established is essential, trusting that your team can execute and make day-to-day adaptations to your plan is what drives real value.

    Final Thoughts

    eDiscovery projects are often six-figure endeavors, like a house. Because of the amount of the investment involved, you can add a lot of value by creating blueprints for your cases and making sure they are fully executed. The housing industry came up with these roles and processes to build efficiencies and we can learn from them and adapt them to our world.

    If you are looking to continue advancing your career within the industry and not quite sure what path to take, I would suggest working towards establishing yourself as the eDiscovery Architect. Organizations are constantly trying to find ways to be more efficient and cost-effective, your ability to help them do this successfully is invaluable.

    So who is your eDiscovery Architect? Let me know and share your thoughts!

    ____________

    Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

    About Acorn 

    Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

    How You Can Accomplish More with Fewer Reviewers

    With the current pandemic and everyone working remote and case teams shrinking, I wanted to take a moment and break down the math behind a lot of problems that people are facing. Because reviewers are working remotely, there is a limitation to how large of a review team you can staff. Sure, you can hire remote reviewers, but without an in-person review center, and in-person monitoring of the review team, there is an upper limit to how many people you can really staff. It stops becoming feasible to staff 50 people at once. The overhead of managing, QC’ing, and communicating with the team creates diseconomies of scale that limit the size of your team.

    So, with that, how do you accomplish the same amount of work, with a smaller team, remotely, in the same amount of time?

    I wanted to step back and look at the math behind that question. Assuming the budget is not an issue here, and the deadline can’t be moved, what are the levers you have to hit your deadline?

    The Baseline Budget

    Let us take the scenario where you have 500k documents that need to be reviewed in 13 weeks and you were planning on using 20 reviewers. With COVID and remote work so prominent now, you don’t think you can successfully staff and manage more than 5 reviewers. If you didn’t change anything about the process, your review would now take 48 weeks.

    If you’re trying to get the review done in less time, the obvious response is to tell those 5 reviewers to work harder and smarter until they figure out how to get it done. And there is probably some truth there. Let’s compare how much smarter or harder they would have to work.

    Current industry standards are that reviewers generally work 40 hours / week and can review 50 documents per hour. If you wanted to have 5 reviewers accomplish what 20 reviewers did, you would need them to work 160 hours/week and off barely 2 hours of sleep. Or, if you wanted them to accomplish it smarter, they would have to review 200 documents/hour which not many people in the world could do, unless you’re Howard Berg.

    Even if you do some combination of better productivity, like say 80 docs/hour, and more hours, like say 60 hours/week. Your team of 5 is still going to going to go 242% over the allocated time (or 17 weeks). So how do you get your full review done in 13 weeks? Well, you start thinking about using AI to reduce the dataset, streamline the review and narrow the scope of the review to let you hit your deadline. Let us look at each improvement you can make, one-by-one, and see how it helps you hit your deadline with your limited team.

    Improvement 1: EaiRLY INSIGHT™ Investigation

    Save 16.4 weeks by using EaiRLY INSIGHTS™ to defensibly eliminate custodians and remove their documents from the data set to be reviewed.

    Investing on front end with AI to further investigate and understand data collected will allow for less costs in review as documents can be eliminated from review. A trained ESI researcher can work with the attorneys to understand the fact patterns and goals of the review. Then, in combination with AI-based tools, they can quickly review a single custodian’s dataset at a scale that would be impossible without the AI. There are many great AI tools out in the market, our team personally utilizes NexLP’s StoryEngine platform.

    With this technique, you can defensibly eliminate custodians and data sources. In a typical matter with 10 custodians, who have had emails, laptops and mobile devices collected, usually, two custodians and one data source can be eliminated through this investigation. In our 500k document example, that would amount to 164,000 documents eliminated (100k documents from two custodians, and 64k from largely non-responsive laptop data sources). This saves 16.4 weeks of review — a pretty good start.

    Improvement 2: EDA and Search Term Analysis

    Save a further 8 weeks by further narrowing the document set with smarter search terms, informed by EDA and EaiRLY INSIGHT ™ Findings.

    Search terms and review feel like a chicken-or-the-egg situation. A lot of times, people feel like it’s hard to refine the search terms, without getting into the review to understand where the terms might be overinclusive. However, combining the knowledge from the EaiRLY INSIGHT ™ investigation with expert search term consultants who can provide information on things such as, strength of search term (in-depth search term analysis) and allow you to increase culling efficiency. This allows you to understand things such as, why particular terms have a lot of false positives. An example of this would be using the word “Beach” when the company’s name is “Beach Front Properties”. You would receive a lot of hits back on the term “Beach” thus raising the amount of documents your team would need to look through increasing reviews hours.

    With this approach, we typically see improvements in culling from 25% to 35% per custodian. In our sample case, that ultimately saves 10k documents per custodian over the 8 custodians, meaning that 80,000 documents don’t need to be reviewed and 8 weeks of time is saved.

    Improvement 3: All-Star Reviewers

    Save 15 weeks by increasing reviewer throughput by leveraging advanced analytics and increase their weekly hours from 40 to 60.

    If your managed review team is the same as the EaiRLY INSIGHT ™ team, they understand the case and story of the data from the outset of the review and it creates increased accuracy and efficiency.

    Often times, it’s the simplest of things that can make the world of difference. Being more strategic about the way you batch documents and prepare them for the review team can make a significant difference in review speeds. For example, batching by concept allows each reviewer to be more familiar with the pertinent issue, so they can move through documents more quickly. Or batching by near-duplicates allows the reviewer to quickly transfer knowledge from the first document reviewed to the remainder of the batch, increasing review throughput. On the more technical side, it may be worthwhile to begin separating all PDF documents so that all reviewers are not waiting for large PDF’s to load, wasting unnecessary time.

    Through this, your reviewers can do the equivalent to 80 Docs/Hour compared to the baseline of 50 Docs/Hour, saving an additional 9.6 weeks of review. When you combine that with increasing each reviewers’ weekly hours to 60, you save 15 weeks in total.

    Closing thoughts

    Obviously, this is more of a thought experiment than an actual real-world scenario. In reality, you’d probably push to staff more reviewers or push to move the deadline. But, I do think it’s helpful to understand the mathematical drivers behind scenarios like this. Ultimately, you can get more done by working reviewers longer and harder. But, you get the most bang for your buck, by figuring out how to reduce the dataset at the outset.

    In our scenario, although we would probably spend a bit more time upfront with the EaiRLY INSIGHT ™ investigation, it more than pays off in the long run. I think because of the difficult times we are all going through with the current pandemic until teams can ramp back up and bring employees back, there are ways you can continue to operate and do more with less. So, if you’re feeling overwhelmed because you’re shorthanded and not quite sure how to progress forward, my hope is that these tips will at least point you in the right direction and perhaps innovate and drive positive change.

    ____________

    Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

    About Acorn 

    Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

    Reduced Total Cost of Review Comes by Working Smarter, Not Cheaper

    From my client’s point of view, the most important budget consideration is the total cost of review. That is, the total cost for all the technology, professional services, and contract review services associated with review.

    It’s easy to assume that if you want to reduce the total cost of review, then you should try to reduce the various line item costs that go into eDiscovery and Review. However, the biggest economic driver in reducing the total cost of review is finding technology-based approaches to reduce the total number of documents reviewed. This approach generates significantly more cost savings than trying to negotiate line item discounts.

    I wanted to step back and illustrate how those economics work. Various pieces of the process fit together to significantly impact the total budget. I’m going to take a standard, illustrative scenario and outline the biggest sources of cost savings from more strategic thinking at the outset.

    The Baseline Budget

    In a traditional linear review, up to 70% of the money is spent on the labor of reviewing the documents. In this traditional approach, documents are collected and culled based on date ranges and search term filters. Then, every document in the culled data set is manually reviewed for responsiveness.



    Of that $100k budget, 70% of the budget spent on review, a further 80% of the effort is spent eliminating non-responsive documents. To put that in context, for a $100k case, $56k is spent doing work that adds little to no value to the merits of the case.

     

    This is a common, widely-recognized problem, and industry experts are trying to take steps to mitigate this “waste”. A common solution is attempting to reduce hourly review rates by offshoring the work to reduce the total cost of review.

     

    However, there are limitation to that approach, as it does not address the underlying issue. Quite simply, reducing the “cost per document reviewed” is not as effective as reducing the “total documents reviewed” by using more strategic planning with an experienced advisor in the beginning of the review process.

    In fact, the biggest economic driver in reducing the total cost of review, is finding technology-based approaches to reduce the total number of non-responsive documents reviewed.

    Let me walk you through the economics, with some examples.

    Example 1: Save $24.5k By Using Artificial Intelligence to Further Narrow the Document Set Before Review Begins



    The fundamental challenge with managing the cost of eDiscovery reviews is the low responsive rate of documents. In our scenario, only 4% of all the documents collected are responsive. That means that out of 400,000 documents collected, only 20,000 are relevant.

    A common approach is to use search terms to narrow the document set. In our scenario, that means that of those 400,000 documents collected, 100,000 still need to be reviewed. At a fully burdened cost of $0.70 per document, that still means that $70k is spent on review.

    AI, combined with professional research services, can be used at the beginning of the matter to further refine search terms, eliminate data sources and narrow custodian lists. When that service is used, the reviewable document set can be reduced by 35%. So, instead of reviewing 100,000 documents, only 65,000 documents are being reviewed. This approach results in a net savings of $24.5k.

    Example 2: Save $9.5k By Using AI to Reduce the Cost of Poor Quality During the Review



    In a typical review, there are first pass reviewers, and second pass reviewers. The first pass reviewers look at the documents in bulk. While the second pass reviewers are more sophisticated reviewers, and randomly sample the first pass reviewers’ work to ensure accuracy. The first pass reviewers are typically billed out at lower rates, for example $0.50 per document. While the second pass reviewers are typically billed out at higher rates, for example $1.00 per document.

    Take the example of the 100k documents that are being reviewed offshore. $20k of the labor cost is due to quality control issues. With more strategic use of the technology, that number can be reduced to $11.5k.

    This is due to a couple factors. First, about 10% of those documents are going to need to be reworked, because the first-pass reviewer won’t meet quality standards. Simply, an inattentive reviewer might code every document the same or, perhaps, missed instructions provided by the review manager. So, 10k of those documents need to be reviewed twice, costing $5k in rework. In addition, 15% of the documents need to be sampled by a subject matter expert to ensure that the review is being completed accurately, costing $15k in quality control measures.

    Using AI to guide the review can drive significant efficiencies in the review operations. First, “bad reviewers” are identified more quickly, so instead of potentially having $5k in rework, you could only have $2k. Second, because the AI is actively monitoring and alerting review managers to the front-line reviewer issues, the sample rate of second pass reviewers can be reduced to 7% instead of 15%. This reduces second pass reviewers’ cost by $8k.

    Successfully identifying problems early on lets you remedy them immediately, saving expensive rework and reducing overall QC costs by $10k. These features typically come at no additional cost, so net savings are $10k.

    Concluding Thoughts – Reduction in the Total Cost of Review Using AI Is $31.6K (32%)

    Using the two techniques outlined above, proper utilization of AI can collectively reduce the total cost of review by 32% This is substantial savings that can either be reallocated towards more valuable legal advisory work or can be used as a cost reduction measure in managing litigation expenses. While this methodology does not completely eliminate the need to review some non-responsive documents, it greatly reduces that portion of the review and the associated costs.



    Acorn’s been a leader in AI and review process design for 7 years. A side benefit to using all this AI is that it requires smaller review teams, which makes staffing easier. The goal isn’t to work more cheaply with lower quality service; it is to work smarter, which in turn, reduces the total cost of review.

    AI Trains Your Model to Be Even More Accurate over Time – Savings Tomorrow, AND Today

    If your firm or corporation works with one or two litigation models—maybe you do all IP or all medical malpractice litigation—you probably end up going through documents, data and even fact patterns that look similar, even as your players and exact circumstances change, or you may, in fact, have the same custodians on multiple matters.

    The right AI eDiscovery system lets you reuse your existing work product on future cases, without comingling data. Once your AI model is properly trained, it can be applied over and over again to different cases, strengthening your model and improving its results each time, without the costs of starting over on every new case. Instead of retraining your team every few months, you can rely on the AI model to keep building on its own training.

    ____________

    Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

    About Acorn 

    Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

    My Take On How The New Michigan eDiscovery Rules Will Affect Michigan Litigators

    I’m hearing a lot of questions being asked about the new Michigan eDiscovery Rules that have just went into effect as of Jan 1, 2020. There are a lot of resources to educate the market on these new rules, including Judge Fresard’s recent Bench and Bar luncheon that addressed these changes and how they are going to impact the legal community. My own company, Acorn, has published a brief primer on the changes. I’m not a lawyer, but here is how I think about what is going on with these new changes.

    Although there will be a transition period, in the long run, the changes are going to make eDiscovery and litigation more manageable for Michigan attorneys.

    There are a lot of eDiscovery experts out there to help Michigan attorneys deal with the new discovery rules. You don’t need to be an expert in eDiscovery, you just need to know an expert and know what questions to ask. A lot of attorneys feel like they need to be the expert themselves, but the truth is that staying on top of eDiscovery is a full-time job. It would be impossible for someone to be both an expert in litigation and in eDiscovery. I’ve been preaching this to client’s and prospective clients for years now.

    The Duty of Technological Competence is something that I believe will help all attorneys not only fulfill their obligation, but more importantly help their overall case strategy. Coming up with a case strategy for discovery early on and assessing what potential ESI is out there, will then allow you to develop ESI protocols, help identify custodians and give you a better understanding of costs as well.

    Custodial interviews are the simplest of ways to get the “lay of the land” within your client’s organization. Take the interviews and add them with the understanding of ESI and your interviews will become more powerful. Understanding how technology is being used daily within an organization and how that will affect ESI will allow you to identify sources, location of sources, potential key communicators, and perhaps additional custodians.

    In the short term, thought leaders in the local Michigan legal community are being very supportive to help litigators across the state develop best practices around the new rules.

    I cannot express this enough. The local support and legal community in Michigan is very strong. There are advocates for education and thought leadership that can be great assets. The Bench and Bar event was one recent event that helped educate on how to be prepared for the new rules that have taken place. Judge Fresard did a great job coming to ACEDS Detroit and really connecting with local Vendors to make sure we understood the importance of supporting the bench and bar event and these new changes. There are many local vendors that have a strong understanding of the technical aspect and legal implications to time and budget, which can provide great value to legal teams.

    If you’re a lawyer, I’ve firsthand seen the passion and push for more education on these changes from attorneys like Scott Petz (Dickinson Wright), Megan McKnight (Tealstone Law) and Jay Yelton (Warner Norcross) just to list a few. All of them can be great external resources and have years of experience dealing with ESI. They are heavily involved in ACEDS Detroit and do a great job introducing new ideas for ESI training & consulting.

    Other states have dealt with similar rule changes – so there is some precedent for how Michigan will likely handle the changes.

    A lot of states are dealing with rule changes around duty of technology competency and with eDiscovery. California was early in pushing these changes in 2015. Although, interestingly enough, they pushed the changes through an ethics opinion rather than a formal rule change. 38 states have adopted duty of technology competency requirements, going as far back as 2013. So, although I’m not a lawyer, I imagine there have to be some examples there for Michigan attorneys.

    Concluding Thoughts.

    Attorneys that have embraced technology and looked at these challenges as opportunities, have had great success. Client expectations for cost effective methods to case strategy and the use of technology continue to rise. If you’re an attorney that decides to look at these new changes and takes the time to understand how it can be utilized as weapon instead of it being looked at as a hurdle, you’ll see even greater success with your clients.

    This is extremely exciting for me personally, because I spend a lot of time trying to advise my clients on the most efficient and cost-effective ways to handle eDiscovery. Remember, the role of your eDiscovery provider is to make this easy and understandable and not super technical and burdensome. Change is never easy, but you’re not in this alone. Do not hesitate to reach out to any of the people I mentioned, or feel free to contact me directly.

    ____________

    Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

    About Acorn 

    Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.