Author: admin
How I Turned A Reluctant Attorney into An Active Learning Enthusiast (Even in a Worst-Case Scenario)
“Based on my encounters with Relativity Active Learning, I started to think active learning was a very blunt and obtuse tool. Our database contained a lot of non-responsive documents that used the same words as the responsive communications. Half the documents in the database were irrelevant, but we had no easy way to weed them out. That actually has been one of the biggest difficulties we had getting a handle on this database.”Over the years I have worked on more cases than I can count, and I still always appreciate the moments when I’m able to translate my experience to solve someone’s issue. Personally, I am an analytics guy through and through, from how I manage projects to how I operate in my own day to day life. Because of this, I try to educate people on how they can apply analytics and advanced technologies into their practices. A case I worked on recently comes to mind of where I wasn’t able to solve all the issues, but was able to offer improvements, including some notable improvements in select areas that could be applied to a wide range of matters.
I was working with a client that was very knowledgeable on active learning, in a dataset that is every TAR guy / gal’s worst nightmare. They came to me because they needed a Hail Mary for the case. The data was produced inconsistently, some had metadata, some didn’t, some in pdf format, you name it, they had it. The client wanted to evaluate whether a combination of my expertise and StoryEngine’s advanced technology might work better than their current methodology with Relativity Active Learning (Prioritized Review) solution.
These are the three takeaways I learned from this case:
Overcoming Roadblocks
First – I utilized NexLP’s StoryEngine, which was far superior at identifying hot documents based on their author. Due to the words and structure of the documents being almost exactly the same, the client was skeptical the software could differentiate between them. However, StoryEngine picked up on hot documents based on who the author was, and we could manipulate the model to prioritize this feature even more. This allowed us to overcome a seemingly unavoidable roadblock in our workflow.Fully Utilizing the Tools
Second – propagation and near duplicates were a hurdle to efficiently and effectively reviewing the document set. While you can turn propagation on in Relativity active learning, StoryEngine does it automatically which streamlines the process. This prevents simply overlooking the feature and not understanding why you aren’t obtaining your expected results or having to backtrack your process to implement it, wasting time and money.Identifying the Smoking Guns
Third – the model was able to stabilize quicker due to a combination of coverage queue and better prediction models in StoryEngine. When combined with StoryEngine’s emotional intelligence and various other capabilities, it could pinpoint specific documents of interest more efficiently and demonstrate the hidden value that could be easily overlooked or buried deep.Concluding Thoughts
The greatest takeaway was seeing the Client’s do a full 180 and fully embracing the technology. Although this solution isn’t a do-all be-all, the model is still very applicable in certain scenarios. This case showed the client and I that active learning can be customized and modified to fit all types of scenarios. With the right guidance and expertise, we can use analytics for all kinds of data types. The difference between being able to pull additional value out of the remaining data comes down to nuanced analysis and expertise, like we did here. I’m proud that even under these bad circumstances, I was able to turn a TAR detractor into a TAR promoter.____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.Three Things That Actually Set eDiscovery Providers Apart
Finding the Correct Type of Vendor
When I was on the tech side of eDiscovery, clients finding the correct type of service provider is something that seemed particularly relevant when it came to international cases. There were corporations that would only focus on which service providers had our product and had locations in multiple countries where the corporation also had offices. The more I have been on the services side, the more I realized that finding the correct match matters. Besides international vendors, a lot of vendors will concentrate on infrastructure as a service or specialize in assisting corporations in general, while others concentrate solely on AM100 law firms or Fortune 500 corporations. The reason why it’s important to know this, is their standard operating procedures and workflows are designed with this type of customer in mind.At Acorn, we are built from the ground up to be the outsourced litigations support team for regional powerhouse law firms and national mid-sized firms. This includes everything from our invoicing procedures, to the technologies we utilized, to the workflows we implement. Acorn understands that this group is often going against AM100 firms, but they don’t have the same resources that their competitors might have. That is where Acorn comes in to bridge that gap and provide our clients with the same, if not better, arsenal of tools and services at their disposal.
Finding the Correct Type of Vendor
When moving from technology to services, I knew that quality project managers were essentially. Part of the reason I chose Acorn over some of the other offers that I had, is because I believed in their project managers so much. What I initially overlooked, but realized is very important, is matching the correct project manager to the client. One reason why this is so important is matching personalities. Any client is going to have a better experience if they can personally connect to the people that they work with. This isn’t saying that not everyone in this industry is likable, its more about knowing which personalities will mesh best together. It’s also my job and the job of the project manager to know what the possible technologies are that can best serve the client and will be beneficial in the future to help with common issues the firm faces. When it comes to using technology, experience matters, and I want my customers to have the best possible experience.Knowing Your Customers
Having the whole team know your clients is an important part of providing good customer service. For service providers to be successful, they must know their biggest clients inside and out. What I’ve noticed since switching over is that there are a lot of large firms and corporations that aren’t their vendors biggest client. In a post article, there were several comments on how leadership in some organizations put an emphasis on efficiency instead of customer service. Unfortunately, this efficiency can lead to clients having different project managers and people working on their cases for every case. It becomes hard to know your client if you only have them for one case. This is the main reason why we have dedicated project managers for our clients. Having myself and the project manager truly understand the client’s needs, personalities, preferences, and power structure allows us to provide better service.Concluding Thoughts
This all seems straightforward in concept, but the challenge is putting it to practice consistently with good execution. So, as you contemplate relationships with eDiscovery vendors, you might want to ask questions like: How do you evaluate project managers’ technical and client-facing skills during recruiting? What sort of on-going education programs do you offer your project managers and business development managers? Does your vendor have integrated service delivery processes for technical, client management and accounting teams? What sort of on-going education program does your vendor offer your clients? Does your vendor have standard operating procedures around management of the entire project portfolio with you, in addition to each individual project? Where do you rank in importance as a client for your vendor?____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.How You Can Accomplish More with Fewer Reviewers
So, with that, how do you accomplish the same amount of work, with a smaller team, remotely, in the same amount of time?
I wanted to step back and look at the math behind that question. Assuming the budget is not an issue here, and the deadline can’t be moved, what are the levers you have to hit your deadline?
The Baseline Budget
Let us take the scenario where you have 500k documents that need to be reviewed in 13 weeks and you were planning on using 20 reviewers. With COVID and remote work so prominent now, you don’t think you can successfully staff and manage more than 5 reviewers. If you didn’t change anything about the process, your review would now take 48 weeks.
Current industry standards are that reviewers generally work 40 hours / week and can review 50 documents per hour. If you wanted to have 5 reviewers accomplish what 20 reviewers did, you would need them to work 160 hours/week and off barely 2 hours of sleep. Or, if you wanted them to accomplish it smarter, they would have to review 200 documents/hour which not many people in the world could do, unless you’re Howard Berg.
Even if you do some combination of better productivity, like say 80 docs/hour, and more hours, like say 60 hours/week. Your team of 5 is still going to going to go 242% over the allocated time (or 17 weeks).

Improvement 1: EaiRLY INSIGHT™ Investigation
Save 16.4 weeks by using EaiRLY INSIGHTS™ to defensibly eliminate custodians and remove their documents from the data set to be reviewed.
With this technique, you can defensibly eliminate custodians and data sources. In a typical matter with 10 custodians, who have had emails, laptops and mobile devices collected, usually, two custodians and one data source can be eliminated through this investigation. In our 500k document example, that would amount to 164,000 documents eliminated (100k documents from two custodians, and 64k from largely non-responsive laptop data sources). This saves 16.4 weeks of review — a pretty good start.
Improvement 2: EDA and Search Term Analysis
Save a further 8 weeks by further narrowing the document set with smarter search terms, informed by EDA and EaiRLY INSIGHT ™ Findings.
With this approach, we typically see improvements in culling from 25% to 35% per custodian. In our sample case, that ultimately saves 10k documents per custodian over the 8 custodians, meaning that 80,000 documents don’t need to be reviewed and 8 weeks of time is saved.
Improvement 3: All-Star Reviewers
Save 15 weeks by increasing reviewer throughput by leveraging advanced analytics and increase their weekly hours from 40 to 60.
Often times, it’s the simplest of things that can make the world of difference. Being more strategic about the way you batch documents and prepare them for the review team can make a significant difference in review speeds. For example, batching by concept allows each reviewer to be more familiar with the pertinent issue, so they can move through documents more quickly. Or batching by near-duplicates allows the reviewer to quickly transfer knowledge from the first document reviewed to the remainder of the batch, increasing review throughput. On the more technical side, it may be worthwhile to begin separating all PDF documents so that all reviewers are not waiting for large PDF’s to load, wasting unnecessary time.
Through this, your reviewers can do the equivalent to 80 Docs/Hour compared to the baseline of 50 Docs/Hour, saving an additional 9.6 weeks of review. When you combine that with increasing each reviewers’ weekly hours to 60, you save 15 weeks in total.

Closing thoughts
Obviously, this is more of a thought experiment than an actual real-world scenario. In reality, you’d probably push to staff more reviewers or push to move the deadline. But, I do think it’s helpful to understand the mathematical drivers behind scenarios like this. Ultimately, you can get more done by working reviewers longer and harder. But, you get the most bang for your buck, by figuring out how to reduce the dataset at the outset.In our scenario, although we would probably spend a bit more time upfront with the EaiRLY INSIGHT ™ investigation, it more than pays off in the long run. I think because of the difficult times we are all going through with the current pandemic until teams can ramp back up and bring employees back, there are ways you can continue to operate and do more with less. So, if you’re feeling overwhelmed because you’re shorthanded and not quite sure how to progress forward, my hope is that these tips will at least point you in the right direction and perhaps innovate and drive positive change.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.eDiscovery Considerations with Business Use of Zoom
The number of daily meeting participants in Zoom jumped from 10 million in January to 300 million in April, a nearly 3000% increase in three months. It is evident from these numbers that Zoom usage has increased exponentially as companies have had to shift to a new way of working. As with any electronic data, any new source of data a company is using must be carefully considered in terms of retention policies and potential legal obligations.
The use of Zoom as a business communication tool was likely implemented quickly and without enough time to consider all the potential eDiscovery implications. Consider Zoom as a new source of electronic data, like mobile phones, or Slack, or social media. It is important to understand what kind of data is being created and how that data should be handled. Additionally, because Zoom allows individual users to record meetings, the location of the data can also become very important. The bottom line is that Zoom is creating discoverable data. The goal of this article is to advise on the potential eDiscovery issues that could arise if a comprehensive and clear policy around the use of Zoom is not created and implemented now.
Zoom Meetings that Are Not Recorded
Most companies can clearly see data containment issues if video is being preserved. But what if a Zoom meeting is not recorded? It is important to understand what types of records are being created when the meetings are not recorded to understand how it could impact discovery obligations in the future. Any time that a zoom meeting occurs, Zoom retains a record of that meeting. As a basic user, you can view under the Meetings tab, a record of every Zoom meeting that you have scheduled or attended. The information contained on that tab includes the date, time, meeting ID, and title of the meeting. This area also allows the individual user to delete a meeting from their history. From a discovery and collection perspective, this information would be both hard to collect and not substantively very useful.Administrators and owners of the account, however, have much deeper visibility into their organization’s usage of the platform. An administrator can view monthly reports on every Zoom meeting that has occurred, regardless of whether the meeting was recorded. Under the Reports tab, and then under Usage Reports, the Administrator can view and export daily usage reports that identify the number of new users, meetings, participants, and total of meeting minutes for each day. They can also drill down further for reports on their Active hosts and export a meeting report for any 30-day period of all the meetings that an organization has conducted over Zoom.
On this report is some particularly valuable information including the topic of the meeting, meeting ID, the user who set up the meeting, the email of that user, department, group, creation of the meeting time, start and end time of the meeting, duration of the meeting, the name and email addresses of the attendees, join time and leave time for each participant, along with duration. Administrators are also able to export reports about inactive users, upcoming events, meeting registration or poll reports, and an overview of the cloud recording storage capacity. These reports can be retrieved for the previous twelve months limited by a 30-day search range. The reports are exported as CSVs. The only way for an administrator or owner to delete the usage activity is to delete the user. Deleting a user will permanently remove the user, including their meetings and recordings from Zoom.
Although the information that is retained for non-recorded meetings is not as expansive as a recorded meeting would be, there are many situations in which the failure to retain this data could run counter to preservation obligations. Federal and most local rules of civil procedure allow discovery regarding any non-privileged information relevant and proportional to the needs of the case. Depending on the litigation, a spreadsheet outlining that certain people were attending certain meetings on certain dates could be relevant. Companies need to be cognizant of the way this information is being maintained, saved, and collected to ensure they can comply with discovery obligations later.
Zoom Meetings that Are Recorded
Zoom also allows for the recording of meetings, either locally to the computer or to the cloud. When a meeting is recorded, the following files are created: MP4 video file; M4A audio file; and a txt file of any chats that occurred during the meeting. Zoom also offers the ability to transcribe the meetings so you have a text record of what was said during the meeting. If you enable the audio transcript option under Cloud Recording, Zoom will automatically transcribe the audio of the meeting and include a separate .vtt text file. In addition to the data preserved for any meeting outlined above, these are additional data sources created by the Zoom meetings that are recorded. Zoom will retain the cloud recordings, including the files listed above, until the storage capacity of the account is exceeded. Administrators can access and preserve any cloud stored meetings. These meetings are located under the Account Management, Recording Management area of the Administrator profile. However, the administrator can only access the recordings that have been stored to the cloud.Depending on the account settings, as determined by the administrator or owner of the account, individual users or participants are also able to locally store recorded copies of their meetings. This means that anyone who attends a meeting can keep a permanent copy of the meeting’s audio, video, and chat. From a discovery perspective, this is a data mapping nightmare. If you have a company with 50 people who are using Zoom on a daily basis to conduct business, and each one of those people is recording all their meetings locally, there are now 50 locations that could contain discoverable data. Picture the costs associated with collecting 50 different people’s local zoom recordings folder from their work or personal computer?
Companies should be proactively thinking about what types of meetings need to be recorded and providing constructive guidance to their employees. There are industries where there is a legal obligation to record transactions such as banking and publicly traded companies. There are industries where recording communications with clients can turn into an ethical morass, such as law firms or in-house counsel at a corporation. Obviously, there is no one size fits all approach to the duty to record or not. Another area to consider is whether you have employees that despite retention and compliance policies are still creating unauthorized recordings. If your company gets sued, and those recordings fall under the relevance and proportionality rules, how are you going to locate and preserve that information?
The major question that should be considered and outlined clearly in policies is the types of meetings that should be recorded. The second is when or if you should delete these recordings. If you hold a Zoom meeting where senior leadership is discussing the termination of an employee that could potentially turn litigious, are you under an obligation to record that because there is a reasonable anticipation of litigation and the duty to preserve applies? There is no clear-cut answer to that question as these cases have not yet been litigated. However, companies should be carefully considering how, when, and why they deploy the recording function.
From an eDiscovery perspective, the Zoom recordings create an expensive proposition. Consider the example outlined above, if you have 50 relevant recordings and they are all located on each person’s individual computer, the company must then go and collect and process all of these files into a platform so they can review. Video and audio files typically are very large file sizes which makes the hosting of this data expensive. Complicating matters further, if a user does not opt for a transcript of the meeting, the video and audio files require significant customization to be searchable. How will a company know if a specific meeting is subject to a legal hold if you can’t text search the content of that meeting?
Best Practices and Considerations
Next, we want to outline some best practices and considerations surrounding the use of Zoom in a business context. Companies should, if they have not already, be drafting and implementing policies regarding the usage of Zoom. These policies should encompass when and how Zoom should be used, when the meetings should be recorded, and where the recordings should be stored. Companies should also be authorizing the Zoom administrators to create system wide permission settings to ensure that the data they retain is contained in one central repository. Employees should not be able to save locally, rather a universal save to the cloud requirement should be enforced. An administrator can enforce company-wide objectives for all users that require how and when they record and where that recording is stored. As administrators can also delete recordings on the cloud, they should be provided with clear instruction of what should be saved and what can be deleted. This way companies can easily identify and collect potentially discoverable information.Administrators should be cognizant of the timelines for deletion previously outlined, and ensure they download any available reports or recordings monthly. They should also be aware of the storage capacity in Zoom so potentially relevant data is not accidentally lost. It would also be beneficial for companies to create an audit log for meetings that occur and ensure that employees update continuously. The audit log would contain the date, time, attendees, and what was discussed. Adding transcription to recorded meetings will also make the information far more searchable and functionally useable in the future.
As we have outlined, there are many business decisions that must be developed around the use of Zoom to collaborate. Companies should be proactively providing guidance to their workforce around the use of these types of applications. We can provide further guidance on best practices.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.How to Prevent Yourself from Going Stir Crazy, While Also Being Productive
I’m in a different boat than most. As a bachelor, I’m staying at home by myself. At first, the Stay at Home Order wasn’t too bad. I got to finish some books, catch up on some TV I’ve had recorded, and binge watch my favorite streaming platforms. After a month, I streamed every show I’ve ever wanted to watch, read every book I haven’t yet read in my condo, and started to do arts and crafts (Cast Away was one of the movies I streamed). I was running out of activities, so recently, I began compiling some things to do to keep from going stir crazy while remaining productive.

1) Watch Webinars and Listen to Podcasts
I have always enjoyed watching webinars because I want to learn as much as possible. What new technologies are out there, what workflows are people implementing, what are new trends that I may not be seeing, to name a few. The problem is, for a lot of the webinars I sign up for I end up being too busy and by the time things cool down, I forget that I even signed up. Here are some of my go to places for webinars, but I’d love to hear if you have more.- ACEDS has more webinars than you’ll know what to do with, especially since local chapters have had to postpone their meetings and events.
- The EDRM has also been putting out a lot of webinars, and has over 100 recorded webinars that you are easily accessible to look back on.
- Acorn holds weekly events, including round tables and mock trials with industry thought leaders as a part of our 20% Interaction Time series.
- Stellar Women in eDiscovery is a monthly podcast, hosted by Relativity, spotlighting and celebrating female leaders in the industry we know and love.
- ILTA has been publishing insightful content for years now, and I’m told that their podcast is no different.
- Uncivil Procedure is a monthly podcast revolving around informal discussions on recent and relevant case law.
2) Study for Certifications
Since I’ve been in eDiscovery, I’ve always loved the idea of obtaining certifications, but it has always been one of those things that I’ve told myself that I’ll get to another day. 6 months later, it was still low on my list of priorities. With an abundance of free time, I think my calling has finally come to start studying. Relativity seems to be on the same page as me and has started offering free training and certifications. I have my eye on the Relativity Certified Sales Pro Certification but there’s an abundance of other options that I’m sure will pique your interest, such as:- ACEDS is the industry standard when it comes to certifications. I’ve always told myself that I was going to study for it by now I finally have time.
- Relativity has new free resources to keep your eDiscovery skills up to par while you’re at home.
3) Attend Virtual Coffees, Lunches and Happy Hours
Social interaction is essential for your mental well being. For those of us living by ourselves right now, it’s more important than ever. Whether it’s doing video calls with your friends, family or colleagues, I highly recommend reaching out in any way you can. I’ve started holding standing Happy Hours every Wednesday night for my industry peers, friends and family every week. This month I launched my first Virtual Happy Hour to reconnect with my industry colleagues and peers. I attended (and hosted), and if I do say so myself, it was a lot of fun connecting with numerous old and new faces and sharing how we’ve been doing inside and outside the “office”. Beyond Happy Hours, my company Acorn has set up a full roster of events to allow our community to stay connected which all can be found on Acorn’s 20% Interaction Time homepage. A few of the events are:- Top Chef Madness: Weekly Discussions with Lia Majid
- Coffee with Colleagues with Zef Deda
- Office Hours with our Project Management Team
This is a tough time for everyone, and I’m hoping that everyone out there is staying safe and healthy, mentally and physically. I’m going to try to use this time to be as productive as possible, learning as much as possible. If anyone wants to connect, feel free to connected with me and we can set something up or collaborate to hold future events.

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.Reduced Total Cost of Review Comes by Working Smarter, Not Cheaper
It’s easy to assume that if you want to reduce the total cost of review, then you should try to reduce the various line item costs that go into eDiscovery and Review. However, the biggest economic driver in reducing the total cost of review is finding technology-based approaches to reduce the total number of documents reviewed. This approach generates significantly more cost savings than trying to negotiate line item discounts.
I wanted to step back and illustrate how those economics work. Various pieces of the process fit together to significantly impact the total budget. I’m going to take a standard, illustrative scenario and outline the biggest sources of cost savings from more strategic thinking at the outset.
The Baseline Budget
In a traditional linear review, up to 70% of the money is spent on the labor of reviewing the documents. In this traditional approach, documents are collected and culled based on date ranges and search term filters. Then, every document in the culled data set is manually reviewed for responsiveness.
Of that $100k budget, 70% of the budget spent on review, a further 80% of the effort is spent eliminating non-responsive documents. To put that in context, for a $100k case, $56k is spent doing work that adds little to no value to the merits of the case.

This is a common, widely-recognized problem, and industry experts are trying to take steps to mitigate this “waste”. A common solution is attempting to reduce hourly review rates by offshoring the work to reduce the total cost of review.

However, there are limitation to that approach, as it does not address the underlying issue. Quite simply, reducing the “cost per document reviewed” is not as effective as reducing the “total documents reviewed” by using more strategic planning with an experienced advisor in the beginning of the review process.
In fact, the biggest economic driver in reducing the total cost of review, is finding technology-based approaches to reduce the total number of non-responsive documents reviewed.
Let me walk you through the economics, with some examples.
Example 1: Save $24.5k By Using Artificial Intelligence to Further Narrow the Document Set Before Review Begins

The fundamental challenge with managing the cost of eDiscovery reviews is the low responsive rate of documents. In our scenario, only 4% of all the documents collected are responsive. That means that out of 400,000 documents collected, only 20,000 are relevant.
A common approach is to use search terms to narrow the document set. In our scenario, that means that of those 400,000 documents collected, 100,000 still need to be reviewed. At a fully burdened cost of $0.70 per document, that still means that $70k is spent on review.
AI, combined with professional research services, can be used at the beginning of the matter to further refine search terms, eliminate data sources and narrow custodian lists. When that service is used, the reviewable document set can be reduced by 35%. So, instead of reviewing 100,000 documents, only 65,000 documents are being reviewed. This approach results in a net savings of $24.5k.
Example 2: Save $9.5k By Using AI to Reduce the Cost of Poor Quality During the Review

In a typical review, there are first pass reviewers, and second pass reviewers. The first pass reviewers look at the documents in bulk. While the second pass reviewers are more sophisticated reviewers, and randomly sample the first pass reviewers’ work to ensure accuracy. The first pass reviewers are typically billed out at lower rates, for example $0.50 per document. While the second pass reviewers are typically billed out at higher rates, for example $1.00 per document.
Take the example of the 100k documents that are being reviewed offshore. $20k of the labor cost is due to quality control issues. With more strategic use of the technology, that number can be reduced to $11.5k.
This is due to a couple factors. First, about 10% of those documents are going to need to be reworked, because the first-pass reviewer won’t meet quality standards. Simply, an inattentive reviewer might code every document the same or, perhaps, missed instructions provided by the review manager. So, 10k of those documents need to be reviewed twice, costing $5k in rework. In addition, 15% of the documents need to be sampled by a subject matter expert to ensure that the review is being completed accurately, costing $15k in quality control measures.
Using AI to guide the review can drive significant efficiencies in the review operations. First, “bad reviewers” are identified more quickly, so instead of potentially having $5k in rework, you could only have $2k. Second, because the AI is actively monitoring and alerting review managers to the front-line reviewer issues, the sample rate of second pass reviewers can be reduced to 7% instead of 15%. This reduces second pass reviewers’ cost by $8k.
Successfully identifying problems early on lets you remedy them immediately, saving expensive rework and reducing overall QC costs by $10k. These features typically come at no additional cost, so net savings are $10k.
Concluding Thoughts – Reduction in the Total Cost of Review Using AI Is $31.6K (32%)
Using the two techniques outlined above, proper utilization of AI can collectively reduce the total cost of review by 32% This is substantial savings that can either be reallocated towards more valuable legal advisory work or can be used as a cost reduction measure in managing litigation expenses. While this methodology does not completely eliminate the need to review some non-responsive documents, it greatly reduces that portion of the review and the associated costs.
Acorn’s been a leader in AI and review process design for 7 years. A side benefit to using all this AI is that it requires smaller review teams, which makes staffing easier. The goal isn’t to work more cheaply with lower quality service; it is to work smarter, which in turn, reduces the total cost of review.
AI Trains Your Model to Be Even More Accurate over Time – Savings Tomorrow, AND Today
If your firm or corporation works with one or two litigation models—maybe you do all IP or all medical malpractice litigation—you probably end up going through documents, data and even fact patterns that look similar, even as your players and exact circumstances change, or you may, in fact, have the same custodians on multiple matters.The right AI eDiscovery system lets you reuse your existing work product on future cases, without comingling data. Once your AI model is properly trained, it can be applied over and over again to different cases, strengthening your model and improving its results each time, without the costs of starting over on every new case. Instead of retraining your team every few months, you can rely on the AI model to keep building on its own training.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.Finding PII and PHI: Three Ways AI Leads to Better Outcomes
With so much data, litigators and their end-clients have historically only had two options for identifying where PII might be: either spend a lot of time and money to review every document that will be produced for potential redactions or rely on regular expressions (RegEx) or take a calculated risk and accept that while RegEx will catch most PII/PHI, some may inadvertently get produced or put in the public record.
Neither option was good. Either litigators and their end-clients had to spend a ton of cash for a comprehensive review, or they had to accept that part of the cost was a high likelihood of reputation damage due to the inadvertent production of PII.
But now, with AI, there’s a new option. And it’s time to reevaluate how you think about the economics of redaction. RegEx is no longer the best way to get the outcomes you need—AI is. And while it can seem like a black box, it’s actually a much less risky approach that can give you a holistic view of relevant data.
With AI, you can efficiently cross-check what needs to be redacted, what your review process has missed, or what has been redacted by mistake. Now, I’ll explain how traditional RegEx software can fail, and provide some concrete examples of how AI-enabled software can provide a solution.
1. AI Can Evaluate Context to Identify PII That Does Not Follow the Typical Regex Format
RegEx have always been an efficient way to find PII that is in a standard format. It’s often used to find phone numbers, SSN, addresses and more. The problem with finding PII with RegEx is that the data needs to be in that exact format to be found.Take for example addresses. Addresses vary so much in how they are formatted that it becomes impossible to write a RegEx that will find all of them. An AI-enabled software program can solve that problem, though. Instead of having to list out every possible address formation, we can teach the AI on the normal RegEx, showing it what is an address and what isn’t by what’s actually written and what context surrounds it.
If you’ve ever typed an address into Google, you know what I’m talking about. How many times have you typed some combination of numbers and street names into Google and had the search engine’s AI figure out exactly where you were talking about, misspellings or abbreviations and all?
With Entity Modeling, a feature of the NexLP software that builds on the same AI we use for technology-assisted review (TAR) and applies it to PII, you can train different models on different pieces of PII. For any given case, you can apply only the models that correspond to the regulations you need to follow for that case.
Once the NexLP’s PII modeling’s AI knows what it’s looking for, it can search for documents that may have that PII, identifying the information you were looking for in all its various forms.
2. AI Can Prevent Over-redaction Risk, Where Names Are Also Commonly Used Words (E.G. Mr. Brown and Brown the Color)
Searching for names, and variations that the names come in have always been the standard when looking to redact names, or product names that need to be redacted. The problem comes into play when the name or product name is also another word. Brown is sometimes a last name and thus may need to be redacted as part of PII. But “brown” can also be a word used to describe the color of a coat or describe the action required to cook onions, and those other uses wouldn’t fall under PII and wouldn’t need to be redacted.Under-redacting can cause problems with regulations and rules, but over-redacting is also not ideal. It can get you into trouble with your opposing counsel or the court, and can present an expensive risk. Standard software can confuse those uses, and an eyeballs-on-documents approach requires significant resources. Context-aware AI can sort through those different contexts like a human, but much faster and much cheaper.
3. AI Can Help You Find PII That You Might Not Know Is in the Document Set
There are times that you will need to redact anyone not relevant to the litigation, but you don’t have a reliable master list to reference. Or you need to comply with a law that requires you to anonymize the names of students or other names, but no comprehensive list of names exists. What are your options?Historically, you were looking at a long and tedious manual review, or trying to custom-program current software. An AI-enabled solution can automatically create a master list of names. The AI model combs the document, using the context it’s been trained on to identify the right names.
This way, instead of reviewing every document yourself, your trained AI model will do that work for you, giving you a holistic view of the exact information you need.
Final Thoughts
I’ve had conversations about redactions on everything from high profile cases to DSARs, with people from corporations, law firms and most vendors in the space. Redactions are expensive, and probably the least favorite thing of any reviewer. I’ve seen a dramatic switch in how people are thinking about PII/PHI.I’ve mentioned some great new ways to find PII/PHI above, but I also think that the people working on it and the process are also important. Some of the workflows above, if used incorrectly, could lead you to producing PII/PHI.
I’d love to hear if you have other workflows to find PII/PHI.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
About Acorn
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.My Take On How The New Michigan eDiscovery Rules Will Affect Michigan Litigators
Although there will be a transition period, in the long run, the changes are going to make eDiscovery and litigation more manageable for Michigan attorneys.
There are a lot of eDiscovery experts out there to help Michigan attorneys deal with the new discovery rules. You don’t need to be an expert in eDiscovery, you just need to know an expert and know what questions to ask. A lot of attorneys feel like they need to be the expert themselves, but the truth is that staying on top of eDiscovery is a full-time job. It would be impossible for someone to be both an expert in litigation and in eDiscovery. I’ve been preaching this to client’s and prospective clients for years now.The Duty of Technological Competence is something that I believe will help all attorneys not only fulfill their obligation, but more importantly help their overall case strategy. Coming up with a case strategy for discovery early on and assessing what potential ESI is out there, will then allow you to develop ESI protocols, help identify custodians and give you a better understanding of costs as well.
Custodial interviews are the simplest of ways to get the “lay of the land” within your client’s organization. Take the interviews and add them with the understanding of ESI and your interviews will become more powerful. Understanding how technology is being used daily within an organization and how that will affect ESI will allow you to identify sources, location of sources, potential key communicators, and perhaps additional custodians.
In the short term, thought leaders in the local Michigan legal community are being very supportive to help litigators across the state develop best practices around the new rules.
I cannot express this enough. The local support and legal community in Michigan is very strong. There are advocates for education and thought leadership that can be great assets. The Bench and Bar event was one recent event that helped educate on how to be prepared for the new rules that have taken place. Judge Fresard did a great job coming to ACEDS Detroit and really connecting with local Vendors to make sure we understood the importance of supporting the bench and bar event and these new changes. There are many local vendors that have a strong understanding of the technical aspect and legal implications to time and budget, which can provide great value to legal teams.If you’re a lawyer, I’ve firsthand seen the passion and push for more education on these changes from attorneys like Scott Petz (Dickinson Wright), Megan McKnight (Tealstone Law) and Jay Yelton (Warner Norcross) just to list a few. All of them can be great external resources and have years of experience dealing with ESI. They are heavily involved in ACEDS Detroit and do a great job introducing new ideas for ESI training & consulting.
Other states have dealt with similar rule changes – so there is some precedent for how Michigan will likely handle the changes.
A lot of states are dealing with rule changes around duty of technology competency and with eDiscovery. California was early in pushing these changes in 2015. Although, interestingly enough, they pushed the changes through an ethics opinion rather than a formal rule change. 38 states have adopted duty of technology competency requirements, going as far back as 2013. So, although I’m not a lawyer, I imagine there have to be some examples there for Michigan attorneys.Concluding Thoughts.
Attorneys that have embraced technology and looked at these challenges as opportunities, have had great success. Client expectations for cost effective methods to case strategy and the use of technology continue to rise. If you’re an attorney that decides to look at these new changes and takes the time to understand how it can be utilized as weapon instead of it being looked at as a hurdle, you’ll see even greater success with your clients.This is extremely exciting for me personally, because I spend a lot of time trying to advise my clients on the most efficient and cost-effective ways to handle eDiscovery. Remember, the role of your eDiscovery provider is to make this easy and understandable and not super technical and burdensome. Change is never easy, but you’re not in this alone. Do not hesitate to reach out to any of the people I mentioned, or feel free to contact me directly.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!