Post Author: Lia Majid

Lia founded Acorn Legal Solutions through the acquisition of the eDiscovery operations of Elijah Ltd in 2017. An industry outsider, Lia brought rigorous project management techniques to Acorn, similar to those used in the aerospace industry, which significantly improve client outcomes. Prior to Acorn, Lia was a Sr. Project Engineer at Parker Hannifin. She has a MBA from Kellogg School of Management at Northwestern University and Electrical Engineering and Mathematics degrees from Case Western Reserve University.

Sizing Up Software at LegalTech – Thoughts and Considerations

As we face the end of mask mandates and a return to “business as the new normal”, the legal tech industry is kicking off its return at Legal Week in New York. This presents great networking opportunities, and a chance to catch up on what the technology companies have been doing under the radar these last few years while we’ve all been locked away at home. 

There have been big changes in the technology space in regard to the “Product.” This year’s Legal Week presents one of the few opportunities where I can see 2 years of progress in a demo versus 2 months since the last trade show. 

The few products I am most excited to see: 

  • Relativity has ramped up their cloud-based solution RelativityOne and gained a lot of momentum around their TRACE product. Their deployment of automated, scalable redactions with their Redact product is probably the most visible advancement during the COVID time period. Second to that is a heavy focus on integration with other cloud-based sources of data with their Verqu acquisition.  
  • Reveal / Brainspace / StoryEngine have all merged. StoryEngine and Brainspace are by far the best AI powered ECA products on the market. This functionality provides tremendous value to customers. Acorn has always been a fan of StoryEngine, and on average, for the cases we have that use StoryEngine there is a reduction in Total Cost of Review of 83%. Integration with Relativity has always been an important part of this process for us and I am keen to see if StoryEngine continues to support that tech stack with their stand-alone product, or if they choose to focus on integrating all the AI into their Reveal review.  
  • Disco’s review platform has always had a leg-up in terms of simplicity and ease-of-use. For individual attorneys running the entire eDiscovery process on their own, it makes a lot of sense. I’m not sold, yet, that it’s as useful for workspace administrators looking to support ultimate flexibility and customization in their work. But Disco is pursuing a channel strategy in earnest now, and I am very interested in seeing how that strategy plays out in the market. 
  • iConect is a product that I’ve always loved. It’s a halfway point between Disco and Relativity. It offers a lot of Relativity’s flexibility, and a lot of Disco’s ease of use and simplicity. They took on new investors ~3 years ago, and I bet they have made a lot of progress in terms of product advancement. I’ll be curious to see and hear about it. This is a bit of a dark horse for me going into Legal Week. 
  • Merlin Search Technologies (a.k.a. The Return Of The Johns – Pappas and Tredennick) is the biggest wildcard for me. This team pioneered Continuous Active Learning / TAR 2.0 in the industry and the original team is largely reunited with a new goal in mind. Based on a quick review of their website, I don’t fully understand their differentiation from other tools. However, with this group of folks, I trust it’s there and am excited to hear about it. 

While I’m looking forward to seeing all the new bells, whistles, and features during my in-person software demos, I’ve learned to start focusing on the “Whole Product” instead of just the “Product.” 

“Whole Product” was a concept developed by a Harvard Business School professor, Ted Levitt, and best understood via example. Let’s use a car as an example. If you buy a car that has the features that you want, but it doesn’t have some of the other critical components of your ownership experience, like a nearby service center, an adequate warranty, availability of parts, etc. then you won’t be happy. The Whole Product is the entire experience, and not just the features of the Product. (See here for an article that explains it a bit more in depth) 

Similarly in eDiscovery and Legal Tech, the Whole Product matters. While the demos will show me the Product, the networking will get me more insight into the Whole Product. Here are the questions I plan on asking about each of the aforementioned technologies: 

Considerations for Whole Product in eDiscovery / Legal Tech Space: 

  • How easy is it to find in-house or out-sourced talent that can support this product / software? 

As a services provider thinking about my “Whole Service” it is hard enough to find professionals with the breadth of skills we look for: education orientation, passion about using technology to drive value, communication skills, client orientation, ability to manage complex projects, etc. I don’t want to limit my pool of candidates further by only looking for people with niche technology skills. I also don’t want to slow down my staff onboarding by requiring my new team members to undergo more training in a new technology. For me, this is a big one where Relativity comes out ahead – and I think is a reason they have been a dominant player in the market. 

  • Support with upgrades, or tech issues 

Although software is a platform, the world of tech gets complicated fast. Does that software work on various browsers, with various versions of that browser, on various types of devices, with various operating systems? Ensuring the Product functions in all those circumstances becomes a geometrically complicated problem. I want my team focused on creating value for clients, not distracted with tech troubleshooting. So, good processes, investments, and expertise from my vendors in this area is key to my Whole Product experience. Alternatively, a software provider who can be very specific and focused on the tech stacks they are designed support also garners my trust. 

  • Access to educational materials on how to get the most out of the Product 

eDiscovery is a devil-is-in-the-details industry. Having good documentation around how the features are configured and the logic around those configurations is key to our services. If we can’t explain to a client why a filter in one software platform generates different results for the same document set than a seemingly duplicate filter in a second software platform, we have a lot of trouble using the software. This is crucial documentation for defensibility considerations. All software providers say they have documentation, support, and access to their experts. However, most run out of bandwidth before we run out of questions.  

  • Security operations, and transparency responsiveness around evolving data security / data breach environment 

Data security is at the forefront of both our industry and the world. There have been several key breaches and vulnerabilities discovered.  The old industry standby of “our data is secure, trust us” doesn’t hold as much water as it used to. I’m not a data security expert, so this is a hard one for me to evaluate. I do generally look for robust processes, contractual commitments to transparency, and dedicated security professionals with both in-industry and outside-the-industry experience to get comfortable when considering a new product in our portfolio. I also rely on a roster of blue-chip clients in financial services to get me comfortable that the data security operations are up to snuff. 

  • Integration with the rest of your tech stack. Integration with the operations / tech stack / talent of the organizations that you need to coordinate / collaborate with 

So, this is another Harvard Business School idea, pioneered by Thales Teixeira. About Decoupling. In short, a lot of technology companies try to force you to use all their modules through coercive technology tactics. Instead, I look for products and companies that support the concept of decoupling. They allow me to choose the tech stack that works for me, my organization, my clients, and my partners – even if it means picking and choosing modules between software companies that are traditionally viewed as competitors. Ultimately, that’s good for everyone because it allows the most value creation in the marketplace. 

Conclusion:

There’s a lot of expertise, hard work, and good intentions in the eDiscovery industry.  Yet there’s still a lot of frustration at clients, at service providers, and at technology firms. The work we do is cross-functional, intellectually challenging, and performed in a high-uptime, contentious, fluid environment. That, in and of itself, makes it hard for everyone to walk out of a project with a good experience. However, understanding and focusing on the Whole Product of eDiscovery technology at the outset of an engagement gives everyone more of a fighting chance at success. Without it, the headwinds are even stronger. 

I refuse to accept the old industry refrain of “all eDiscovery providers suck, but at least we know how ours sucks and ours sucks less than any of the other ones out there.” There is a path to plan and resource our way to success on projects in this industry. And we can learn from other industries. 

I look forward to seeing you all there, and jumping back into the pool of great people, great solutions and great parties at Legal Tech! 

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

Four Ideas I Bring to eDiscovery from R&D

As an eDiscovery outsider, I founded Acorn Legal Solutions to bring advanced technology and rigorous process that doesn’t compromise agility to the industry. The longer I’m in the industry, the more I realize that my career in R&D strategy consulting was a great foundation for this work. Like R&D, eDiscovery is a constantly evolving technology landscape, which requires cross-functional teams to act intentionally in realizing innovative concepts, and whose success depends on consistently applied agile process.

Like R&D, the first step to improving eDiscovery process and adopting new tools is prototyping (aka conducting Proof of Concepts, POCs). This is where a lot of great ideas fail to gain traction because there’s not enough organizational buy-in or clarity around the benefits of the new approach. To ensure ideas maintain momentum at this stage, below are the four things I require my team to outline in every proof-of-concept — whether with internal projects or client-facing projects – before I commit to investing resources in those projects.

1. Clearly and Specifically Define the Proposed Concept Being Proven.

A fully defined concept includes the following six questions: (i) Who is going to use this new technology / workflow? (ii) What benefit is that person seeking from this? (iii) Where in the investigation / litigation process will this be used? (iv) When will the proof of concept start and conclude? (v) Why is this proof of concept worth spending time on (vi) How many resources will be required for this?

2. Concretely Understand the Next Best Alternative to Proposed Solution

It’s very easy to skip this step because the answer seems obvious. The next best alternative to the proof of concept is whatever we’re doing today! However, being disciplined about explicitly outlining what the next best alternative is ensures that all the stakeholders and team members are starting from the same baseline. You would be surprised how often no single person fully understands what an organization’s current solution is. Given that, it’s no wonder that it’s hard to get organizations to change.

3. Define Key Performance Indicators (KPIs)

An old engineering adage is “In God we trust, all others must provide data.” While there is always a qualitative aspect to a proof-of-concept, quantitative measures are a must for evaluating a proof of concept. They provide a level of objectivity to the discussion and serve to align all stakeholders to the same goals

4. Outline Goal Required to Justify Change

For a proof of concept to successfully actualize into adoption of new tools or processes, the improvements need to be substantial enough to justify the cost and risk of switching. Typically, I advise clients that they achieve 30% to 50% improvements in their KPIs. Otherwise, the improvement isn’t beneficial enough to outweigh the pain and risk of change. Outlining this number concretely at the outset of the project allows for better collaboration. The internal and external team can iterate on the technologies and processes over the course of the proof of concept, always knowing where their North Star is.

Concluding Thoughts:

The above steps are simple, but they ain’t easy! Like most technology challenges, it’s about the people and the process, not just the technology. Let me illustrate with two examples from my new life in eDiscovery

Example 1 – Typical POC Approach: I want to do a proof of concept with StoryEngine. I’m evaluating it against using Relativity to see if it finds the relevant data more efficiently. If it’s better, we’ll switch.

Example 2 – Disciplined POC Approach: An attorney at my firm is willing to spend 4 hours learning how to use StoryEngine to investigate the document repository when trying to establish fact patterns in support of the initial pleading. Their alternative would be to run searches in Relativity on priority custodians with keywords and have a paralegal manually create a key chronology from the responsive documents. If the StoryEngine solution can reduce calendar-days-to-chronology-completion by 50% (5 business days, instead of 10 business days) and reduces the total-cost-of-chronology-creation to the end-client by 30%, I would be interested in switching our standard process.

The Typical Approach is more comfortable because there’s a lot of unknowns at the outset of a proof of concept. It’s hard to commit to hard numbers and specifics when there’s not a lot of information. And, committing in uncertain environments means the POC is more likely to fail than succeed. R&D Organizations I worked with had the same problem. But, by somebody (anybody, really!) drawing a line in the sand, organizations were able to be more efficient at continuous innovation and more effective in the long-run. Ironically, failing at POCs is one of the best indicators for long-term innovation success. You can always change the proposal, alternative, KPIs or goal as you go through the process and learn more.

Failure contributes to learning and growth which lead to new ideas for new proof of concepts that ultimately succeed. Being disciplined about consistently applying a structured process to a fluid environment actually results in the best outcomes.

____________

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

The Semantics of Machine Learning & Artificial Intelligence in eDiscovery

Precision Of Language Is Important – Or Is It?

I recently was asked to speak at the SoCal Women in eDiscovery Technology Symposium on the Future of AI + eDiscovery with Cat Casey from Disco (and previously Gibson Dunn) and Christine Milliron at Robins Geller. The panel talked about numerous predictive and automation technologies being adopted in legal services. But we spent the vast majority of our time discussing variations on Technology Assisted Review and Machine Learning in eDiscovery review applications.

Spirited debate on the differences between TAR, Machine Learning, and AI is a favored pastime of slightly tipsy conference veterans [1]. After the panel, an attendee posed a thought-provoking question to me, that I’ve continued to revisit over the past 3 weeks:

“Does drawing a distinction between Machine Learning and Artificial Intelligence really, ultimately, matter?”
I had very mixed feelings on the topic.

  • On the one hand, I work in an industry fundamentally based on the notion that precision of language is important. Linguistic shortcuts conflate concepts and prevent clarity of thought. Using the terms interchangeably today creates potential for miscommunications tomorrow.


  • Moreover, as a company laser-focused on value delivery, precision in our language is important towards our everyday commitment to setting clear, actionable expectations as to the services that we provide. At Acorn, we mean what we say — we have to; our clients are skilled (professionally trained!) at looking past pretense and fuzzy promises.


  • On the other hand, if sexy terminology drives market exposure to emerging technologies, then who cares about semantics? The pragmatist in me only cares about whether we, as an industry, are collectively raising the bar on efficiency and quality.
After three weeks of ruminating on the topic, I have somewhat ironically settled at a lawyer’s answer: It depends. And what it ultimately depends on is the audience. When I am talking to those who hold themselves out to be sophisticated actors in the space, I will use precision of language on topics like this as a measure of their actual expertise. Experts should have a point-of-view on this. However, when I am encouraging people to explore new technology solutions to old problems, I’ll stay more focused on their excitement than their diction.

Ultimately, eDiscovery is a very tactical field. So, in my day-to-day life, differences in AI semantics matter much less than being specific about technologies, workflows, budgets, timelines, and risks.

~~~~~~~~~~~~~~~~~~~~~~~

[1] I don’t even think there’s consensus on the definitions of either, other than that they’re vaguely related but still different. Try googling “difference between machine learning and AI” and see if you can make heads-or-tails of the results.

____________

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com. 

The Five Types Of People In eDiscovery

Images I recently attended the inaugural ACEDS Ohio event. It was an educational program filled with engaging, interactive content. I found that the most interesting content highlighted the variety of approaches towards handling eDiscovery problems. In particular, the workshop at the end was really illustrative.

Audience members were presented a scenario. There is on-going business-to-business litigation. An employee has responsive ESI on their private phone: text messages they sent to other employees and marketing videos. But, the employee doesn’t want to give you access to the phone. How would you handle that situation?

The answers spanned the gamut. But, in general, it seems like there are five types of professionals in eDiscovery.

1. The Pragmatic Attorney

“Dealing with this issue seems like an unnecessary and counterproductive distraction. Perhaps I should call up the opposing side and see if we can both agree to keep personal devices out of the scope of Discovery. Sometimes what’s good for the goose is good for the gander.”



2. The Lawful Attorney

“That employee has an obligation to preserve and produce all information that may be relevant to the litigation. I need to inform them of that obligation. And if they won’t produce willingly, I’ll file a third party subpoena and compel them to produce.”



3. The Accommodating Attorney

“We should ask the employee, ‘Why not?’ Then we can come up with creative solutions that get the job done. The employee might not want to share personal information with their employer. If so, they could engage their own counsel to review and produce what’s only relevant to our litigation.”



4. The Pragmatic Technologist

“Let’s scale the problem. How much ESI is uniquely on the phone? Almost all the data can probably be recovered from their iCloud account on their company laptop, from their company email or from the company devices of other employees. Worst case scenario, they email us responsive screenshots and videos.”



5. The Completionist Technologist

“We absolutely must have the device and collect it in a forensically sound manner to 100% avoid any risk of appearing non-responsive to our preservation obligation and to avoid spoliation claims with certainty, regardless of the cost and disruption.”



Not Present: The Blatant Defier

“Run it over with a car. One of the great challenges of eDiscovery is that when one party fails to preserve evidence, it’s on the other party (the moving party) to establish something remiss has happened.”

Concluding Thoughts

It’s always fun to get multiple points-of-view in a room to tackle challenges together. I thought that this event underscored some of the key challenges of eDiscovery. Namely, that it’s cross-functional, and not everyone is aware of all the options that are available to them. The technical folks don’t always see the legal options and the legal folks don’t always see the technical options.

____________

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights! 

About Acorn 

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.  

The 2 Key Ideas from ILTACON’s “Predictive Coding for Dummies (Everybody)” Panel

I have been passionate about predictive coding since I first got into the eDiscovery industry. It’s been an area of high focus for Acorn since I acquired the company about 2.5 years ago. Since then, we have performed Technology Assisted Review (TAR) projects on 4 different platforms for everything from a Second Request to a 10,000 document review. I have spoken at numerous CLEs and industry events in front of GCs, in house attorneys and support staff on the topic, in an effort to facilitate adoption in the industry.

Recently, I was invited to speak at ILTACON2019 on the topic with four other industry insiders. I was surprised at the number of folks in the audience who had first-hand experience with TAR. About 2/3 of the audience had used predictive coding on at least one case, although a smaller number – only around 15% — had used it on multiple cases.

In an industry rooted in pragmatism and practice, the panel talked extensively about the industry shift from TAR 1.0 and TAR 2.0 ( as covered in law.com article ). The panelists were super knowledgeable, and we were lucky to have a very engaged audience. A number of thought-provoking questions were asked, which reflects an increasingly sophisticated audience.

Here are the two big ideas that caught my attention:

CostShifting/Sharing: Facilitator of TAR Adoption and Discovery Cooperation

The 2015 Federal Rules amendments added subsection (c)(1)(B) to Rule 26 to expressly address cost-shifting in discovery. Courts can impose costs on the receiving party for “non-core discovery” activities or for a failure to make a “threshold showing of merit.” ( as covered in a Legal Backgrounder article hosted on Pepper Hamilton’s website span> )

The central challenges in industry adoption of TAR in a “not every document gets an eyeball” application are questions of disclosure requirements, cooperation with the opposing side and judicial discomfort in adjudicating disputes over questions of TAR. Discovery cost-shifting means both opposing parties have skin in the game — to the loser goes the Discovery bills. If eDiscovery cost-shifting becomes more prevalent in the industry, it will create more of an economic incentive for both sides to cooperate in leveraging technology for review efficiency. It seems like a win-win-win.

Cooperation between Plaintiff and Defendant is an area Chad Roberts of eDiscovery CoCounsel has spoken about on a national stage at RelativityFest – and I want his take on this matter, especially as relates to asymmetrical litigation.

The Next Big Frontier: TAR AI Models and Document-Set Mega-Trends

For a few minutes, the panelists were allowed to put on our Futurist Hats and asked to speculate about where the technology may evolve. Two actionable ideas were surfaced.

First, creating AI models from previous matters to leverage that expertise on future matters. This is a natural extension of the idea behind predictive coding. If you can use a baseline of review work-product to predict the coding outcome of the remaining documents in a case, then you should be able to use a baseline of work-product from a previous, similar case with similar issues or similar clients, and use that to predict the coding outcome of documents in the new case. Privilege review seems a very natural starting point for this advancement in eDiscovery.

Second, the trends across the entire document set will become evidence as much as the documents themselves. For example, if a salesperson is conspiring with a competitor to transition their client base and circumvent their non-solicit, they might be smart enough to not leave a paper trail. But, you can probably see other trends that point you to what’s happening. Maybe their communications with a key client from their company email drop relative to historical values. Maybe their after-hours communications to non-company email addresses are higher relative to historical values. Maybe they take longer to reply to their supervisor’s calls than normal. Looking at any documents individually won’t tell you that story – but seeing the whole trend will.

In my view, Jay Leib and NexLP are at the forefront of innovation in the industry on both of these topics. They have been developing technologies like these TAR 5.0 Imaginings for a couple years. With their product, we already have the capability to use AI models across cases — it is particularly valuable for employment matters. And we are already aggregating data trends to find the hidden stories – a particularly compelling example is when StoryEngine was used for document-universe trends in a price-fixing matter.

Conclusion

I really enjoyed the opportunity to gather and discuss predictive coding in the industry. We are seeing adoption of TAR increase and people hone their skills. I am excited to be a part of a dynamic and engaged community. And I look forward to having the opportunity to discuss how predictive coding can be used by all skill levels in all different types of matters. Erin Tomine at Conduent summarized the panel very nicely, “The human aspect is still so important” in determining the appropriate TAR approach, in clarifying the objectives of TAR, and in managing communications across the team. “Even as we embrace technology and AI, it can’t do everything for us.”

____________

Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!

About Acorn

Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.