Like R&D, the first step to improving eDiscovery process and adopting new tools is prototyping (aka conducting Proof of Concepts, POCs). This is where a lot of great ideas fail to gain traction because there’s not enough organizational buy-in or clarity around the benefits of the new approach. To ensure ideas maintain momentum at this stage, below are the four things I require my team to outline in every proof-of-concept — whether with internal projects or client-facing projects – before I commit to investing resources in those projects.
1. Clearly and Specifically Define the Proposed Concept Being Proven.
A fully defined concept includes the following six questions: (i) Who is going to use this new technology / workflow? (ii) What benefit is that person seeking from this? (iii) Where in the investigation / litigation process will this be used? (iv) When will the proof of concept start and conclude? (v) Why is this proof of concept worth spending time on (vi) How many resources will be required for this?2. Concretely Understand the Next Best Alternative to Proposed Solution
It’s very easy to skip this step because the answer seems obvious. The next best alternative to the proof of concept is whatever we’re doing today! However, being disciplined about explicitly outlining what the next best alternative is ensures that all the stakeholders and team members are starting from the same baseline. You would be surprised how often no single person fully understands what an organization’s current solution is. Given that, it’s no wonder that it’s hard to get organizations to change.3. Define Key Performance Indicators (KPIs)
An old engineering adage is “In God we trust, all others must provide data.” While there is always a qualitative aspect to a proof-of-concept, quantitative measures are a must for evaluating a proof of concept. They provide a level of objectivity to the discussion and serve to align all stakeholders to the same goals4. Outline Goal Required to Justify Change
For a proof of concept to successfully actualize into adoption of new tools or processes, the improvements need to be substantial enough to justify the cost and risk of switching. Typically, I advise clients that they achieve 30% to 50% improvements in their KPIs. Otherwise, the improvement isn’t beneficial enough to outweigh the pain and risk of change. Outlining this number concretely at the outset of the project allows for better collaboration. The internal and external team can iterate on the technologies and processes over the course of the proof of concept, always knowing where their North Star is.Concluding Thoughts:
The above steps are simple, but they ain’t easy! Like most technology challenges, it’s about the people and the process, not just the technology. Let me illustrate with two examples from my new life in eDiscoveryExample 1 – Typical POC Approach: I want to do a proof of concept with StoryEngine. I’m evaluating it against using Relativity to see if it finds the relevant data more efficiently. If it’s better, we’ll switch.
Example 2 – Disciplined POC Approach: An attorney at my firm is willing to spend 4 hours learning how to use StoryEngine to investigate the document repository when trying to establish fact patterns in support of the initial pleading. Their alternative would be to run searches in Relativity on priority custodians with keywords and have a paralegal manually create a key chronology from the responsive documents. If the StoryEngine solution can reduce calendar-days-to-chronology-completion by 50% (5 business days, instead of 10 business days) and reduces the total-cost-of-chronology-creation to the end-client by 30%, I would be interested in switching our standard process.
The Typical Approach is more comfortable because there’s a lot of unknowns at the outset of a proof of concept. It’s hard to commit to hard numbers and specifics when there’s not a lot of information. And, committing in uncertain environments means the POC is more likely to fail than succeed. R&D Organizations I worked with had the same problem. But, by somebody (anybody, really!) drawing a line in the sand, organizations were able to be more efficient at continuous innovation and more effective in the long-run. Ironically, failing at POCs is one of the best indicators for long-term innovation success. You can always change the proposal, alternative, KPIs or goal as you go through the process and learn more.
Failure contributes to learning and growth which lead to new ideas for new proof of concepts that ultimately succeed. Being disciplined about consistently applying a structured process to a fluid environment actually results in the best outcomes.
____________
Be Sure to Follow Me for the Latest Content and Subscribe For the Latest Acorn Insights!
