What is Sentiment Analysis and Why Does It Matter?
Relativity’s newest tool, sentiment analysis, allows users to identify positive, negative, and other targeted emotions or “sentiments” across a data set. This AI-powered technology computes a score on a scale from 1 to 5 of the likelihood a document contains one or more of the targeted sentiments. In addition to the score, Relativity also offers visualizations and integration within the document viewer which allows users to see highlighted sentences to quickly understand the different sentiments identified within the context of the overall document.
Detecting unusual or emotionally charged communications can be highly valuable in a variety of litigation and investigation contexts. Armed with this insight, users can look at the data from a new perspective to help identify potentially impactful conversations and/or participants more quickly. This type of analysis allows users to identify and prioritize these potential hot spots early on leading to more cost effective and strategic decisions around building a case strategy.
Sentiment analysis provides users an easy-to-use tool that offers new insights and perspectives into document collections at no additional cost. Any tool that can add value to our clients while potentially reducing the cost of discovery and accelerating case strategy is an exciting development.
Our initial test results are promising. The technology, when layered with some of Relativity’s other tools and features, has already led to some interesting conversations and creative new workflows. We believe it is a great first step for Relativity and we are excited about the development roadmap in the near future. Ultimately, if users are mindful that sentiment analysis provides predictions, not guarantees, and the final evaluation should always be up to the user, it can be a very beneficial tool.
As with any new technology, it will take time for us to work with this tool and understand where it can be most useful. It will also take time for the technology company to continue to improve this feature as well. As a RelativityOne certified partner, the team at Acorn is confident that we will work alongside Relativity and our clients to make this tool as easy to use as possible. I want to also make sure that I share my view of the early applications and workflows where I think sentiment analysis would be most valuable.
First Use Cases:
Overall, this is a good tool to get the lay of the land of a small document set. It can be used to hone in on 50-100 key documents to review first as clients form their case strategy and understand the substance of the evidence repository. The reason this tool is best for small datasets is because there is currently a 50k document limit, so we want to balance the time investment in setting up the technology against the benefits of having the information from the sentiment index.
- Investigations – Sentiment analysis can be best used as an investigation tool for projects that are around 50k to 200k documents. It is beneficial to use as an early look at the documents of interest in the dataset and see where we can focus on high impact documents.
- Opposing Productions – This tool may also be useful for projects in the 50k to 200k document range to get a first impression on incoming productions on a rolling basis.
- Hail Mary Review Scenarios – Normally, I would not use this tool for a review unless there was a specific issue or timing constraint where you need to prioritize often or make a decision quickly. However, in these “Hail Mary” scenarios, I would use sentiment analysis to run searches for highly emotional documents to find any insights that can be gained, whether positive or negative. You could also layer this on top of a key word search to prioritize early review and gain insight when you’re just starting to get into the documents. Most of the time, though, I would recommend using a keyword search or batched review.
My First Impressions – Potential for Future Improvements to Sentiment Analysis
While the new sentiment analysis offering is a beneficial first step for Relativity and the early results have been very promising, I will be interested to see if Relativity broadens the applicability of the tool by removing some current limitations.
- English Only – Relativity’s sentiment analysis tool was designed only for use within English-language documents. Even if translated, foreign language documents may not produce accurate results, making sentiments from other languages unclear for users. I will be curious to see if they move to other Latin languages, or even CJK languages.
- Lack of Variety in Sentiments – Similar tools from other competitors offer a wider range of sentiments that can be detected and used within a data set such as Intent, Opportunity, Pressure, and Rationalization.
- Model Fidelity with Modern Communications (e.g., Emojis) – Language is constantly changing, especially with short messaging apps becoming more and more popular. Users are continually creating new abbreviations/acronyms, utilizing emojis, and/or using poor grammar and spelling. This level of variation and evolution can be difficult for algorithms and lead to unreliable results.
- Small Size of Data Sets –Relativity recommends running sentiment analysis on smaller, more focused data sets under 50,000 documents. Moreover, the tool is also currently unavailable in ECA workspaces. Using the tool to identify key communications and participants as early as possible across larger sets could be very beneficial especially in an early case assessment workflow.
What I Would Like to See Implemented in the Tech
In addition, as we work with the tool more and more, I can already start to see a few avenues to improve users’ experience with sentiment analysis by:
- Refining the Model – Because of some of concerns mentioned above, it would beneficial if users could interact with the model to optimize against a specific data set. Even the most advanced AI-driven sentiment analysis tools will require human intervention in order to maintain consistency and accuracy in analysis. Providing statistics around validating the model is key for defensibility and has been a large part of my work in TAR so far.
- Adding Integrated Dashboards – Current integrations in Relativity’s document viewer are very valuable for users to quickly assess sentiment predictions on a document. What is missing is better integrations and visualizations via pre-configured Relativity dashboards. The ability to better interact with data visualizations to uncover important concepts, relationships and patterns will provide users greater insight into the overall data set earlier and easier.
- Optimizing the Input – Based on some of our initial testing, additional clean-up to remove noise and parts of the email text such as signature blocks, footers, and/or attachment names could lead to better overall results.
- Linguistic & Cultural Context – Sentiment is expressed differently across languages, regions, cultures, and generations. When reviewing the sentiment scores, consider how each might affect the overall document population and thus sentiment predictions.
If you would like to see a demo of sentiment analysis or Relativity’s Review Center, where it’s housed, please feel free to reach out to me at Josh.Treat@AcornLS.com. I am also happy to discuss how this technology may work for your case, or exchange thoughts on what other technology in the market may fit.
About Josh:
As a seasoned litigation support specialist with 15+ years of experience, Josh has saved millions of dollars in review costs through the development of proprietary assisted review workflows in a variety of platforms. Josh has been involved in numerous landmark cases that leverage Advanced Analytics. In addition, Josh leverages quantitative forecasting and rigorous, best-in-class project management tools for on-time, under-budget completion of complex litigation review. He speaks fluent geek, salesperson, trainer, and difficult client. As a result, Josh has a unique ability to help manage and navigate large complex projects with creative custom solutions to meet any budget or timeline requirements. Josh is a Certified Relativity Master and holds a Bachelor‘s Degree in Management Information Systems from Central Michigan University.
About Acorn:
Acorn is a legal data consulting firm that specializes in AI and Advanced Analytics for litigation applications, while providing rigorous customer service to the eDiscovery industry. Acorn primarily works with large regional, midsize national and boutique litigation firms. Acorn provides a high-touch, customized litigation support services with a heavy emphasis on seamless communications. For more information, please visit www.acornls.com.
