Case studies & key findings of Machine Learning from LawtechUK’s AI Discussion Paper

It has long been thought that artificial intelligence (AI) has the potential to transform legal services. With recent developments in generative AI, the pace at which that potential might be realised appears to have accelerated. As we seek to harness AI’s transformational potential, an awareness of the technology's limitations is as important as its potential to drive positive outcomes for consumers of legal services.

As part of our efforts to promote responsible use of AI in the legal industry, LawtechUK conducted a consultation with legal professionals, organisations and industry experts, and gathered case studies of Machine Learning (ML) in use today. The aim of this exercise was to understand ML’s current applications in legal services and how legal service regulators could support responsible use for the benefit of consumers. Our published case studies represent the diverse considerations taken into account when adopting ML in legal services and offer valuable insights into what the process may involve in practice.

Our exercise focused on the adoption of Machine Learning (ML) in legal services, not the courts or justice system. It does not constitute a policy statement or regulatory advice, and it should be construed solely as a set of observations, aiming to inform a better understanding of ML and help it to be applied for the benefit of consumers of legal services. We want to emphasise the importance of recognising that not all AI is the same, and having an understanding of this technology and an appreciation of its different applications is crucial for us to further develop it for good.

Keeping an expert in the loop

The case studies identified five broad types of ‘function’ ML powered systems were performing when being used in legal services, which included:

• Administrative: automated triage systems to support client and matter intake (see LegalBeagles) or collecting and organising documents

• Profiling: profiling consumers to understand cognition of legal text (see Amplified Global) or identifying vulnerability (see Transparently), in order to help tailor advice to different clients and achieve the best outcomes for different circumstances

• Search: identifying relevant cases (e.g. Lexis Nexis)

• Legal risk identification or prediction: in transaction or property due diligence (see Orbital Witness or Avail), e-discovery (see Luminance) or insurance fraud (see Kennedys IQ)

• Legal text generation: in contract drafting (see Shoosmiths Cia®)

ML powered legal solutions used a human in the loop (often a lawyer) to check tasks and decisions involving substantive legal outputs (e.g. legal risk identification). ML was not being used as a substitute for human expertise (lawyers); rather, it is used to augment and enhance legal services that are still delivered by lawyers and other experts. It is important to recognise that legal professional input is often essential for training the algorithm and conducting risk assurance activities.

Consumers of legal services stand to benefit from responsible use of ML

The case studies demonstrated that, where ML is used responsibly, it results in improvements in the standard and transparency of the services delivered. Legal service providers utilising AI to enhance their services were able to communicate to a client the reasoning behind an algorithmic output more easily than the logic or rationale behind a human decision. This more unexpected finding may help provide additional clarity to consumers who can more easily compare the quality and cost-effectiveness of different legal services and remedies.

Whilst it is important to be mindful of the potential risks and harms of using ML, this should not over-shadow or hamper our ability to realise its benefits for consumers. We recommend further research to understand the potential benefits and how best to achieve these to improve legal outcomes for consumers, for example into whether improving transparency leads to better quality service.

Additionally, as one of ML’s advantages is scalability, ML has the potential to reduce the costs and improve availability of legal services, addressing the unmet legal needs of those who cannot afford to consult a lawyer. For example, solutions utilising Large Language Models (LLMs) like GPT-4 may facilitate this directly by providing a self-service facility for individuals seeking basic legal information, or indirectly by offering law firms cost savings which can then be passed on to clients. There are some examples of start-ups (such as LegalBeagles and Transparently) developing ML powered legal services for consumers but many we identified are still in development / proof of concept. We believe that more support should be provided to develop, test and scale ML applications that deliver consumer facing legal services.

ML developers of legal services understand the risks involved and how to mitigate them

The development, training and deployment of ML powered tools requires specialist knowledge, resources, and collaboration with AI engineers and software developers. Legal and technology teams tasked with implementing and overseeing ML systems and applications were well trained, and consistently demonstrated a clear understanding of the risk assurance and management activities required to deploy AI responsibly. The case studies set out different approaches to identifying and managing risk.

When using ML on client data/matters, legal service providers were seeking permission from clients and taking care to communicate potential risks, allowing clients to make informed decisions. This is about demonstrating a higher degree of transparency with clients and providing data around the accuracy of the output – something that isn’t possible in respect of advice provided by a lawyer, perhaps because a lawyer commands a degree of trust as a qualified professional.

ML has the potential to support a well-functioning, competitive legal services market

There is potential for ML to amplify existing disparities within the legal services market, but these disparities appeared to support a well-functioning, competitive market that could drive better outcomes for consumers.

1. Disparities between small and large regulated firms

Smaller regulated firms have less capital and resources to develop and adopt technology than larger firms, meaning the AI transformation may be uneven. Additionally, the sector may face limitations in adoption and development of ML because, depending on the type of work they handle, law firms may lack access to high volumes of structured high-quality data to train the models. These limitations are likely to impact smaller providers to a larger degree, potentially causing further polarisation in the sector.

It is worth noting, however, that regulated firms are most likely to introduce ML into their firm by purchasing third-party tools. As these third-party tools improve and reach product-market-fit, less specialist expertise and data to train these tools is needed to successfully implement them. Whilst, the cost of some of these tools may be prohibitive for some small-to-medium firms (SME), this is changing to. As these third-party-tools continue to develop they are finding a cost point that is attractive for smaller firms, lessening the impact of these existing disparities between large and smaller firms.

One disparity that requires further investigation is in relation to professional indemnity insurance (PII). A number of the large firms indicated that their activities involving ML and predictive analytics were not substantially affected by requirements around PII. However, insurers and brokers are still getting up to speed with the considerations and implications of new technologies and their emerging use cases for legal businesses. Some firms have found that their PII premiums reduced following the introduction of ML to automate client onboarding workflows: when the client inputs their own data, there is no chance (that a client would make a claim in respect) of the lawyer misinterpreting the details of the matter/getting the initial details wrong. Client onboarding workflows that include compliance such as identity checks, anti-money laundering checks etc mitigate risks around fraud, reducing the chances of an insurance claim.

On the other hand, some respondents reported that insurance companies are beginning to explicitly exclude AI from SME firms' PII policies. On our recommendation, the Regulatory Response Unit plans to host a roundtable with insurers to ensure that insurers understand the risks and benefits associated with AI adoption in legal services. Insurers expressed that from a practical perspective, where operational functions or services are outsourced, it’s important for firms to check the terms and cover of the professional indemnity insurance of the outsourced provider.

2. Disparities between regulated vs unregulated providers of legal services

Whether operating in a regulated firm or within unregulated legal services, those developing ML tools in the UK hold themselves to high professional standards. This may be because they are operating within a regulated environment, even if they are not regulated directly by legal service regulators.

Some concern was expressed that new, unregulated legal services providers that are developing ML-powered products and services have an unfair advantage by not being regulated and, as corporate entities, having limited liability.

As there are currently only a few reserved legal activities which are regulated, there are plenty of significant opportunities for ML start-ups to make a difference in the UK, particularly in the access to justice (A2J) space, driven by the digitalisation of the courts service and online access to case law via the National Archives.

However, unregulated ML start-ups face other challenges. Some of these relate to funding and other resourcing, while others are specific to legal and professional services, such as, limited access to training data. Unregulated providers utilising ML also face challenges obtaining suitable PII coverage and don’t have the same access to regulators or professional guidance and support, and they may encounter hostility from incumbent firms and vendors who do not welcome new competition.

For this reason, these disparities may contribute to a well-functioning, competitive legal services market, which will produce better legal outcomes for consumers. Legal service regulators should continue to monitor the impact of these technologies on outcomes for consumers of legal services.

We need to improve the quality and accessibility of legal data

Data forms the foundation of the ML opportunities for the legal sector. Many respondents emphasised that access to structured, high quality data will be essential to improving the accuracy of ML systems and mitigating the risk of bias. Consequently, the absence of sufficiently large data sets will likely stifle innovation in this space, even for the largest providers.

Our research flagged up one important data challenge that is faced by law firms of all sizes and corporate legal teams: they are trusted advisers who routinely require confidential data which may include personal data and commercially sensitive data in order to advise/represent their clients. Using/sharing this data requires permission and it would have to be cleansed/anonymised before it could be used by ML applications (or to train them). Furthermore, because the client data they work with relates to specific matters, it may not be representative of the wider market, and therefore not relevant or appropriate for training ML algorithms. Respondents also flagged up the lack of current/relevant market data around legal matters - compared with other markets like retail or automotive.

There is much to be gained from promoting better data practices across the legal sector. While LawtechUK has taken significant steps to further this goal (Legal Data Vision), there is scope for legal service regulators to facilitate further progress, particularly in respect of encouraging collaborative data sharing and issuing clear guidance on how legal matter data can be used to develop ML systems whilst remaining compliant. Although the variations in available datasets cause disparities between small and large providers, it is important to highlight that even large firms rarely hold sufficient volumes of data to fully benefit from advanced AI.

LawtechUK recommends

On the basis of the received feedback, we have highlighted several outstanding challenges facing the industry. While LawtechUK may not be able to take all these actions forward, we would like to encourage the regulators and other market participants to tackle them in the near future, especially in respect of:

• More research should be carried out to understand the potential benefits that the development of ML powered legal services systems can deliver to consumers and how best to achieve those benefits

• There is an opportunity for legal service regulators and legal trade bodies to drive responsible use of ML for the benefit of the consumer, while providing support and guidance to regulated entities to do this

• There is an apparent demand for more R&D funding as well as general support for innovators to develop, test and scale ML powered uses cases that improve accessibility of legal services, whether through the Lawtech Sandbox or incubator style support

• In order to facilitate a closer collaboration with the insurance sector, the Regulatory Response Unit should host a meeting with PII insurers to raise awareness of the considerations and implications of AI and the emerging use cases in legal services to support a better understanding amongst insurers

• A robust data ecosystem will facilitate further innovation in this space, and as such it is critical that improved data practices are promoted and collaboratively channelled across the legal industry. For more information, visit: Legal Data Vision.

Thank you

Thank you to all those that provided case studies and responses to the consultation process, which have contributed to improving the sector's understanding and furthering the responsible adoption of ML in legal services, for the benefit of society and the economy.

If you are interested in following up on any of the recommendations of findings, please do get in touch at lawtechuk@technation.io.