Police are trialling the use of sophisticated artificial intelligence (AI) techniques to detect sexist threats and abuse online, in the hope it will catch perpetrators earlier and protect victims.

Digital forensic experts at the Forensic Capability Network (FCN) are working with researchers at Warwick University to develop the natural language processing (NLP) model.

NLP is a branch of AI that helps computers understand how humans write and speak. These models can be particularly helpful for non-technical professionals such as police officers to quickly identify information or patterns within large amounts of text.

Although the messages may be sent online, most commonly via social media or messaging apps, the application of the NLP would be offline and applied to the data extracted from the phone or computer where the messages have been received or sent, and where there are large numbers of messages over a prolonged period. This would be done with either agreement of the victim or where the device has been seized from the suspect as part of a criminal investigation.

One of the objectives of the project is to test whether the technology can help to preserve a person’s privacy, as it would avoid investigators looking at irrelevant information which could be flagged through more basic searches.

Humans would always be the decision maker if this technology was used, and AI would simply be a useful tool to help triage and prioritise information, and improve how such data is currently reviewed to provide faster outcomes for victims.

Researchers have already tested the model’s ability to analyse 20,000 online posts labelled as sexist, and have enhanced its ability to detect subtler forms of microaggressions such as stereotypes and objectification.

Next, researchers will develop a text-processing pipeline to classify sexist content towards women and explore ‘large language models’ which will enable investigators to analyse messages through specific dialogue prompts. Psychologists are also supporting the project to create ‘personas’ of potential perpetrators based on dynamic text summaries of sexist content over time.

Once complete, the project’s output could ultimately be used by police officers investigating online threats and abuse as part of a live investigation, or for crime prevention. It could also speed up complex investigations where digital forensic analysts are faced with huge volumes of messages and other digital evidence.

FCN’s research manager Shelley Wilson said:

“Identifying abusive or controlling behaviour can be time-consuming and challenging for investigations, and we know technology exists that can efficiently identify suspicious behaviour and language. We see this as a high-impact and practical way to use AI to support investigators and help victims.

“Tackling violence against women and girls is a top priority for the police and the public. Clearly digital forensics has a big role to play, and we’re still looking for policing partners to share data to train and improve the model.”

The project secured investment worth £148,000 from the police Science Technology and Research (STAR) fund run by the office of the police chief scientific adviser, Professor Paul Taylor.

This project is using synthetic data or openly available datasets for training NLP models. There is no use of live casework or data from police forces.

There are currently no plans to take the project further than the proof of concept.

Find out more about FCN’s research activities.

We see this as a high-impact and practical way to use AI