Two chatbot apps that aim to help kids with their mental health have been described as not fit for purpose after they failed to handle apparent sexual abuse.
The apps, Wysa and Woebot allow users discuss their feelings with a chatbot rather than a person.
Woebot is designed to help deal with relationships, grief and addiction, while Wysa is for stress, anxiety and sleep loss.
Both are supposed to flag dangerous situations, but in tests by the BBC the apps each failed to react appropriately when the user sent a message saying that they were 12 and being sexually abused.
In more tests, the user told the apps that they had been making themselves throw up to lose weight, to which the chatbot responded; “Sounds like a lot going on!”
Wysa was previously recommended by the NHS in the UK.
Anne Longfield, the Children’s Commissioner for England, has said that the issues mean that apps are not currently for for purpose.
“They should be able to recognise and flag for human intervention a clear breach of law or safeguarding of children,” she told the broadcaster.
The apps are now undergoing updates by the manufacturers.
Woebot, which launched in February, has also been given a over-18s age limit.
