AI & Social Media: Detecting Health Side Effects

“Help me please … I can’t calm down‍ without laying on the ground and freaking out ⁣for a good 20 minutes … Should ⁤I ⁢get medical help?”

This plea came from a post on the social ⁢media site Reddit. The person who⁤ posted the question had been having ⁢panic attacks for several days ⁤after smoking marijuana. Usually, this type of post goes unnoticed‍ by people working in public health. But in a ⁤recent experiment, an AI tool was paying attention.

The tool, called Waldo, reviewed more than 430,000 past ⁢posts on Reddit forums ⁤related to cannabis use. It flagged the post‍ above and over 28,000 others as potentially describing unexpected or harmful side ⁣effects.⁤ The researchers checked 250 ‍of ‍the posts that⁢ Waldo had flagged and ‍verified that 86 percent of them indeed represented problematic experiences with cannabis products, researchers report September ⁣30 in PLOS Digital Health.‍ If this type⁢ of⁤ scanning became commonplace, the information could help public ‍health workers protect consumers from harmful products.

The beauty of the work,says Richard Lomotey,is that it shows researchers can actually gain information from sources that government agencies,such as the U.S. Centers for ‍Disease ‍Control and Prevention, may not be looking at. ⁢The CDC and other‍ agencies ⁣take surveys or collect self-reported side effects⁤ of illness ‍but do not monitor social media. This ⁣is ⁣where “people express themselves freely,” says Lomotey, an information technology expert ‍at Penn State.

Many people⁣ don’t have access ⁢to a doctor⁤ or ⁣don’t⁣ know about the official ⁤way to report a bad⁢ experience with ⁢a product, says John Ayers, a ⁢public health researcher at the⁤ University of California, San Diego in la‍ Jolla who worked on Waldo. ‍Lots of people share health experiences online. “We need to go where they are,” he says.

Karan Desai, a medical student at the ⁤University of Michigan Medical School in Ann‍ Arbor, says ‍the team ⁣chose to focus on cannabis products because they are very popular yet largely unregulated.⁣ “People in my age demographic, in their 20s, grew up ⁢in ⁣high school and college with these JUULs, these vapes, these cannabis products,” he says.“I think it’s important for ⁢us to know what side effects people are experiencing ‍with using these.”

To prepare Waldo, the team⁢ began with a smaller group of 10,000 different Reddit posts about cannabis use. Other ‍researchers had gone through these and identified problematic side⁣ effects by hand. Desai and colleagues trained Waldo on a portion of these posts, then tested it on the remaining⁣ ones.On this task,the tool outperformed ChatGPT. The general-purpose bot marked 18 times more ⁣false positives, indicating posts⁤ contained side effects when they didn’t.⁢ But it did not outperform the⁢ human reviewers.

This ⁢all happened before the team’s main experiment, in which Waldo tagged that panic attack post and tens of thousands more.

It remains to be seen whether Waldo would work⁤ as well searching for issues related ⁣to any kind⁣ of drug, vitamin or other product, Lomotey says. ‍AI tools trained on one task may not work as well⁣ even on very similar tasks. “We have to be cautious,” he says.

Still,⁤ Lomotey⁢ imagines a ⁣future where tools like Waldo would‍ help keep an eye on⁢ social media. This would⁢ need to be done carefully, “in an ethical way,” he says.When a person posts about a rare⁢ side effect,such tools could flag the issue and pass it on to health officials,with privacy protections in place.⁣ He imagines that ⁢this could be especially useful in ⁢countries that don’t have robust systems in place to monitor and report on drug side effects.

Someday, tools like Waldo might help ⁢link people who⁤ need help to the public⁤ health workers who can provide it. “Even⁣ when [side effects] can be ⁤rare, when they happen⁢ to you, it means all the world,” Ayers says.