Google loves to tell us how people are using its generative AI products to create new things, grow businesses, and save the world. It is believed. Of course, people are also using AI for crime. Google has announced a new legal defense aimed at a Chinese group called Outsider Enterprise, which is allegedly responsible for a massive AI-powered scam campaign. Google says it’s working with law enforcement and mobile carriers to counter it.
According to Google’s legal filings, Outsider Enterprises operates through Telegram. The group provides phishing-as-a-service to individuals who may not be technologically savvy enough to set up fraudulent websites and text campaigns themselves. In its Telegram channels, Outsider Enterprise reportedly posted instructions to use Google’s Gemini AI to create a website that mimics Google, YouTube, and government agencies like New York’s E-ZPass. The group offered approximately 300 scam templates.
Google says more than 2.5 million text messages were sent to Android users as a result of scams enabled by Outsider Enterprise. About 55,000 of these messages occurred over a two-week period last month. In total, Google tracked 9,000 fake websites and 1 million URLs associated with the scam network.
The text messages often contained claims about account issues or problems with package delivery. When users clicked on the link, they were redirected to one of the fraudulent websites that were designed by Gemini to appear legitimate. Cyber criminals use these sites to steal personal data and banking details. Google’s filing did not estimate the amount of money stolen through outsider enterprise scams, but the blog post said hundreds of people may have lost some of the funds.
Google worked with AT&T, Verizon, and T-Mobile to block many of these malicious text messages, and Google notes that its on-device scam detection in Google Messages also likely helped reduce the number of successful phishing attempts. This AI-powered feature apparently blocks 10 billion scam messages every month, so it’s reasonable to expect that it has caught at least some outsider enterprise activity.
<a href