Earlier this month, Joseph Thacker’s neighbor told him that she had pre-ordered some stuffed dinosaur toys for her children. She chose toys called Bondas because they offered an AI chat feature that lets kids talk to the toy like a machine-learning-enabled imaginary friend. But she knew that security researcher Thacker had worked on AI risks to children, and she was curious to hear about his thoughts.
So Thacker took notice of this. With just a few minutes of work, he and a web security researcher friend named Joel Margolis made a startling discovery: Bondu’s web-based portal is intended to allow parents to scrutinize their children’s interactions and allow Bondu staff to monitor the products’ use and performance, as well as give anyone with a Gmail account access to a transcript of nearly every interaction Bondu’s child users have with toys.
Without carrying out any actual hacking, by simply logging in with an arbitrary Google account, the two researchers immediately found themselves eavesdropping on kids’ private conversations, the pet names kids gave to their Bondu, the likes and dislikes of the kids’ owners of toys, their favorite snacks, and dance moves.
Overall, Margolis and Thacker found that the data Bondu had left unsecured — accessible to anyone who logged into the company’s public-facing web console with their Google username — included children’s names, birth dates, names of family members, the “purpose” for the child chosen by the parents, and, most disturbingly, detailed summaries and transcripts of every previous interaction between the child and their Bondu, a The toy was practically designed to achieve intimacy. One-on-one conversation. Bondu confirmed in conversation with researchers that more than 50,000 chat transcripts were accessible through the exposed web portal, essentially all conversations except for toys that were manually deleted by parents or staff.
<a href