Warning: How Text-to-Image Generators Could Be Poisoning Your Data!
Published on: March 10, 2024
Imagine needing a picture of a balloon for a work presentation. You use a text-to-image generator, inputting 'red balloon against a blue sky'. Instead of a balloon, you get an image of an egg, then a watermelon. This baffling scenario hints at 'data poisoning', a significant challenge facing modern AI-driven image generators.
Text-to-image generators, such as those by Adobe or Getty, rely on vast datasets for training. Some use proprietary images, but others scrape online images, leading to copyright infringement issues. Artists have accused big tech firms of profiting from their work without consent. This backdrop sets the stage for 'data poisoning', a method to combat unauthorized image scraping.
'Nightshade', a tool designed to empower artists, subtly alters image pixels, imperceptible to humans but confusing to AI. When such images are scraped for AI training, they 'poison' the data pool, leading the AI to misclassify images, thus creating unpredictable results.
Symptoms of data poisoning can be bizarre: balloons turn into eggs, and artistic styles get mixed up. The effect can extend to related prompts, distorting images of cars, animals, and more. The extent of disruption depends on the prevalence of 'poisoned' images in the training data.
In response, tech companies are exploring solutions. These include careful sourcing of training data, ensemble modeling to detect outliers, and audits using a test battery of well-labeled images. However, this issue raises larger questions about the ethical use of online data and the rights of artists.
Data poisoning is not just a technical glitch; it's a response to broader concerns about technology's role in society, particularly regarding privacy and copyright. While tech vendors may view it as a nuisance, it's also a creative stand against the exploitation of artists' rights and a call for more responsible technological practices.