Google's AI Chatbot Faces Copycat Attempts: A Brief Insight
Google's Gemini, a highly advanced AI chatbot, has been under attack recently. A group with commercial interests has reportedly attempted to clone the chatbot's knowledge base. The attackers went as far as prompting the model over 100,000 times in several non-English languages with the aim of using the responses to train a less expensive imitation.
The practice, known as "model extraction", is seen by Google as theft of intellectual property. The irony of this situation cannot be overlooked, as Google's LLM was developed using online materials without permission. Moreover, Google itself was previously accused of similar practices.
A Look into the Past
A few years ago, it was reported that Google's Bard team was alleged to have used outputs from ChatGPT, a chatbot conversation sharing platform, to train its own chatbot. These actions led to the resignation of a senior AI researcher from Google, who joined OpenAI after alleging this practice violated OpenAI’s terms of service. Despite denying the accusations, Google reportedly stopped using the data.
Regardless of past events, Google's current terms of service prohibit extraction of data from its AI models in this manner. The company believes the attackers are mostly private companies and researchers looking to gain a competitive edge. The attacks have reportedly originated from various parts of the globe. However, Google has refrained from naming any suspects.
The Art of Distillation
Within the industry, the practice of training a new model using a previous model’s outputs is commonly referred to as "distillation". For those lacking the resources and time that Google invested in training Gemini, using a previously trained LLM can serve as a shortcut.
Distillation involves feeding an existing AI model with thousands of carefully selected prompts, collecting all the responses, and then using these input-output pairs to train a smaller, cheaper model. Although the resulting model typically mimics the parent model’s output behavior, it is generally smaller in size. While not perfect, distillation can be a more efficient training technique than relying on random Internet data, which often contains a lot of noise.
The resulting model never sees the parent model’s code or training data. However, by examining enough of its outputs, it can learn to replicate many of its capabilities. It can be compared to reverse-engineering a chef’s recipes by tasting every dish on the menu and working backward.
Continuing Threats
Google's threat intelligence group has reported a rising number of distillation attacks against Gemini. Many of these attacks specifically targeted the algorithms that help the model perform simulated reasoning tasks. After identifying the 100,000-prompt campaign, Google adjusted Gemini’s defenses, though the specifics of these countermeasures have not been detailed.
Industry-wide Concerns
Distillation isn’t a concern for Google alone. It was previously reported that Chinese competitor DeepSeek used distillation to enhance its own models, and the technique has become a standard practice across the industry for building cheaper, smaller AI models from larger ones. The boundary between standard distillation and theft is blurry and depends on whose model is being distilled and whether permission has been granted. This is a distinction that tech companies spend billions to protect but has not yet been tested in court.
Distillation is also a common practice within companies to create smaller, faster versions of older, larger AI models. For example, OpenAI created a mini-version of its GPT-4o, and Microsoft built its compact Phi-3 model family using synthetic data generated by larger models. Furthermore, DeepSeek has officially published six distilled versions of its R1 reasoning model, with the smallest one being able to run on a laptop.
As long as an LLM is accessible to the public, there doesn't seem to be a foolproof technical barrier that can prevent a determined actor from cloning someone else’s model over time. This is exactly what Google alleges happened to Gemini.