Researchers poison their own data when stolen by an AI to ruin results

• Researchers from China and Singapore proposed AURA (Active Utility Reduction via Adulteration) to protect GraphRAG systems
• AURA deliberately poisons proprietary knowledge graphs so stolen data produces hallucinations and wrong answers
• Correct outputs require a secret key; tests showed ~94% effectiveness in degrading stolen KG utility

Researchers from universities in China and Singapore came up with a creative way to prevent the theft of data used in Generative AI.

Among other things, there are two important elements in today’s Large Language Models (LLM): training data, and retrieval-augmented generation (RAG).

Training data teaches an LLM how language works and gives it broad knowledge up to a cutoff point. It doesn’t give the model access to new information, private documents, or fast-changing facts. Once training is done, that knowledge is frozen.

Replacing outdated gear

RAG, on the other hand, exists because many real questions depend on current, specific, or proprietary data (such as company policies, recent news, internal reports, or niche technical documents). Instead of retraining the model every time data changes, RAG lets the model fetch relevant information on demand and then write an answer based on it.

In 2024, Microsoft came up with GraphRAG - a version of RAG that organizes retrieved information as a knowledge graph instead of a flat list of documents. This helps the model understand how entities, facts, and relationships connect to each other. As a result, the AI can answer more complex questions, follow links between concepts, and reduce contradictions by reasoning over structured relationships rather than isolated text.

Since these knowledge graphs can be rather expensive, they could be targeted by cybercriminals, nation-states, and other malicious entities.

In their research paper, titled Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systems, authors Weijie Wang, Peizhuo Lv, et al. proposed a defense mechanism called Active Utility Reduction via Adulteration, or AURA - which poisons the KGs, making the LLM give wrong answers and hallucinate.

The only way to get correct answers is to have a secret key. The researchers said the system is not without its flaws, but that it works great in most cases (94%).

"By degrading the stolen KG's utility, AURA offers a practical solution for protecting intellectual property in GraphRAG," the authors stated.

Via The Register

The best antivirus for all budgets

➡️ Read our full guide to the best antivirus
1. Best overall:
Bitdefender Total Security
2. Best for families:
Norton 360 with LifeLock
3. Best for mobile:
McAfee Mobile Security

Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the Follow button!

And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form, and get regular updates from us on WhatsApp too.