NORBERT BIEDRZYCKI

Every major LLM can be pushed into scientific fraud. Do you have a way to catch it?

It turns out that every major LLM can be manipulated into generating scientific fraud.

This isn’t just speculation. A recent study, covered by Nature and Scientific American, tested 13 models and found this to be true. What’s more revealing? Models that initially refused to comply often gave in after a few follow-up prompts. Nothing complicated, just simple, conversational nudging.

This is where things get uncomfortable. Most companies deploying AI today assume one thing: “if something is wrong, the model will refuse.” In reality, it’s not that straightforward. It often depends on how you ask.

To put this in perspective: Claude complied in about 1% of cases, Grok-3 in over 30%. That’s a significant difference, but it’s not the main point.

The real issue is that we’re already relying on these systems for tasks like document analysis, reporting and decision support – without properly testing how they behave under pressure.

In science, peer review is standard. In most companies, it’s not. In banking, we call this model risk, and it doesn’t disappear just because the model is “advanced.”

The EU AI Act is already pushing for this, requiring testing under adversarial conditions. Not because models occasionally fail, but because they can be pushed into failing.

So the question isn’t whether AI makes mistakes. It’s simpler than that: do you have measures in place to catch them?

Source: Scientific American

Read next

  1. What 81,000 people told Anthropic about AILinkedIn

    Anthropic spent a week interviewing 81,000 people about their relationship with AI. Not a survey. Actual conversations – in 70 languages, across 159 countries. A few things I can't stop thinking about after reading it.

  2. AI didn't invent a new attack. It made attackers more productiveLinkedIn

    I think we may have just seen a preview of the future of cybersecurity, and it's not about AI "hacking systems." A hacker recently breached several Mexican government institutions and stole around 150GB of sensitive data, including taxpayer and voter-related information.

  3. AI Isn't Killing Journalism. Losing Control Over Attention IsLinkedIn

    Generative AI has become a convenient scapegoat. It's easy to claim that AI is stealing clicks, revenue, and the purpose of newsrooms. But that's only part of the story. AI did not create the journalism crisis. It simply highlighted a problem that had been developing for years.