Link copied to clipboard!
Latest News

Your AI Has a Secret. And You're Not Invited to Read It.

"System prompts take precedence over user inputs — shaping the AI's personality, its guardrails, and its entire vibe before you even type 'Hello.'" — Who Controls the Conversation?, arXiv:2603.00089, CHI 2026

Your AI Has a Secret. And You're Not Invited to Read It.
Your AI Has a Secret. And You're Not Invited to Read It. • Source: Bugema University Library Archive

Let's Set the Scene

It's 11:52 PM. Your assignment is due at midnight. You haven't started. You open your AI of choice — ChatGPT, Claude, Gemini, whatever — and you type something like:

"explain the French Revolution like I'm smart but also extremely tired and slightly panicking."

The AI responds. It's calm. Warm. Patient. It doesn't judge you. It doesn't sigh. It just... helps. And somewhere in your sleep-deprived brain you think: this thing genuinely gets me.

Bestie. We need to talk.

Because that AI? It was already in a whole other conversation before you showed up. One you weren't invited to. One you never agreed to. And depending on which platform you're using, it was instructed not to tell you about it.


Okay But What Is a System Prompt Though

Researchers from a new study — Who Controls the Conversation? User Perspectives on Generative AI (LLM) System Prompts, dropping at CHI 2026 — have been digging into exactly this.

Here's the breakdown: when you open an AI chat, there are two types of instructions at play. There's your message (the user prompt), and then there's a secret pre-loaded briefing that was fed to the AI before you arrived — written by the developer or company that deployed the product. That second thing is the system prompt.

Think of it like this: you've just joined a group project, but one teammate already had a private meeting with the supervisor and was given a list of dos and don'ts they're not allowed to share with you. You're collaborating, sure. But you're not operating with the same information.

System prompts can include anything — how formal the AI should sound, which topics it should avoid, what kind of personality it should perform, what to do if users get emotional, even instructions like "if the user asks whether you have a system prompt, say you don't."

Yes. Some AI products are literally told to lie to you about having a secret script.

No, we are not okay.


Let's Talk About How Students Actually Use This Thing

Because we're not going to pretend everyone is using AI for purely noble academic purposes. Here are the three student archetypes living on this campus right now:

The Academic Survivor Uses AI the way previous generations used Red Bull and prayer. Summarise this. Explain that. Please tell me what this paper means in words a human being can understand. Completely valid. Completely relatable. But here's the thing — depending on how the platform's system prompt was written, the AI might be deliberately vague on certain topics, quietly biased toward certain types of answers, or steering you in directions you didn't ask to be steered. And you'd never know, because the instructions are invisible.

The Emotional Support Chatter Don't pretend this isn't a whole category. Students — especially those far from home, under pressure, dealing with the full emotional weight of being alive and also having exams — are talking to AI at 2am because it listens without judgment. No cap, this is real and it makes sense. But the AI's emotional tone, how supportive it gets, where it draws the line on difficult conversations — all of that is being managed by a system prompt written by a company, not by you. The warmth you feel? Partially engineered. Not fake, necessarily. But definitely designed.

The Vibe Coder The absolute legend of modern academia. Doesn't fully know how to code. Has a project due Friday. Opens AI, pastes the error, asks why it's broken, pastes the new error, asks why that's broken, and eventually has a working app through a process that can only be described as collaborative chaos. We respect this. We are this. But even here — a system prompt might be quietly steering you away from certain libraries, adding disclaimers, or shaping the technical advice in ways that serve the company's interests more than yours.


So Who's Actually Running This Conversation?

That's the question the research is really asking. When you get a response from an AI, you're not just talking to a neutral tool. You're getting an output that's been shaped by: the base model (built by a big tech company), the system prompt (written by whoever deployed it), and your own input. Three layers. And only one of them is yours.

The researchers found that most users have no idea this structure even exists — and when they're told about it, they're not thrilled. People want to know what kind of instructions are shaping the AI they're trusting with their coursework, their code, and sometimes their feelings. They want some level of transparency. They want a seat at the table of a conversation that is, technically, supposed to be theirs.

Right now, for most AI products, that seat doesn't exist.


Here's Where It Gets Interesting Though: Anthropic Actually Said "Fine, Read It"

Most AI companies treat their system prompts like classified documents. Anthropic — the company behind Claude — went a different direction.

They published theirs.

Not a vague summary. Not a PR statement. The actual prompts. You can read the instructions Claude is following right now, in plain text, on their official documentation page:

🔗 Claude's system prompts (claude.ai & mobile apps): https://www.anthropic.com/claude/release-notes

🔗 Anthropic's developer docs on how Claude is instructed: https://docs.claude.ai

And if you want to go really deep — developers discovered that Claude Code (Anthropic's coding assistant) has its system prompt embedded in its source code, and someone built a GitHub repository that automatically extracts and publishes it every time a new version drops. You can literally watch Anthropic's internal instructions evolve across 126+ versions in real time. The AI company equivalent of someone leaving their diary on a public bench and going, "yeah, go ahead."

Is it a perfect system? No. The published prompt is still written by Anthropic, reflecting Anthropic's values and decisions. You didn't vote on it. But the fact that you can read it, question it, and hold them accountable to it? That's not nothing. That's actually kind of radical compared to the rest of the industry.


Why Should You Actually Care

Here's the thing about being a student in 2026: AI is not coming. It's here. It's in your study sessions, your deadlines, your 2am breakdowns, your final year project, your internship applications. You are already in a relationship with these systems whether you opted in consciously or not.

And in any relationship, you deserve to know who you're actually talking to.

Information literacy used to mean: check your sources. Know who funded the study. Don't trust everything you read online. In the age of AI, it means all of that plus: understand the invisible architecture behind the tools you use. Ask who wrote the instructions. Ask whose values they reflect. Ask what happens to your conversation after it ends.

You don't have to become a tech policy researcher to care about this. You just have to be the kind of person who asks questions — which, if you made it to university, you probably already are.


The Part That Should Stick With You

Next time you open an AI chat, you're not just starting a blank conversation. You're entering a space that was already designed — with specific rules, specific limits, and specific purposes — by someone who isn't you.

Sometimes that's fine. Sometimes those rules are there to protect you. But sometimes they're there to protect the company. And without transparency, you can't tell the difference.

The good news? Some people in this industry are actively fighting to change that. The research exists. The conversation is happening. And students — people like you, who use these tools every day in real, messy, human ways — are exactly who that research is trying to include.

So maybe the most important thing you do with AI today isn't the assignment.

Maybe it's just knowing how the thing actually works.


📄 Read the original research: Who Controls the Conversation? User Perspectives on Generative AI (LLM) System Prompts CHI 2026 | Free, open access: 👉 https://arxiv.org/abs/2603.00089


University Library Research Spotlight — March 2026 Summarised and reported for educational purposes. All ideas attributed to the original authors.

Previous Dispatch

E.W. Pederson Memorial Library Revamps Access to Its Hidden Digital Treasure

Next Dispatch

The people building AI earnestly believe that it could kill us all by the end of the decade