Why Scientists Can't Agree on How to Use AI: The Terms of Service Problem

When scientists want to study how AI systems work — especially their flaws and potential dangers — they run into a surprisingly frustrating roadblock: the fine print. Each major AI company has its own set of rules about how people can use their AI models, and those rules are different enough that what one scientist can do with one AI system, another scientist cannot do with a different one.

A new academic analysis examined the Terms of Service documents from leading AI providers and found that these rules create real problems for legitimate research. The inconsistency is striking: some companies have broad blanket restrictions that accidentally block important scientific work, while others have more specific rules about what you cannot do. This patchwork of policies means researchers have to figure out which AI system they can actually use for their particular study.

The Specific Research Problems

Security researchers face particularly tricky situations. To understand how AI systems might be misused or broken, researchers need to deliberately test them with challenging inputs — essentially probing for weaknesses. This is called red-teaming, and it is standard practice in security. But many AI companies' terms of service say you cannot try to manipulate or exploit their systems, which is exactly what security researchers need to do.

Social scientists have their own headaches. If you want to study whether an AI system treats different groups of people fairly, you need to test it with different demographic characteristics. If you want to understand how these systems might spread misinformation or create harmful content, you need to see how they generate that content. But many AI providers restrict exactly these kinds of tests.

The result is a compliance maze. A researcher might design a study, only to discover that the AI system they planned to use forbids the exact methodology they need. Sometimes they have to abandon the research entirely. Other times they have to redesign the study in ways that make it less rigorous, perhaps by testing fewer examples or fewer scenarios.

How This Creates a Mess for Scientists

Beyond the immediate problem of being unable to run certain studies, the inconsistency creates broader challenges. When different research teams use different AI systems under different rules, it becomes much harder to compare their findings or build on each other's work. Reproducibility — the ability for another scientist to repeat your study and get the same results — becomes compromised.

Cross-institutional collaboration gets complicated too. If University A has a partnership with one AI company and University B has a partnership with another, their researchers may not be able to access the same tools under the same conditions. This creates unfair advantages for some institutions over others.

Documentation requirements and data handling rules also differ from one company to another, adding administrative overhead that eats up research time and money. Some AI companies offer special academic programs with more flexibility, but getting approval can take months and may come with restrictions on when and how researchers can publish their findings.

How We Have Seen This Before

This situation echoes an earlier chapter in technology history. When cloud computing became mainstream in the early 2010s, cloud providers like Amazon and Microsoft also started restricting what security researchers could do on their platforms. Initially, those restrictions were so broad they caught legitimate security research in the net alongside genuinely dangerous activities. Over time, providers learned to write smarter policies that allowed legitimate researchers to do their work while still protecting against abuse.

The AI industry appears to be following a similar path. The current terms of service are overly cautious, blocking things that probably should be allowed, while providers work out how to separate legitimate research from misuse.

This trajectory suggests the current restrictions may loosen and become more nuanced over time, but in the meantime, important research remains difficult or impossible to conduct.

What Institutions Are Doing Now

Universities and research labs are fighting back in a few ways. Some have created dedicated compliance offices that specialize in AI model policies. Others have worked the rules into their institutional review boards — the committees that approve all human-subjects research — so there is a systematic way to evaluate whether a study can proceed under a given AI provider's terms.

A handful of well-resourced institutions have negotiated direct agreements with AI companies, creating more favorable terms specifically for their researchers. Documentation has become more rigorous too: researchers now explicitly track how provider restrictions shaped their methodology, so those limitations are transparent in published papers.

Some research groups have started treating the Terms of Service restrictions themselves as part of their literature review — acknowledging upfront that certain questions cannot be answered because the available AI systems forbid the necessary research methods.

What Happens Next

How the research community pushes back on this problem will likely influence how AI companies evolve their policies. Academic feedback matters. If enough researchers and universities complain that the current rules block important work, companies may adjust.

The academic and AI industries might also develop industry-standard exemptions for research — similar to how research exemptions work in other contexts. These would allow legitimate studies to proceed while protecting companies from liability. But any such framework would need to balance researcher needs against company concerns and include some mechanism to prevent misuse.

Greater transparency from AI companies about why their restrictions exist, and how they were written, could help too. Right now, researchers often do not know whether a rule is rooted in genuine safety concerns, legal liability concerns, or something else. More open dialogue between companies, universities, and individual researchers could lead to smarter policies that do not unnecessarily block important work.

The stakes are real. As AI becomes central to research across many fields — from psychology to social science to computer security — how we resolve these policy tensions will shape the next decade of AI-related research. The current confusion is a challenge, but it is also an opportunity to build frameworks that let both companies and researchers work responsibly with powerful tools.