AI Companies Pose an Imminent Threat to Scientific Culture

During the preceding weeks I have followed a string of AI-driven “breakthroughs” in pure mathematics with initial excitement, growing discomfort, and now with a mix of fear, anger, and deep sadness. Recently, American AI companies, namely OpenAI and Anthropic, have achieved major milestones in the intertwined histories of math and computer science by offering up solutions to the Unit Distance Conjecture and a blowup for the Navier-Stokes—with more major results rumored to be announced pending the development of press materials. These results were pursued with total indifference towards the feelings and livelihoods of the mathematical community; these results were pursued without curiosity or with any interest in their scientific consequences. Instead, open problems that were intended to serve as rallying points for the creation of new human knowledge have become, have been very intentionally transformed into, more sterile benchmarks used to promote upcoming model capabilities. Below, I will briefly summarize my thoughts on these matters as a non-mathematician.

Beginning in the 20th century, new and powerful technologies began to substantially augment long-standing academic disciplines. Take for instance the atmospheric sciences. The advent of automatic computing machines (henceforth “computers”) introduced after World War II enabled high-resolution, low-latency simulations of the atmosphere. Another general description of a computer is: a device that follows protocols of symbolic manipulation. That is, a computer is a great big machine for doing, carrying out instructions from human operators tasked with the job of thinking up routines. In the case of atmospheric science, a great number of graduate students, academics, and scientists devoted their careers during the digital computer age to the development of new parameterizations–essentially simplified versions of the theoretical physical equations that describe the motions of our atmosphere. I can imagine that these developments caused a row within the community. But ultimately, meteorology survived and thrived because of new technology, growing into the vibrant community it is today.

I find the similarities between this story and the current situation in mathematics striking. Once again, intellectuals in a well-established discipline are being forced to confront a fundamental change in their lives and in the character of their work. Today, yet again, a new technology seems primed to yank practitioners of abstract reasoning up to the land of applications. Except that this time, the speed and violent nature of AI-integration, forced AI-assimilation, seem to me more consequential. First, an alien notion of competition dominates the present discourse. While meteorologists worked alone, participating in collegial discussions, to adopt computers into their working lives responsibly, AI companies have created an “us versus them” dynamic, where OpenAI and their ilk are posturing as the new kids on the block. Competition, specifically bitter, winner-take-all competition, is inherently antithetical to the spirit of science. Second, I struggle to recall a time in the history of science during which a private company actively invaded and undermined the culture of a group of academics. Mathematicians are (in my very limited experience) an insular yet warm brood who treat each other with fierce loyalty and compassion, and who are more than happy to share their work with the world provided they can do so on their own time, and on their own terms. Historically, this approach has worked wonders to build a self-contained school of rigorous thinkers. And to this point, this school has comprised a very rich and a very important culture that has benefited society in manifold ways as it has grown organically. AI companies have demonstrated quite unambiguously that they have no interest in the preservation of cultural traditions. In fact, recent events seem to conform disturbingly well to patterns of colonialism: a group of outsiders possessing powerful technologies presumes to know what’s best for another group of people—and goes on to “solve” their problems without prior consent or discourse.

So what is mathematical culture? Of course, this is not for me to say. But more generally, I will argue that many academic pursuits share the aim of human development: expanding the capabilities of individuals to explore, understand, and positively affect the world. What I find personally exciting about mathematics is not the knowledge that some theorem is true or false, but the opportunity to expand my own understanding of what a human brain can do! The success of human mathematicians empowers us all to become more intellectually daring. This fact also partially explains the value of instruction, outreach, and communication in the life of any academic. This somewhat sentimental argument is not intended to undermine the vital role mathematics has played in forming the logical basis for all empirical science. This task, essentially involving the formation of a set of rigorous standards for logical reasoning, is simply too important, too messy and too subjective to leave in the hands of a generative model.

And what must be done? For the short term, first, I can offer a prediction. The math community will broker a formal truce with major AI companies—a ceasefire of sorts—that will afford academics the time they need to restructure their profession for a new technological era. But in the long term, serious regulatory action is required to prevent hostile takeovers of other scientific disciplines. In the same way certain clinical studies in the biological sciences require formal pre-approval, AI companies should at the least be required to obtain express permission to conduct and report on compute-intensive experiments containing the possibility to cause sudden and disruptive impacts to society. To support these reforms, American citizens have the option to appeal to their democratic representatives directly. And all others who are dissatisfied with the recent behavior of large AI firms may vote with their dollars by choosing to use open source models. That is, we ordinary folk should focus our criticisms toward AI companies—rather than launch vague assaults on AI as a technology.

TLDR; The fate of mathematics may serve as a cautionary tale for all academics during years to come. We cannot forget, that the objectives of for-profit corporations are fundamentally different from our own. If the walls separating vibrant intellectual communities from the outside world shatter, we risk eroding the abilities to create and maintain our human knowledge. And for anyone else taking interest in recent events, do not be fooled; trillion dollar companies are not on the team of populism; remember that the position of a mathematician in society is, and always has been, a very fragile thing.