Sophie Huiberts 批评 OpenAI 预印本质量低劣：正常水平的糟糕工作

[This is a guest post by Sophie Huiberts, being an expanded version of a post on her own blog. This blog post was initially written in a different file format and converted using AI. — T.]

When you prove a theorem, you are imparted a responsibility to the field of mathematics. At minimum you need to explain what you did, including which steps required new ideas and which steps are already well-known. More substantively, you are expected to nurture the literature. If an important idea was poorly explained in its first incarnation, then you have a duty to explain it better in your work. The responsibility is yours because you have the opportunity to publish a paper on the subject.

These demands are broadly accepted and are neither new nor controversial. Any time you exceed the demands, the resulting paper is highly appreciated and valued. Any time you fail the demands, you will get a “revise and resubmit” for a journal submission, or a straight rejection for a conference. Until you fix your exposition, your ideas will not enter the canon. If you post unpolished manuscripts to the arXiv, you are embarrassing yourself.

Compare this to OpenAI’s behavior. They do not care about their “preprints”. They don’t bother putting in appropriate attribution of ideas, they salami-slice 372 results over 719 pdfs, their work contains retraction-worthy errors, they promise code but neglect to include it, and their proofs are fiendishly hard to parse. Their pdfs actively set back the state of the literature. This is poor quality work and these pdfs are all easy rejects.

We all know that OpenAI is not doing this “for the good of science”. These pdfs are exhaust fumes from their internal benchmarking effort. We do not need them to release this unprocessed exhaust into the atmosphere. OpenAI’s new internal model may have produced these texts with the press of a button. But similar results would have been found soon by experts, with or without LLM prompting. Some results were already found by ChatGPT users and were in the process of being written up properly. Even without alleging theft, it is a bad look for a supplier of scientific software to actively scoop their customers.

If an expert prompts a result, they take up the mantle of responsibility I describe above. OpenAI not only refuses to take their responsibility, but they prevent their own customers from doing so.

October 6th was not a watershed moment for mathematics. OpenAI’s droppings on GitHub are normal progress: normal for the current state of technology. Or at least normal for the available technology in 2 months from now. The only thing that is exceptional is OpenAI’s lack of shame for releasing poor quality work.

注意转载授权 · 阅读原文