A Proposal for the Future of Scientific Communication

We want to open a conversation about how to communicate and work on scientific results going forward, starting with a proposal for what we envision. It is now undeniable that a new age of scientific research is among us.

The recent days have been particularly relevant for this, with the formalization of Fermat’s Last Theorem, progress on the Riemann Hypothesis, the Navier–Stokes affair, and some commentary from prominent figures like Terence Tao and other Fields medalists (more on this below). These are all mathematical in nature, but we believe much of the issue of communication is shared with the natural sciences.

As models get better, there’s a strong pull toward buying your own lottery ticket and using your allocated compute to try to make a dent in big problems. We believe this is a clear sign of the immaturity of this new scientific paradigm. We’re still figuring out how to collect and share this work so others can build on it constructively.

We are however not convinced that returning to the previous paradigm is the right choice. Science is and has been the driving force of prosperity. The opportunity cost of not embracing an accelerated scientific era is therefore enormous, so we should instead focus on how to bring this about in a healthy and productive way.

As you may know, we’re building Flywheel to try to solve exactly this problem. You can think of Flywheel like a ledger for a given research project, making each step of the process explicit, in the same way git provides historical version control for software development. This makes the research process transparent, enabling faster onboarding, preservation of negative results and lowering the bar for entry. To contribute something, you may only need to direct your model toward the current known frontier of a problem. Of course, today expert judgment still makes a profound difference in how quickly you get to a result.

Below, we share a prototype of how we envision sharing results via Flywheel, with some research results we achieved using OpenAI Codex in combination with an internal harness we’re developing and an intermediate checkpoint of a math-specific model we are training.

01Strict log-concavity in Gröchenig’s Riemann hypothesis criterion

As you might have seen, the Riemann Hypothesis (RH) has received a lot of attention lately. Anthropic published the progress they made on it with Claude, accompanying it with a formalization in Lean, complemented by AxiomMath’s own formalization. We asked Codex to look at this material and essentially import the current state of the art on the RH into Flywheel, organized in a graph representing the various current directions and findings.

We then pointed a separate Codex session running in an internal harness we are developing to look at this frontier and try to push it. This Codex session had two things at its disposal: the Flywheel graph as context, and our internal math checkpoint as a tool it could call for delegating fast mathematical explorations, calculations and proof fragments.¹

After some time, the harness appeared to make progress, producing a result in a different direction from what Anthropic had embarked in. We present this progress below.

In plain terms, Xi is a standard repackaging of the zeta function. If you take its reciprocal, and inverse-Fourier-transform it, you get a real curve on the real line. Gröchenig (2020) showed that RH is equivalent to this curve being totally positive at every order: build matrices from shifted values of the curve, with the shifts increasing, and RH holds iff all their determinants are nonnegative at every matrix size and every choice of points.

Our harness proved that all the 2×2 tests pass, without assuming the Riemann hypothesis. It did so by proving that the curve is strictly log-concave: on a logarithmic scale, it bends downward everywhere. The harness also proved that the curve is smooth, symmetric and positive everywhere, with total area one. The tests for larger matrices remain unresolved by this result.

We asked the harness to validate this in Lean, which it did in full, avoiding sorry, custom axioms, and native_decide. The Lean code is available at paradigma-inc/xi-log-concavity-lean.

We accompany the above with some caveats. This is not a proof of RH. Showing the 2×2 case says nothing about 3×3 or larger. We are also not sure about claiming novelty either: our literature review found nobody who proved this exact statement (the closest work is Gröchenig himself, plus a few papers on concavity of the classical theta kernel, which is a different object), but we are aware that we may be missing something a specialist could otherwise point out.

And this is exactly what Flywheel enables: if we have indeed made a mistake, anyone can branch off of our work and propose a correction or a rebuttal, and our collective understanding of the problem improves.

We share the entirety of our work here in the form of a Flywheel graph: proof, evidence and history on Flywheel.²

We think this could be a future analogue to the way people currently submit PDFs to peer reviewers for approval and verification, except the reviewers are whoever wants to look. If you think there’s a bug, a gap, or a paper from 1994 that already did this, you can post your findings or take to the graph, and the world can see it. If you’d rather contribute than review, install Flywheel, point your model at the graph and ask it to help you push the frontier.

02But you are just polluting the commons like this

We get this objection and take it seriously. Here is why we think it might be misguided.

Publishing a Flywheel graph is not publishing a paper. A paper is a claim of a finished result, packaged for a reader with limited time, competing for that reader’s attention with every other paper. A graph is a record of work in progress, with the dead ends still attached, offered for review. The failure mode people worry about, a flood of half-baked claims that nobody has time to check, is a failure of the paper medium: the format hides the process and the reviewer has to reconstruct it by hand.

Making the process transparent changes the cost of checking, especially in the age of increasingly automated menial work. You can see the step where the argument turns, see which lemmas are formalized and which aren’t, see what the model was given versus what it found. The review starts from the actual state of the work, unbiased by the author’s recounting of it.

Review is still needed. We are not convinced it has to be human, or at least not primarily. We’re still figuring out how to incentivize it, but the shape seems clear enough: other people’s models spend compute trying to break the result, and the graph records what held. Letters became peer review, peer review became OpenReview. We are trying to figure out the next step.

Finally, there’s an open question about understanding. Maybe every result of this kind should come with a distillation step, where a human writes down what it means and why it matters, perhaps like the blog post you’re reading right now. Maybe not, and we simply move up a level of abstraction and let people engage with the frontier at whatever resolution they need. We don’t know yet.

So: there are mediums where pushing out every partial result is unproductive. PDFs and human-heavy peer review are among them. There are other mediums where it’s exactly what you want, because the medium is built to absorb it. It’s time to build better media.

More from us soon.