Hello everybody!
First, a fast introduction. I’m a physics and pc science instructor in France, not a Go skilled. I wrote somewhat Go code a couple of years ago for a telemetry venture involving an electrical go-kart, however that’s about it!
Not too long ago, two articles caught my consideration:
I discovered the Go staff’s place notably fascinating. Contributors can use AI, however they continue to be liable for verifying their work, and the Go staff critiques submissions line by line.
That is smart, particularly for a compiler. However it additionally made me marvel: may AI brokers assist examine the correctness of Go itself, supplied their findings are independently verifiable?
So I made a decision to experiment!
Introducing go-in-lean
go-in-lean is an experimental open-source venture impressed by julia-in-lean.
The venture is essentially developed with AI coding brokers, combining:
-
AI brokers to discover Go’s supply code, construct formal fashions, generate Lean proofs, and examine potential bugs.
-
Lean 4 to confirm properties of normal library features and compiler SSA optimizations.
-
Differential and metamorphic testing to check anticipated conduct towards precise Go implementations.
-
QEMU on my private homelab to check ARM, MIPS, RISC-V, PowerPC, s390x, and different architectures.
-
Human evaluation to validate findings earlier than reporting them upstream.
How is it completely different from julia-in-lean?
Miles Cranmer’s venture already makes use of AI brokers, formal proofs, and differential testing. I’m constructing on that methodology, however exploring a couple of further instructions:
-
Compiler internals: investigating SSA rewrite guidelines and architecture-specific optimizations, past customary library features.
-
Bit-precise reasoning: utilizing Lean’s
BitVecandbv_decideto confirm fixed-width integer operations, with express auditing of the extra proof assumptions. -
Cross-architecture testing: working QEMU-based fuzzing campaigns on my homelab, while not having bodily {hardware} for each goal structure.
The QEMU half turned out to be notably helpful. Some compiler bugs solely seem on particular architectures.
Early outcomes
The experiment has already uncovered a number of reproducible Go compiler miscompilations, together with architecture-specific points, reported upstream.
A number of examples:
In fact, proving a property of a Lean mannequin doesn’t show the correctness of the particular Go implementation. Each suspected bug have to be reproduced, and QEMU testing has its limitations.
Why am I sharing this?
I’m within the methodology as a lot because the bugs themselves.
Can AI brokers assist us examine advanced software program outdoors our major areas of experience, whereas formal strategies, reproducible testing, and human evaluation assist set up confidence within the outcomes?
I’m definitely studying lots about Go and its compiler alongside the way in which, and I’d admire suggestions from individuals who know them a lot better than I do.
Suggestions, criticism, and contributions are very welcome!
