Yuan-Tian's New Firm Releases Flint-40B-A2B, Built Entirely by AI

Using AI research systems to develop AI models.

Synced Machine Editorial Team

Since the official announcement in May of this year, Recursive_SI, a new company co-founded by Yuandong Tian, has finally unveiled its first model achievement!

Flint-40B-A2B is a fast agentic coding model designed for local hardware execution, built entirely with the assistance of Eureka Machine, Recursive's proprietary automated research system.

As a Mixture-of-Experts (MoE) model, Flint has a total parameter count of 40 billion, activating only 2.5 billion parameters per generated token. With minimal human staff and significant involvement from AI agents, the model was developed from scratch in less than four months.

On agentic coding benchmarks such as SWE-Bench Pro v2, Terminal-Bench 2.1, and SWE-Bench Verified, Flint outperforms other MoE models with total parameter counts below 50 billion. Its coding capabilities rival those of dense models with 25–40 billion parameters, while running more than 5 times faster.

The inference engine has also been specifically optimized. With shorter contexts, Flint achieves generation speeds exceeding 125 tokens per second on a MacBook Pro equipped with an M4 Pro chip, and over 150 tokens per second on an NVIDIA RTX Spark with 64GB of memory. This represents up to a 2x speed improvement compared to similar models.

Flint pushes the balance between speed and capability for local models to a new level on the MacBook Pro M4 Pro.

"Dude... Recursive just released a coding model that runs locally on a MacBook Pro at speeds up to 125 tokens per second!"

Developing Flint also served as a practical test for Eureka Machine, the large-scale automated AI research and development system. This system assisted Recursive in developing nearly every stage of the Flint model construction pipeline, including data processing, training strategies, infrastructure, and inference optimization.

Currently, Flint's model weights are open-sourced on Hugging Face under the Apache 2.0 license. After downloading the model, users can run it locally, keeping data on their own devices, or fine-tune it according to specific application requirements.

• Hugging Face URL: https://huggingface.co/RecursiveSI/flint
• rec project URL: https://github.com/recursive-oss/rec

Flint is compatible with various tools and frameworks, including llama.cpp, Ollama, Unsloth, LM Studio, vLLM, and SGLang. Additionally, the open-source lightweight inference engine rec has been specifically optimized for running Flint on M4 Pro and M5 Pro/Max chips, offering a simple configuration process to help users get started quickly.

One developer measured Recursive Flint's generation speed at 114–132 tokens per second on a MacBook Pro M5 Max with 64GB of memory.

Image source: @WangCankun

Another developer successfully ran this 40-billion-parameter coding model on an RTX 3060 with only 12GB of VRAM. It supports a 256K context window and achieves a generation speed of approximately 46 tokens/s.

Local Agentic Coding Reaches SOTA

In agentic coding benchmarks, Flint is the top-performing MoE model among those with fewer than 50 billion total parameters. Its performance rivals that of leading dense models of similar parameter scale, while running significantly faster.

Specifically, Flint achieves the highest scores on IFBench and IFEval, outperforming other listed sparse MoE models on Terminal-Bench 2.1 and SWE-bench Pro v2. It also remains highly competitive on SWE-bench Verified and LiveCodeBench v6. However, some dense models still achieve higher scores in certain coding benchmarks.

Flint is well-suited as a daily coding model for executing tasks locally, eliminating the need to send data to remote servers. For complex tasks, it is recommended to break work down into smaller subtasks and provide clear instructions. While Flint already possesses considerable capabilities, compared to the large closed-source coding models familiar to many users, it may require more guidance during use.

Flint supports a context window of up to 512K tokens. For coding and other tasks, Recursive recommends keeping the context length within 256K tokens to achieve better results.

To ensure Flint's computational efficiency translates into a faster coding experience, Recursive optimized the entire inference pipeline, improving everything from how model weights are represented to the low-level kernels executing calculations on local hardware.

Faster Performance on Local AI Devices

Flint brings strong coding capabilities to users' own hardware. You can use it in offline environments, keeping data on your computer without paying for API calls.

With its optimized inference engine, Flint runs up to 2x faster than comparable models on Apple Silicon and NVIDIA RTX Spark. Compared to dense models with similar coding benchmark scores and parameter counts between 25B and 40B, Flint is over 5x faster.

Automated R&D Behind Flint

Currently, Eureka Machine can automatically complete an entire research workflow: proposing ideas, writing implementations, running tests, and deciding what to try next based on test results. It can explore multiple technical paths simultaneously, leverage results from existing experiments to make progress, and combine high-performing solutions from different experiments.

Recursive has previously detailed how this system works and its performance in initial test tasks.

For more details: https://recursive.com/updates/first-steps-toward-automated-ai-research

The entire development process of Flint utilized Eureka Machine. The system contributed to R&D across data, optimizers, training schemes, infrastructure, and inference performance. The improvements discovered by Eureka Machine helped a small team build Flint from scratch, enabling the model to reach state-of-the-art levels on target benchmarks in less than four months.

Flint's local inference speed also reflects how these improvements compound across different stages of model development.

Model Architecture

The first step was finding a model architecture better suited for local hardware, aiming to activate fewer parameters per token to improve generation speed.

Starting from a baseline model with 35 billion total parameters and 3 billion activated parameters per token, Eureka Machine experimented with various expert configurations and network widths. After multiple rounds of exploration, the system predicted that the final chosen Flint architecture would offer a better balance between inference speed and coding capability among the tested configurations: 40 billion total parameters with only 2.5 billion activated per token.

Quantization

After model training was complete, Eureka Machine continued searching for suitable quantization schemes to reduce Flint's memory footprint while preserving model capabilities as much as possible.

Unlike common practices, the system quantized the output layer to 4 bits as well. Since Flint activates only 2.5 billion parameters per token, the output layer constitutes a significant proportion of the weights read during each token generation. Quantizing this layer further reduces inference overhead but may impact model accuracy.

To compensate for the loss in precision, the system introduced an offline distillation step: it first pre-samples responses from an automatically discovered core sample set, then trains the quantized model using these responses, with the 16-bit model serving as the teacher.

Compared to previous Quantization-Aware Training (QAT) baselines, this approach reduced the number of training steps by nearly half. The resulting NVFP4 model scores within 0.7 points of the 16-bit model across various benchmarks.

Inference Kernels

Next, Eureka Machine further optimized how the quantized models run on Apple Silicon and NVIDIA RTX Spark.

It developed specialized kernels for low-bit projection computations, MoE operations, attention calculations, and prompt processing, tuning them according to the model architecture and hardware characteristics.

On M4 and M5 chips, the system also developed a large fused kernel based on Metal that consolidates the entire forward pass of the decoding stage into a single kernel execution, reducing overhead during computation.

Flint’s current inference speed is the result of continuous improvements made by Eureka Machine across different stages. Recursive applied similar methods in data processing, model training, and infrastructure, combining effective solutions found in various experiments to further enhance the final model's performance.

Next Steps

Recursive aims to build superintelligence capable of recursive self-improvement, automating the knowledge discovery process. Currently starting with model development, they hope to establish a research workflow that continuously improves with each generation of models produced.

Flint marks the beginning of this effort. A small team built this model from scratch in less than four months. Eureka Machine was responsible for proposing ideas, attempting implementations, and validating whether these solutions were effective at every stage of model development.

Recursive believes that more capable models will enable Eureka Machine to become a better researcher, helping to develop next-generation models faster and achieving stronger capabilities at the same cost.

Going forward, Recursive will continue developing more powerful local models and the infrastructure required to run them, extending this work to broader areas of AI research and development.

Blog URL: https://recursive.com/updates/introducing-recursive-flint

© THE END

Please contact this official account for authorization before reprinting.

For submissions or media inquiries: [email protected]