Why I Stopped Training LLMs

In 2022 I spent 10 months training a 7B LLM from scratch. No fine tuning. No starting model like GPT2. Just some Python code, lots of training data, and lots of overheating GPUs. That model is still in use on our Symbient.ai platform and works great for what it was built for.

A year ago I would've told you the model was everything.

Bigger context windows. Better benchmarks. Whoever had the smartest model won.

I don't believe that anymore.

Here's the thing few people in AI want to admit: the frontier labs are converging. GPT, Claude, Gemini, the model from that space-guy, the open source stuff nipping at their heels - they're all getting really good, really fast, and the gap between them keeps shrinking. Chasing "the best model" is chasing a moving target that's basically catching up to itself.

So I stopped.

What actually matters now isn't which brain you're using. It's what you build around it. The harness. The agent. The scaffolding that takes a raw model and turns it into something that actually does your job - reliably, repeatedly, without babysitting.

A great harness makes a good model perform like a great one. A bad harness makes a great model useless. I've watched both happen. A good harness can even make my 7B model feel like it can run with the big dogs.

And here's the part that should matter to you. Whether you're a team of one or a team of ten thousand, this isn't an "AI company" problem anymore. It's a "how does my business actually run" problem. The model is a commodity. The system you build with it is your edge.

That's where I now put all my energy. Not training models. Building the thing that makes any model work .... better.