Why your company doesn't need a custom LLM (yet)

The Hype

Every week, a CEO reads an article about a competitor "building their own AI" and picks up the phone. The conversation usually starts the same way: "We need our own ChatGPT."

It's an understandable reaction. AI is moving fast, the stakes feel high, and training a custom model sounds like the kind of decisive, future-proof investment that separates leaders from laggards. But in most cases, it's the wrong call — and an expensive one.

Here's the reality: training a custom large language model from scratch costs between $2M and $50M, takes 6-18 months, and requires a team of ML engineers who are genuinely hard to hire. And after all of that, you'll have a model that is almost certainly worse at general reasoning than GPT-4o or Claude, which you could have been using on day one.

What You Actually Need

The confusion comes from conflating three very different things:

• —Training a model — teaching a neural network from scratch on your data
• —Fine-tuning a model — adapting an existing model to your domain
• —Prompting and orchestrating a model — building intelligent workflows on top of existing APIs

For the vast majority of business use cases, the third option delivers 80-90% of the value at 1-5% of the cost and complexity.

A legal firm that wants to extract key clauses from NDAs doesn't need a custom model. They need a well-designed prompt, the right context window, and a reliable pipeline. A logistics company that wants to auto-generate dispatch summaries doesn't need fine-tuning. They need their data fed to the model in the right format.

When Fine-Tuning Actually Makes Sense

There are legitimate cases where fine-tuning on your proprietary data delivers real lift. Consider it when:

• —You have a narrow, high-volume task with a consistent input/output format (e.g., classifying support tickets into 12 specific categories)
• —Your domain has highly specialized vocabulary that general models struggle with (medical device regulatory filings, financial derivatives contracts)
• —You need sub-100ms latency and can't afford full API call overhead
• —You're running millions of inferences per month and the economics of a smaller, tuned model beat API costs

Fine-tuning on GPT-4o, Claude, or Gemini is meaningfully different from training from scratch. It's measured in days and thousands of dollars, not months and millions. It's worth evaluating once you've proven the use case with the base model first.

The "Data Moat" Fallacy

The other common argument for custom models is the data moat: "Our proprietary data is our competitive advantage, so we should train on it."

This is true in principle and mostly irrelevant in practice — especially for companies under $100M in revenue.

Data moats matter when you're building a product where AI inference quality is the core competitive differentiator (think Midjourney, Perplexity, or Cursor). For an operations team trying to automate invoice processing or a sales team trying to improve outreach quality, the competitive advantage isn't in the model — it's in the workflow, the integrations, and the institutional knowledge encoded in the prompts.

That's something a good consulting engagement can help you build in weeks, not years.

Where to Start Instead

If you're a business leader trying to get real value from AI in the next 90 days, the playbook is straightforward:

• —Identify your highest-volume, most repetitive knowledge work — things your team does dozens of times a day that follow a predictable pattern
• —Pick one process and build a working prototype using the best available API model
• —Measure the output quality against human baseline — you'll usually find it's good enough for 70-80% of cases, and that's enough to matter
• —Build the human-in-the-loop review process for the remaining 20-30% — this is where you capture edge cases and improve prompts over time

The companies winning with AI right now aren't the ones with the biggest models. They're the ones with the most thoughtful implementations — where the AI fits cleanly into the existing workflow, the outputs are trusted, and the team actually uses it.

That's what we build at Ether Labs. Not custom models — operational AI systems that work.