Rendered at 20:50:02 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
extr 1 days ago [-]
I would be really curious to hear from devs at Databricks what the experience of development is like internally. I work at a small startup with essentially unlimited AI spend budget - the entire point is that I should be turning to it at every opportunity since our human labor is so expensive relative to tokens. So generally it's like:
- Spend most time prioritizing/discussing what to do.
- Once that's agreed, use Fable 5 High + 5.6 Sol XHigh come up with a design + plan. Agree on the high level plan. (Usually this just comes down to choosing where the change belongs on the spectrum between minimal patch <-> full redesign)
- Use Opus 5 or Sol Med to execute
- Auto-fix bugs and CI until green + thermonuclear review skill x3.
- Manual interrogation of change/nits
- Come up with QA plan and have Codex Computer Use execute on it
- Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc)
I probably spend like $80 a day at least but I produce the output of 3 or 4 2022 engineers and probably at better quality. So it's easily worth it. Would I save money by switching to GLM 5.2 and such...perhaps? IDK. At our scale it's not worth the time spent building the eval harness to actually understand the performance tradeoff.
pizza234 1 days ago [-]
In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered.
When I personally want tight code, I have to spend a considerable amount of time adjusting it manually:
- It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating
- I need to consider whether there are solutions based on higher-level assumptions, that AIs typically miss
- I need to check whether there are off-the-shelf solutions - AIs like to reinvent the wheel
IMO, software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
It took me a few seconds of deliberating if The Homer was a reference to baseball or "The Odyssey" and then realized there was a footnote
> software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
Agreed.
GoofGarage 20 hours ago [-]
I believe “The Homer” is a reference to Season 2, Episode 15 of The Simpsons, “Oh Brother, Where Art Thou?”
Homer reunites with his long lost brother, who runs car company Powell Motors. Homer is ultimately tasked by his brother with helping to design a car for the “average man” that ultimately bankrupts the company for being wildly overengineered and costing too much ($82,000 in 1991-money).
I largely agree, but when this point is brought up I always feel the need to go one step further:
For the vast majority of software out there, this quality gap simply doesn't matter. Not everything is big tech, needing internet scale. Most software built by developers at non-tech companies are simple line-of-business apps used by a handful to a few dozen or maybe a few hundred people to do a thing a bit better than the Excel spreadsheet that was used before.
And AI has essentially automated the development of this kind of software. Or rather, it has dramatically increased the ability of skilled developers to produce larger amounts of these kinds of systems.
Excel hardest hit.
zobzu 5 hours ago [-]
i don't think people writing this understand how computers work. there is no "Internet scale hard" problem.
its reliability, safety, efficiency problems. these problems exist at any scale.
ai sucks at it.
redhale 1 hours ago [-]
> its reliability, safety, efficiency problems. these problems exist at any scale
Respectfully, I disagree. When you have 5 users using an app sandboxed within the corporate firewall that just does some CRUD and basic workflows, these problems you're calling out really just don't matter much. Some do some of the time, but not much.
Obviously not all software falls into this category. But so, so much does. Do you know how many thousands of tiny applications are maintained by big all non-big-tech corporations? I've seen many of them up close -- the code is usually complete garbage (seriously, it's bad; and alas, most was written before the age of AI coding agents), and yet, the business continues to operate and use the software.
myegorov 3 hours ago [-]
> For the vast majority of software out there, this quality gap simply doesn't matter.
You don't mean the majority of paid for software can behave non-deterministically? That's the value add of the human in the loop.
2 hours ago [-]
pianopatrick 23 hours ago [-]
If software has become a mass-produced commodity then seems to me the software business will become a much more finance focused business
You will really have to weigh the cost of making the software against the expected revenue.
roncesvalles 22 hours ago [-]
Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.
In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change.
Even before LLMs, you could've commissioned a half-assed clone of any app you wanted from a 3rd world consultancy for a few thousand dollars. LLMs are basically Bangalore-as-API.
pianopatrick 20 hours ago [-]
I think part of the reason software was winner take most was the difficulty of making software.
I remember hearing a story that in the past movies were so technically difficult to make that any movie that got made had a good chance to be a profitable hit. But as movies got cheaper to make, more movies got made. Nowadays movie studio execs have to really calculate out the audience and expected revenue for any new movie and balance that against the budget and the cost of the studio's failed movies.
I think a similar dynamic may happen in software
pcox 17 hours ago [-]
> I think part of the reason software was winner take most was the difficulty of making software.
That might be part of it, but I think it also has to do with the reality of replicating and scaling. Hardware or physical goods simply don’t scale like digital goods. There can be hundreds of knock-off physical products that have lower quality and lower cost but serve 90% of the same purpose, because physical capacity for raw materials, construction, labor, shipping, etc. have scaling limits in each market and economy. Digital goods are just so much easier to replicate and scale, so it often doesn’t make sense to buy software at lower quality and lower price if it doesn’t do most of the job. There are still limits of course, and different from physical goods, but I think this is a key reason why software is seen as winner-take-all.
xbmcuser 9 hours ago [-]
I disagree the worth of software will become a lot less as LLM take hold and more people outside software devs know how to use them properly. I am the default IT manager for our small office as I am the guy most interested in tech and in the last 2-3 years I am getting llm to write python scripts for me to do work that would have cost $100s of dollars. Currently you need to pay $100-1000s for a software with 100s of features while you might actually need 1 or 2 of the features which you can get the llm to write for you.
As the LLM's keep getting better at writing software actual coding skills required will decrease which will result in a huge influx of software crashing prices. You can see this in already in open source projects in the last 6 months to a year Jellyfin had a huge influx of custom clients made by different people for fixing their own itches about the official software.
pianopatrick 16 hours ago [-]
Yes, but there are other digital goods, like music, movies and books which are not quite as hard to make as software. In those you have a hit driven market dynamic with lots of niches instead of winner take all.
I.e. because software was hard to make and complex to copy you would tend to have "natural monopolies" that were hard to compete with. Who wants to try to build a new desktop OS to compete with Windows? Or a web browser from scratch? Or a new search engine? Etc.
Those and other pieces of software were complex and hard to make. The cost to copy and compete was very high. So one winner took most because that winner was the company who could figure that software out.
But as we can see with Kimi Work and other such things, software is now much easier to copy. Let's say it took $1 billion to make a copycat piece of software with people but now takes $100 million or $10 million with AI. Suddenly a copy and compete tactic makes much more sense than before
For example, with AI it might make financial sense to build a Chinese Native OS instead of Windows. Similarly for Russia, Iran, the EU, and a whole bunch of other places. All of a sudden, Windows might not be the winner take most OS, we might have lots of Operating Systems, with smaller markets and lower profits, which require much more careful financial analysis to stay profitable.
This would be just like Movies, TV Shows, Books or Music. When something works, people relentlessly copy it and different regions put their own spin on the idea. After Iron Man succeeded we had so many super hero movies. Etc. So there is not really a winner take most dynamic in these other digital products. Software may be moving that way
hyperadvanced 20 hours ago [-]
That honestly sounds a lot healthier than “just ship what the CTO/Product team wants” with as much hand waving as is necessary to very roughly estimate ROI and then pray it hits with the market. In anything that’s not a startup operating in a new industry, the “old way” is a hard way to run a business
sevenzero 10 hours ago [-]
>Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.
This could not be further from the truth. It might be true if the software works without supervisors in a B2C scenario where everything is self explanatory and nothing can go wrong, movie streaming services for example, and even these have tons of variants and multiple big players racking up billions in market share. So even there "winner-takes-all" does not apply. It gets even worse in B2B software, as there is SO MUCH software thats commonly used and needs tons of supervising by experienced devs. If we had a "winner-takes-all" market a lot more software would already be dead. And a lot of companies would have way less options when adopting new software for something.
roncesvalles 4 hours ago [-]
"Winner-takes-all" in the software context is never meant literally. It's more like, an oligarchy of a few winners will emerge out of hundreds of contenders.
It's not like restaurants where for example you can open a "Larry's Pizzeria" and still build a valuable business despite a Domino's nearby.
oblio 21 hours ago [-]
> Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.
Fairly sure software dev was always an iceberg. Most software and most software devs aren't working on horizontal software, but on vertical software, in cost centers. Sales for that kind of software don't scale as much.
roncesvalles 21 hours ago [-]
Most vertical software is just horizontal software glued together
oblio 7 hours ago [-]
Horizontal open source modules... And making the horizontal modules doesn't employ many people, both because of FOSS and also because you only need so many reusable modules.
When we were making brick houses we needed a few brick factories with thousands of people for million of bricklayers.
frevib 1 days ago [-]
No surprise, LLM companies optimize for waste. More tokens, and more prompts means more revenue. Reminds of Google’s Prabhakar Raghavan story: deliberately making search worse [1]
Or, more likely, it's that concise code requires a much deeper, wholistic, understanding that these models just are capable of yet.
Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade.
I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.
lilbigdoot 16 hours ago [-]
Personal anecdote: I spent a few days hacking on my compiler to remove 1k lines of code (about 15% of total code) while preserving behavior
I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases
I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
frevib 4 hours ago [-]
> I don't think this clear lack of ability is some conspiracy.
Maybe currently not. But we will never be able to know, as models are undeterministic and benchmarks are kind of scams. When you cannot prove that something gets worse, rest assured companies will to it.
bonoboTP 4 hours ago [-]
Bullshit. You can be cynical, it's fine but this is just nonsense. People have so much demand for coding that there is no need to make it waste tokens. People are eager to implement more features, do more testing, more platforms, more file formats, bla bla. There is absolutely no incentive to waste tokens. They can barely serve the demanded tokens anyway. There is in fact incentive to save tokens, so subsidized subscriptions don't consume as much.
sandworm101 5 hours ago [-]
>> In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating
Which ones don't?
I used to play Wing Commander II. It was 30mb. Today, the software to adjust the colored lights on my keyboard requires literally a thousand times as much memory.
eitally 22 hours ago [-]
As a business user, the same thing is true for non-code documents. The biggest exertion is reducing the excessive slop down to concise, clear points.
sublinear 23 hours ago [-]
> software production has become a mass-produced commodity
For who?
The public? The public has never liked buying software at any price.
Businesses? Businesses need higher quality software when it's relevant to their core competencies, so they hire people instead. Buying competing SaaS or depending too much on AI is throwing the baby out with the bathwater.
extr 1 days ago [-]
This was more true a few months ago but Fable has improved the situation considerably.
Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be better abstracted when that's actually needed.
dieselgate 1 days ago [-]
> The average SWE costs $200/hr
This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI.
How is this not true? Taking a Senior SWE @ ~$200K, even just the base salary cost / 2080 working hours is $100/hr. Fully loaded employer cost + accounting for non-coding time gets you to upper 100s easily.
Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so.
Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.
anon73044 1 days ago [-]
Company time != Pay rate, if you're working somewhere that's publicly traded check out "revenue per employee" metrics sometime.
pdhborges 24 hours ago [-]
I have a hard time believe their time is worth less than $75/hr or so.
In many places in Europe it is.
rustystump 20 hours ago [-]
It is. And the quality is on par with any us eng. People here forget that the big comp packages are a minority even in the US. The cost tho is much higher than just salary.
Foobar8568 23 hours ago [-]
Western Europe is mostly consultancy, and the rate paid by client is usually higher, and doesn't matter if it's eastern Europe, Portugal or even India.
jknoepfler 22 hours ago [-]
I hire contractors for a large enterprise in the US. The going rate is typically $85-$100/hr for a senior dev, depending on specialization. Lead-level maybe $120 for the right skill set.
17 hours ago [-]
sublinear 22 hours ago [-]
Of course, the SWEs making that much (over 200k) are not representative of the broader field. That's the point.
Pay hits a ceiling, and that ceiling is moving lower regardless of experience. That has nothing to do with AI, but what the market will bear. Hiring counts of humans must increase no matter what. Moving some of the spend to AI reduces the risk of hiring less qualified employees they might have rejected a decade ago.
Wages at the top end are stagnating to subsidize this. That's undeniable.
copperx 1 days ago [-]
$15, where we're going.
fragmede 21 hours ago [-]
Federal minimum wage is $7.25 per hour
russellbeattie 24 hours ago [-]
An MBA's rule of thumb is that a full time employee's hourly cost to a business is at least 1.5x to 2x times their salary depending on employer taxes, benefits, offices, travel, training, hardware, perks, etc.
BobbyJo 1 days ago [-]
Minimalist code is necessary to keep AI agents working well for longer than a month on a system IME. At a certain point, their own machinations overwhelm them and they both slow down, and make worse and worse decisions.
gracehenry45 1 days ago [-]
[dead]
j-bos 1 days ago [-]
> The average SWE costs $200/hr.
And this is how I find out I'm woefully underpaid.
junto 7 minutes ago [-]
You might be paid $100/hr but it costs the employee $200/hr. It’s worth mentioning because cost to the company is not just your salary, but the taxes, pension and contributions on the other side, plus a variety of other things that the employee doesn’t consider.
jacquesm 24 hours ago [-]
Whatever you are making this year as SWE you'll be making less next year if the current trend in improvement of AI coding aids is going to be sustained. Think about it : programmers used to derive a lot of their value from the fact that it was a hard skill to acquire. My kids can now 'vibe code' stuff faster (and better looking) than what I could come up with as the beginnings of a design plan. And then I still need to implement it.
alfalfasprout 23 hours ago [-]
There's a massive difference between your kids vibe coding something and an engineer using AI to implement something. If you're unable to discern the difference, that's something to reflect on :)
throw-the-towel 23 hours ago [-]
It doesn't matter if GP is able to discern the difference, it matters if your CEO is forced to care about the difference.
ipaddr 16 hours ago [-]
CEOs will always want someone who knows to implement so we're safe from kids vibe coding their way in but in a short while it becomes AI who knows who is managing less expensive AI.
lelanthran 12 hours ago [-]
If he is unable to discern a difference, then a paying customer almost certainly won't.
We're entering a market of lemons in software. It's going to be interesting indeed.
icedchai 23 hours ago [-]
That "cost" includes all the overhead provided by the company: benefits, rent for offices, utilities, equipment, etc. The average SWE is not taking home anything close to that, outside of Silicon Valley and a few other limited areas.
sejje 24 hours ago [-]
Also, you can probably think about it like this:
"Will I benefit from this code being minimalist before [date]", where [date] is whenever you think the agent will be good enough to come back and make the corrections you would make today.
Terr_ 17 hours ago [-]
I'd caution that some corrections become harder to make over time, rather than easier. A bad architecture now can become much harder to fix once other things have grown up around it.
Even as a self-contained unit, you can't step in the same river twice, and on [date] some important details may have seriously faded, both in terms of text that can be mined and also in terms of human "why did we do that" and "what was the reason we did it this way and not that way" etc.
slopinthebag 23 hours ago [-]
The average SWE makes $400k a year? Are you being serious?
fragmede 21 hours ago [-]
Costs, not makes. That includes
employer taxes, benefits, offices, travel, training, hardware, any perks.
reqo 1 days ago [-]
IME this works until it does not. This approach works well at the beginning of a greenfield project, but at the same time because it is so easy to add features, you will likely ship something that is way too over engineered. And that complexity will not amortize over next increments and will more likely lead to the entire project being a black box only fully understood by AI. However a more careful use of AI for targeted surgical changes is far more ”productive” in the long term IMO.
jarboot 18 hours ago [-]
I disagree, the approach works well in a legacy project, since there are structures and standards that already exist, which them model can draw from (if you aren't more explicit about it in AGENTS.md)
extr 1 days ago [-]
Disagree. I operate this way inside a multi-million line legacy codebase.
nujabe 1 days ago [-]
> I work at a small startup
How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing
darkwater 1 days ago [-]
> How does a “small startup” end up with a multi million line “legacy” codebase?
Easy! The output of 6 months ago Opus! Which seemed so wonderful at the time.
champagnepapi 23 hours ago [-]
This! I don't think folks understand how easy it is to go from greenfield to brownfield with these tools, esp if your organization is only valuing velocity. Meaning your doing full agentic development on large features, barely reviewing any code, and shipping without much refinement. It's insane, but this appears to be the status quo in SF startups.
extr 1 days ago [-]
Have you worked at many startups?
Karrot_Kream 1 days ago [-]
Something isn't clear about the size of your codebase here and the level of reliability your customers expect, as a reader of your comments. Clarity there will help.
My observation has been:
- Initial greenfield work by an LLM is fast and very effective with minimal or no human oversight.
- Subsequent work ends up being over engineered and very verbose. Assumptions are made that aren't suited to the problem at hand (for example I find Fable is extremely regex happy where structured data would work much better from a readability perspective.)
- Once code bloats beyond a certain point due to unguided LLM usage, complexity is high enough that only LLMs can operate on the codebase with any economical amount of time.
- Rinse repeat and your code ends up unclear about any state that's not explicitly being tested and verified in QA loops
For some of our products this has been fine, for others it's been problematic. An understanding of your size and reliability requirements will help make the conversation more productive.
extr 1 days ago [-]
> unguided LLM usage
Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage?
> any state that's not explicitly being tested and verified in QA loops
As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?
Karrot_Kream 24 hours ago [-]
> Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage?
You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided (e.g. a bit of oversight, not the kind of thing that heavy code reviews used to involve pre-agents.) Feel free to add clarity on your actual usage loop.
> As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?
In my experience, most engineers are quite good at reasoning about code behavior for non-QAed code paths. Obviously things fall through the cracks. But I've been in the ground floor of plenty of Big Techs in their early stages before agents and, yes, a lot of initial development had spotty test coverage and yet most of the engineers had good mental models of what was happening. It used to be a very valuable skill to wrap your head around a torrid piece of code with few or no tests but was nonetheless a core piece of your application. Conversely, agentic development can bring cognitive debt [1].
===
This isn't a fight. We aren't sparring over what's right and wrong. I'm just curious how other people use agents in their work as someone who is also now in a startup that uses LLM agents heavily and has no limitations on spend.
> You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided
I feel like this statement betrays your lack of advanced experience coding with LLMs.
OP's elaboration of the steps they are going through (planning, agreeing on plan, getting one LLM to draft execution plan, approving it, then executing with a separate LLM, then reviewing/testing) made it super obvious to me that they are guiding their LLMs quite considerably as part of their work.
Anyone making blanket statements about LLMs producing garbage is just telling on themselves about not having proper SDLC practices in place.
Karrot_Kream 23 hours ago [-]
Planning, agreeing on a plan, separating planning and implementation LLM, using separate review LLMs, these are all table stakes. This isn't "guidance" if you're getting paid to write software. If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do.
I find for my own work that I need to read the diff the LLM produces then offer feedback on the diff in its own loop before I am satisfied, and this is after all the unattended QA steps through Codex Computer or Claude MCPs happen. Then auto reviewers come in and then reviewers come in. Of course, at our stage, we rarely have this luxury and it's only reserved for the very core of our codebase.
This is still much less guidance than we used to do for code before agents became popular. Even at Series A companies, before agents, we used to socialize tech specs, get buy-in from multiple engineers, create test plans, etc etc.
> Anyone making blanket statements about LLMs producing garbage is just telling on themselves about not having proper SDLC practices in place.
> I feel like this statement betrays your lack of advanced experience coding with LLMs.
Are we in school debate club? I don't know what's going on lol, I'm just curious how people are using LLMs! Is it just that irresistable to take a cheap shot at each other?
sbarre 21 hours ago [-]
> Are we in school debate club?
Not that I know of but that's the conclusion I drew from your statement.
It's not a cheap shot unless you took it personally?
I suppose I could have said "the fact that OP's explanation of how they work did not lead you to conclude they were in fact guiding their LLM usage quite a bit tells me that perhaps you have not been working with LLMs in any advanced capacity".
For the SDLC comment I admit it was a broader statement (based on observing people generalizing that "LLMs produce bad outputs") and not specifically aimed at you, and I didn't make that clear, so my bad.
extr 22 hours ago [-]
> "unguided" means "I typed a prompt into claude code and waited yolo"
Yes, this is literally what that means.
skinfaxi 19 hours ago [-]
> If you think "unguided" means "I typed a prompt into claude code and waited yolo" I don't know what to say but, you have a very different idea of what professionals do than I do.
What exactly does "unguided" mean to you, then?
Karrot_Kream 16 hours ago [-]
Not having human input in the loop, i.e. allowing agents to act without guidance. I understand the idea of having agents guide agents, but really how much do we gain when Sol scolds Fable?
sbarre 7 hours ago [-]
> how much do we gain when Sol scolds Fable?
You might be surprised at how much benefit you get from the model council[0] pattern.
The common pattern I've seen is "build with Claude, review with GPT", or vice-versa.
I use it. We have more workstreams than people and so we have to rely on this pattern for most of our new products, but I still find the code doesn't meet my bar. To me it produces passable but not good, maintainable code. Good enough for our experimental products with the cognitive debt associated with heavy agentic development. We take this calculated tradeoff of velocity for cognitive debt and middling quality because we're a startup and we feel that velocity matters more than correctness, but that is just a bet as the nature of startups go.
Foobar8568 23 hours ago [-]
AI is an accelerate tool for any organizations, management thinks it'll solve their organization issue because it accelerates it. Most often, it accelerates toward a wall.
Design is too expensive, we do agile.
QA too expensive, we fire all of them, and claim devops is the now, which allows us to fire the Ops team too, 100% ownership from deisng to ops on devs.
One person with an agent can replace all these teams. Yeah mo profits.
nujabe 1 days ago [-]
No, but not relevant.
What is the point of working at a startup if you’re dealing with millions of lines of legacy code ? Isn’t the whole point of startups to create & innovate with a clean slate and modern tools?
extr 1 days ago [-]
No, actually. The point is to build a profitable business.
sarchertech 23 hours ago [-]
How long has your startup been around? I’ve worked at plenty of startups over the past 20 years. Including one that was still calling themselves a startup 10 years out. The org I work at now was a startup before my tech giant employer acquired them. We have a very bloated and very profitable 8 year old codebase that is barely 500k LOC.
I’ve never seen a startup with a multi million line legacy codebase.
dgellow 22 hours ago [-]
They may have forked something
sarchertech 22 hours ago [-]
Definitely possible, but up thread they wrote:
>”Have you worked at many startups?”
In response to a question about a legacy codebase at a startup. That implies that they think whatever they are doing is common. And forking a multi million line codebase and heavily developing it isn’t common for startups.
dgellow 11 hours ago [-]
You’re right, very confusing thread
fatata123 21 hours ago [-]
[dead]
chris_money202 1 days ago [-]
I don't know if that's the whole point, but I agree with the sentiment, why would a startup be working in legacy code and where would that code come from if this is truly the start of something.
OP might just be working at a small software company or for one that broke from a bigger one and is now "startup" like?
icedchai 23 hours ago [-]
You'd be surprised. I met a guy last week who was proud to tell me he had vibe coded an almost 2 million line code base. The app did not sound that complicated, so I'm assuming it's full of copy-pasta flavored slop.
lilbigdoot 15 hours ago [-]
Meanwhile I'm so proud that I'm keeping my hobby project under 10k lines and spend time trying to delete code for my own sanity
gamblor956 24 hours ago [-]
"startup" and "legacy codebase" are diametrically opposed concepts.
And if you're saying (based on your other comments) that a 6 month window is enough to create a legacy codebase...that indicates a serious lack of experience or understanding as to what a legacy codebase is, or why they exist.
sbarre 23 hours ago [-]
Man, so many people in this thread just arguing pointless semantics, making weirdo absolutist (and incorrect) statements.
Accept that other people may ascribe different meanings/interpretations to words than you, and that if your reading of their statement doesn't make sense to you, perhaps you are simply reading it wrong.
Trying to hold someone else to your definition of words suits what purpose exactly? Are you just trying to "win" ?
extr 22 hours ago [-]
Yes lol. Of all things people are getting on me for it's the number of LoC x Years In Business of this startup. I don't fucking know, I didn't start the company and I wasn't here for several of those industrious years. Looking now it looks like we have slightly fewer LoC than that, I was counting some of the generated stuff.
But who cares? The point is any codebase over a few years old with lots of customers and a big surface area has lots of code, much of it "legacy" from the standpoint of a guy in 2026.
rustystump 20 hours ago [-]
You are wrong and they are wrong but that is ok.
loose-cannon 23 hours ago [-]
I agree with your larger point. Though I think it's pretty natural to wonder how the poster ended up with a multi million line codebase.
sbarre 20 hours ago [-]
2M lines of code is 15 people committing ~26k lines of code per year (~100 lines per working day) for 5 years.
15 people is a pretty small startup, what if this is a 50-person startup?
Doesn't seem like that much to me, depending on what you're building and the size of your team.
sarchertech 11 hours ago [-]
I’ve worked at many startups and consulted at many more over the last 20 years. I’ve never seen 2 million lines of code happen that fast at a small startup.
15 devs putting out 400k LOC a year into the same codebase is not normal at all. I’ve never seen anything close to that kind of rate of growth across that number of people.
And 50 engineers is not even remotely close to a small startup. I have worked at a startup that had 50 engineers after a few years, but it was a multi billion dollar unicorn.
rustystump 20 hours ago [-]
This is an ultra cop out. There are standards in language that are not all “left means right for me so you cannot assume when i say right it is right and not left”
This whole thread around loc is depressing. It speaks volumes of some peoples inexperience working on actual legacy code. Legacy code is not just age or size but that the technical foundation is dated in a fundamental way. A giant monolith running on a now defunk framework using a database only one guy in canada knows about.
Case and point in my day job. The org that owns XMM development does not know how to recover a physical bench that is bricked because everyone who knew how has left. So now they just use simulators…
Interestingly, AI figured out some of this pretty easily for me. But the org has the exact same AI as i do. At the same time another org is close to a year into a greenfield rewrite that has been developed via agentic swarms. Absolute trainwreck.
AI doesnt make bad engineers good. Anyone who says they are doing 4 eng work likely would be without ai too. Those that claim otherwise, are the bad engineers.
sbarre 7 hours ago [-]
> There are standards in language that are not all “left means right for me so you cannot assume when i say right it is right and not left”
Yes of course there are words that have very clear meaning, and that's not the case here.
"startup" and "legacy" - used in the context of this discussion - are not words like "left" and "right" where everyone has a universal agreed-upon understanding of their meaning.
nujabe 24 hours ago [-]
exactly. Usually legacy code forms when people lose context and confidence in parts of the codebase due to staff turnover etc and ppl avoid touching or enhancing those parts for long periods. Six months is a short time to accrue that much tech debt, its enough time where most of the people who created that "legacy" are probably still around. As you said indicates bigger problems.
hunterpayne 23 hours ago [-]
So basically any LLM codebase of sufficient size is immediately legacy.
22 hours ago [-]
vonneumannstan 23 hours ago [-]
Wow how many years of experience with Claude Code and Codex do you have? lol
hunterpayne 23 hours ago [-]
The job requires 10 years of those technologies ;)
827a 17 hours ago [-]
I'm probably between $50-$200/day depending on the day; we also have effectively unlimited budget, though a lot of that is because Azure gives startups $150,000 in credits for 2 years, which we've wired up to a LiteLLM gateway & OpenCode. Without that I think our appetite would be more around $400/month/employee.
A lot of my high costs is because I just throw Sol at everything. If I were more selective and brought in Luna or v4 Flash every once in a while, I think I'd be more like ~$400/month. That's why I'm not aligned with the notion that "tokens are subsidized so that's why people are using so much": its not that I'll have to adjust to using less, its just that I'd need to think before I prompt a bit and be more judicious. I could easily see my raw token counts doubling or tripling in the coming months. I don't think that will change as subsidization subsides; though maybe lab revenue will; intelligence per dollar is getting cheaper every week. Its solely a function of adaptation to process, which takes time.
The productivity gains per token are the single most asymmetrical thing I've ever seen in engineering. The engineers on our team are pretty effective with tokens; easily that 2x-4x output as you're seeing, spending $20-$200/day. Some of our security folks have also started contributing more-and-more code, and they're on the other side: they'll spend hundreds a day running in circles, eventually producing these +/-30k loc pull requests that take ages to get merged and are littered with issues. They weren't writing much code before, so arguably they're more productive by some multiplier greater than 1, but I think the drag on the rest of the team, and potential issues with what they produce, has overall created a net-negative situation. Inversely, some other company functions have produced a few one-off websites for things like sales processes, and those have been a huge win. The asymmetry is wild. There's almost a valley of incoming skill where if you know nothing about code, you'll leverage it well; if you know just a little bit, it makes you super dangerous; if you know a lot, you're the biggest winner. Really difficult situation to navigate.
jchook 1 days ago [-]
This is very close to my workflow but you forgot one important step:
- Suggest a better approach that makes the AI say, “That’s much simpler. And you’re right. My original plan was over-engineered.”
blcknight 22 hours ago [-]
$80 sounds extremely low for what you're describing - are you on API token plans?
I have had some $3,000 token days - even without Fable. I don't see how this is sustainable.
My personal 20x plans get so much usage for so cheap. The consumer subsidies are crazy, but alas I can't use them for work.
extr 22 hours ago [-]
$80 is definitely low now that I look at my numbers. but not OOMs low, it's closer to like $200 on heavy days. i don't know how you're doing $3k/day, that's wild. i'm pretty aggressive about compaction and session restarts, and i reserve Fable 5/Sol XHigh for "main thread" orchestration
Footprint0521 10 hours ago [-]
Dude $3k? Holy heck you should look into K3/Deepseek V4 Flash
RugnirViking 1 days ago [-]
Do you have issues with performance at the moment? Right now I tend to find that it produces absolutely terrible design patterns and especially performance. I mean maybe I don't know exactly what area you're looking at but yeah for us we tend to find it's terrible wrt dB/caching/scaling and often any performance improvements it proposes end up actually shooting itself in the foot and being worse than before but it's not very good at testing in an organized way to even notice it made it worse despite repeated prompts to do so I mean if I prompt it to test performance in a handheld structured way (it is very bad at finding out what performance to test and why) before making changes I can usually figure it out but it usually takes insistence on the specifics to really ensure a good solution that will actually fix the problem
extr 1 days ago [-]
Performance is better than ever. It's never been more practical to set up wildly complex synthetic test environments and measure perf wins. Plus the models will find every possible algorithmic/design improvement.
It actually gives me quite an uncanny feeling, bulldozing over years of human optimization work with a newer, "perfect" design. Like bringing an AK-47 back to the middle ages.
RugnirViking 23 hours ago [-]
> the models will find every possible algorithmic/design improvement
it's so hard to square such totalizing statements with my day to day experience with fable and sol, (every possible, improvement, really?? they are NOT omniscient) arguing with them/my colleagues' agents that no they have slowed down the system 200x with their terrible change, doing string operations on millions of db rows, trying to get it to understand that I don't care that it's calling it a "cache" if a cache hit is slower than what we had before.
These agents do let you learn codebases quickly, and produce code way faster. I don't look at IDEs all that often. But literally multiple times every single day I catch them doing something stupid.
I don't think its impossible that we could get better performance from the agents. I know ive tried all sorts of workflows and skills, few of which seem to have much effect on the things the models struggle with. I think a big part of it is encoding enough context for large codebases, and providing it with all the tools it needs to make it successful, things to automatically check its work, etc. But that's not automatic, in fact its generally a terrible judge of what it needs or what its bad at
extr 21 hours ago [-]
yeah it's true, you do have to guide them. i find that the key is you have to know what's possible. you have to have the instinct for "this really shouldn't be so difficult". my junior SWE coworkers have the same trouble as your coworkers.
but the revolution is it doesn't take that long. in like 15 minutes you can chat with fable and get to the meat of whatever the issue is with repeated questioning. and then it does the solution for you. so it's not magic but it's still like a 100x speedup.
app13 1 days ago [-]
I needed to thoroughly test rerankers on my companies rather unique corpus.
Opus and I wrote a parallelized test harness and labeled groundtruth in around 2 hours.
In 2022 that would've likely been all I did for a couple sprints
steve_adams_86 23 hours ago [-]
I encounter this regularly and it still feels weird.
That sense that you did something better in a few days than you would have in a month 5 years ago. It's like buying a table saw for wood working.
One crazy thing I think about often is how there are so many correctness and testing harnesses that would have taken weeks to build in the past so we simply never would have. We'd just do our best then wait and see what comes to the surface. This is a huge part of what makes it possible to actually make better software with LLMs in my opinion. It isn't just 'LLM codes better than I ever could' (that's often untrue still) but 'LLM enables me to make assertions about the program to degrees that would have been absurdly impractical in the past'. It's huge
extr 1 days ago [-]
Yes 100%. This morning I casually prompted Codex to drive the browser to complete extensive performance testing in-situ that would have literally been weeks of work before. Probably in reality it just wouldn't have been done, and performance guarantees would have been attempted up front via more careful design.
In this case the design was also AI generated, and there were limited wins to be found because the design was already superb.
manmal 1 days ago [-]
Are you using the SOTA models at very high reasoning during planning? IME that makes a LOT of a difference. I‘d also never let them just rip into the architecture, but always push back and ask for alternatives first. Once the overall plan is nailed, not that much can go wrong. Provided it’s a reasonable change set and not a 20k LOC PR.
RugnirViking 23 hours ago [-]
fable or sol w/ very high both planning and execution, yeah. I feel the "push back" part is a big part of my job now (on every step, planning, execution, and review) yeah, but that feels pretty incompatible with the sorts of "just let it do what it wants" which other people seem to be claiming
imilev 9 hours ago [-]
Interested to dive deeper on the upfront design discussion. Have you found these to more often then not translate into the real product.
In my experience at the begining of the full agentic coding loop in our company we were more hands on with the codebase and had better judgement over the plans. Now it is quite often that the inital plan after executed needs more refinement and that made the plan review somewhat obsolete for us.
mrlongroots 16 hours ago [-]
In my experience, code is a small fraction of the work.
I'm in an infra team and for the last 2 weeks or so I've been trying to understand whether a particular workload will catch fire if a switch is flicked. I'm also new to the team so partly it is me wearing training wheels, familiarizing myself with the telemetry etc, but I will state that I'm not completely lousy at this stuff.
No model in my experience can do anything remotely comparable to the work "what happens to the workload if this switch is flicked" needs. They can't even design a reliable quick experiment to answer what cast should be applied to the binary trace_id in table A for the join to table B to work. They will happily do something idiotic and then conclude that the join does not work.
ianmarcinkowski 8 hours ago [-]
An AI maximalist on my team put up 2 pull requests with ~120-140 changed files this week. If they spent $200 on tokens to achieve this, we spent $2500-3500 in human salary and opportunity cost reviewing it.
z0mghii 3 hours ago [-]
You need to change the way you think about reviews
ajcp 22 hours ago [-]
Only spending $80 a day on Opus 5/Fable 5/GPT 5.6 Sol feels very low. I'll roll through a couple hundred dollars worth of credits a day with those models, the vast majority of which would be on non-coding tasks, and it's still a huge cost savings over me or my team having to do these things manually, if we'd even be able to do them at all.
But that's also why it's now easy to justify the cost of an Nvidia or Intel inference server with Kimi K3 locked and loaded :)
lelanthran 12 hours ago [-]
Is the revenue up by 2x or 4x?
drTobiasFunke 20 hours ago [-]
Output of 3 or 4 2022 engineers? Its that your self assessment? Output as in number of lines of code?
swader999 8 hours ago [-]
Not op, but we are at 2400 total points delivered over seven years. 1000 of those in the last six months. About 2-4 devs over that period, just two the last six months.
drTobiasFunke 4 hours ago [-]
Noone is questioning the volume of llm output. My question is whether all those points delivered improved your product and software in any meaningful way, or do you now have 20x more code that noone understands with the same quality of software product?
bryan0 22 hours ago [-]
> Spend most time prioritizing/discussing what to do.
you should probably be doing this discussion work along with Fable 5. It will give good feedback if you're working on the correct things.
> Come up with QA plan and have Codex Computer Use execute on it
QA plan should be part of the above "design + plan", not after it. The implementer needs to be able to fully test before publishing a PR. This is true whether humans or agents are writing the code.
> Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc)
unfortunately this is not really scalable with amount of code agents can produce, so you need independent (fresh context) agent reviewers to help. Ideally they only escalate to a human when really stuck.
> I probably spend like $80 a day at least
at a small startup you should be on the $200/month plan(s).
tfehring 16 hours ago [-]
I'm also at a startup. My workflow is similar but I have Fable 5 xhigh drive the whole thing: it gets Codex CLI installed in its environment with an API key, and it's instructed to delegate ~everything to Codex and review its work, especially for code quality/conciseness. Fable delegates to Sol or Luna (fast mode) xhigh/max depending on the task - I think Luna xhigh on fast mode is basically a Pareto improvement over Sol medium.
tosh 7 hours ago [-]
for complex open ended coding tasks better models are better
(and mid-to-long-term, often also short-term end up cheaper than weaker models)
this might change soon if we are reaching a certain capability threshold
but right now that's still the case
unless you are working on throw-away trivial stuff where iteration speed and trying many speculative things might give you an edge
samesense 17 hours ago [-]
You have an unlimited budget, and you only spend $80/day? I’m up to $3k/week, and still expanding.
deadlast2 12 hours ago [-]
Does it not end. Like is there not a point with all this speedups and infinite intelligence that your software system is essentially done.
eru 15 hours ago [-]
Interesting. I would probably start with the QA plan first, or at least before implementation (and perhaps even before design.)
K3UL 23 hours ago [-]
The output yes, but do you produce the impact and value of 3 engineers? I have seen this workflow being toyed with too, and I find it to produce massively overengineered stuff that actual people don't really wanna use
what 18 hours ago [-]
He only spot checks 1000s loc diffs, so probably has no clue.
matsemann 23 hours ago [-]
My experience is that your description works for a certain time, since you're knowledgeable of the codebase and can guide it. But after too many iterations with not hand-holding the llm, it quickly gets unwieldy.
aetherspawn 20 hours ago [-]
I was just about to say, how could routing possibly be worth it at the risk that the work output is sub par?
catfood 23 hours ago [-]
>Auto-fix bugs and CI until green + thermonuclear review skill x3.
Gotta love this loop, I have it running while I'm asleep all the time.
krzyk 10 hours ago [-]
How much does it cost you?
gamblor956 24 hours ago [-]
but I produce the output of 3 or 4 2022 engineers and probably at better quality.
Possibly, but the output of a 2022 engineer is about 1/10th of the output of a 2010 engineer, so it's an extremely low bar.
Krei-se 24 hours ago [-]
also - as always with these claims there's no actual product / repo / whatever one could check.
I would love to see what these tools create but outside slop there's never: This works, is in production, here's the code.
Any day now.
dgellow 22 hours ago [-]
It’s crazy how we are like ~2y in this AI revolution and still do not have an answer to this question: can you show us the ROI? Where is the revolutionary software your team of agents created?
bonoboTP 2 hours ago [-]
Why would it need to be revolutionary? It can be some ordinary thing. Software is mostly ordinary.
what 18 hours ago [-]
I found an interesting project recently. As I was looking through the source something felt off. Turned out to be entirely LLM written. There was duplicated code everywhere, same function defined in dozens of files (same name, same intended behavior) but none of them would produce the same output for an input. Dead code all over the place. Over architected. Useless comments.
It was all generated in the last 4 months, so don’t come at me with the “but did they use a model from the last 6 months” nonsense.
bonoboTP 2 hours ago [-]
4 months is ancient. Fable and Sol are a different animal.
biophysboy 1 days ago [-]
Do you have tips for generating clean productive output per dollar?
the_sleaze_ 1 days ago [-]
in my humble experience it boils down to mastery.
Are you at least conversational in the subject matter? You're gonna have a good time just by paying attention and adjusting your workflow. If you're getting a lot of back and forth with it, its asking a lot of planning type questions, stop, step back, rethink the whole feature, and start again from the beginning with everything more fleshed out.
If you are in a brand new field, there's no way to bridge that divide. The issue is you don't know what is good or bad, or whether what you have learned is good or bad. You're in a sports car and you don't know how to drive much less what's track and what's field.
You can spend a lot of effort getting good at prompting towards writing tests and E2E tests to at least verify your app does what you expect it to, regardless of experience.
extr 1 days ago [-]
This is a great point and I agree. My own productivity varies based on what part of the codebase I'm working on. If it's "been in there before" and I know the right questions to ask, I can one-shot a good design/improvement. If I'm spending 20-30 minutes asking Fable to "draw a diagram so I can understand" - probably less so. But notably, I CAN get there in a fraction of the time it would have taken before. You can general personalized onboarding docs to ~anything.
grigri907 1 days ago [-]
I appreciate this non-judgmental description of what it's like to approach a topic/technology from a newcomer's perspective. Thanks!
extr 1 days ago [-]
Keep the decision-making and execution separate. Use the high IQ models to chat about the design and make them drive subagents to do the actual work. "Chat" style threads are actually quite cheap. Where it gets expensive is having Fable 5 output thousands of lines of implementation where 95% of it was already overdetermined and there were only a few important judgement calls.
I actually have no doubt that I could replace my Opus 5 Low/Medium subagent profiles with Grok 4.5/GLM 5.2/Deepseek v4 Flash and perf would probably be pretty similar.
On top of that - highly recommend adding accurate cost counters to your statusline. You can't improve what you don't measure! (Or even have any intuition about).
ai_fry_ur_brain 1 days ago [-]
[dead]
ffsm8 14 hours ago [-]
> I probably spend like $80
mate, if youre not using subscription then youre spending waaaaaaay more. the plan itself with fable/sol will most likely have already cost more then $80 -- ime thats more like 500-2k/day of usage.
most harnesses let you see the usage in the status bar, i encourage you to enable it
nujabe 1 days ago [-]
> essentially unlimited AI spend budget
> I probably spend like $80 a day
This doesn’t sound like “unlimited”, I spend more than this out of pocket per day and I have a strict budget.
extr 1 days ago [-]
It's a fair point, it's not truly unlimited and I do wonder how that would change my workflow. I can definitely imagine if I was inside Anthropic or OAI with unlimited "fast" tokens, you would be more tempted to hand over even more of this process. I completely understand why they talk about "graph engineering" and such, my entire workflow above could be a graph and I could try to increase my leverage even further. Realistically though I am bounded by product decision making, not code output right now.
newsicanuse 9 hours ago [-]
Makes me wonder the kind of startup this peron is working for where slop is encouraged
willsmith72 16 hours ago [-]
> I probably spend like $80 a day
Wait what? I don't understand these numbers. I spend $1k/day
Your story about being told to use AI for everything I was expecting you to be well over that
bdangubic 21 hours ago [-]
do the same across 20 terminals (as you should) and now you are up to $1.6k/day. would that give you output or 60-80 engineers? not a chance, right?
no one’s AI spent will be in question working a single terminal with carefully planned out and executed process you do
1 days ago [-]
catlover76 1 days ago [-]
[dead]
lbriner 1 days ago [-]
There are a surprising number of articles like this along the lines of, "we started using AI tools and ended up spending millions per year".
On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs.
Otherwise it reads a bit like a fake problem, because it didn't really happen, you just foresaw it (as you should) and added a few guide rails.
habosa 23 hours ago [-]
There really has never been another product priced like AI is being priced right now. Each of these things has been done before, but all of them together is new.
1. Insanely discounted starter plans. Claude $200/mo plan is like $5k-$8k of API rate usage.
2. Very limited cost visibility, they make it hard to figure out where you spent money (unless you're on the enterprise plan which is for people with unlimited money).
3. Nobody, not even the model provider, knows what your request will cost before it returns. You're writing a blank check every time you hit enter.
4. When you run out you run out very suddenly and disruptively. It's very hard to tell a developer on the 28th of the month "sorry, code by hand until the 1st of next month" so you tend to grant exceptions.
5. The price is changing all the time. New models come in, old models come out, prices change, caching behavior changes, harnesses change, etc. The cost of doing a single task is not predictable even if the task does not change.
6. Basically no volume discounting. Anthropic offered us 2% off for committing to $1M+ per year at API rates.
I manage AI spend for my team at work and I try really hard to keep costs under control but it's absolutely herding cats. Much harder than any other spending I've ever had to manage at work.
Happy to give you (or anyone here) some trial credits if interested! Email address in my profile.
ankitmathur 24 hours ago [-]
Something underlying a lot of this is that pricing models for enterprise coding tools have changed from seat-based to consumption-based pretty quickly, as AI usage has exploded. For months, engineers were able to use unlimited AI for no marginal cost, but that's changed quickly.
In addition, we're seeing people applying AI to more and more use cases, so token growth is very significant. Paired with consumption pricing, it's brought this problem to the forefront very quickly for lots of companies.
dan_q 9 hours ago [-]
> On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs.
The AI rollout has in large part been about giving management an opportunity to say "fuck you" to engineering.
Engineering payroll needed to be slashed and the C-Suite was happy to let their managers get some lashes in on the uppity workers.
The point of AI is:
1. It's bad engineering and everyone knows it.
2. Management is pushing it through anyway.
3. Deal with it, asshole.
The ROI is besides the point. It's about breaking the spirit of engineering labor with irrational decisions made outside of their control.
Edit: It is just as much about affirming the exalted status of management in comparison to labor. It's a bitter reinscription of the status hierarchy in tech; Management is good and labor is bad; Management is considerate and labor is ignorant; Management is strong and labor is weak; Management is superior and labor is inferior.
pwendell 24 hours ago [-]
The issue is the growth rates can cause costs to drastically change quickly. If you have 1,000 employees and the average is spending $100/month you're at a $1.2M run rate. But suddenly a new model comes out that's twice as expensive, there are some changes to the harness (we found randomly Claude Code and other harnesses will make changes that drastically impact efficiency), and then maybe you have some organic user growth as well and BOOM suddenly you're at a $10M run rate within 60 days. And it's now impossible to forecast future growth.
It is true that this problem can be mostly managed by the techniques we mention here. Those are actually pretty difficult to set up at scale, so many companies (including us) we only really did this in earnest once we started to see those large cost oscillations.
The main reason we shared this here is to maybe help other companies get infrastructure in place before massive cost swings rather than after.
jgalt212 1 days ago [-]
> we started using AI tools and ended up spending millions per year
This is how AWS made its fortune.
chadash 1 days ago [-]
Not only this, but perhaps even more nefarious is that AWS gives lots of startups $100k+ in credits. This feels generous when you get it. In reality, it means that (unless you are in a compute intensive startup) you can go for months or years before you hit this, but by the time you do, you already have very solid monthly spend.
Initially, you picked the Multi-ZA RDS db.t3.2xlarge instance because you figured "eh i have credits anyway". Two years later, someone looks at this and says "hey, this is expensive and I bet we can do everything we need on a machine half the size". But then they think "if i downsize it and that works, i'll get a thumbs up emoji on a slack thread. If i downsize it and it causes problems, i'll draw the ire of the whole team. I better leave it alone." And the truth is... by the time your company hits the end of those credits, you're probably at the point where that savings isn't gonna do much. Or maybe you are out of business.
And that is how almost every successful company that uses AWS eventually ends up paying six-figures or more annually.
nitwit005 22 hours ago [-]
Someone did notice, as they panicked at the cost.
What didn't happen is any analysis of cost/benefit up front. Many of the corporate decisions around AI have seemed characterized by companies blindly copying each other.
K3UL 23 hours ago [-]
Weirdly a lot of the come from company that sell Ai credits in some capacity, and who are also selling (or will soon) some kind of AI gateway or router
therealdrag0 24 hours ago [-]
On this planet?
They’re not saying they regret doing it, or that it was a mistake.
They’re just saying they’ve gained experience and have leveraged the tools to an extent their usage can be optimized.
Pretty standard business or life iteration.
sashank_1509 20 hours ago [-]
I suspect that when it comes to hard complex software products, you’re better off ignoring agents and doing “trad coding”. What you lose in short term speed you gain in manageable complex codebases.
If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term.
Now of course, there are products that just involve lots of code but are not actually complex. This is generally the project with like hundreds or thousands of features but most of the features are separate and don’t actually interact in complex ways. Think a task management app with hundreds of features like calendar, email integration etc. there I think agents gives you more bang for the buck. Just my thought, using agents at work.
lazarie 8 hours ago [-]
I work on and maintain a 900k+ LoC codebase with mixed languages, runtimes, deployment models, cloud and airgapped ops and it's 99% written by AI. The difference I see the most between our approach and others is that we see the end-user product as an output of our context engineering internal machine, we focus on building that machine and we measure the output like madmen across all and any qualitative software metrics and signals (traces,collectors,otel,snapshots,profilers,custom debuggers,videos and more). For any change we do, less than 5% of wall time is an AI doing reasoning or coding, 95% is running the verification deterministically. Of that 5%, about 90% is spent on non-feature aligned code.
Velocity and quality is through the roof, "10x" would be a regression at this point, the bottleneck is only figuring out what is relevant/good to deliver, the implementation is the output of the machine.
sashank_1509 1 hours ago [-]
What exactly are you building? Does it need 1M lines of code, or is it 10X the needed code already. SQLite + tests is around 1.5M lines of code, somehow I strongly doubt you’re building anything as complex as SQLite.
ianmarcinkowski 8 hours ago [-]
I'm totally willing to believe that new orders of magnitude of software complexity are going to be possible with this kind of engineering. And also, my company brings in a lot of revenue off of 200k lines of code. My old startup brought in millions of revenue on ... very little code tbh.
Have you ever listened to a politician talk for an hour and you can't remember what they talked about?
If some VC wanted to fund my crazy ideas for building personalized private versions of the internet because people are sick of the nightmare, I would vibe code the hell out of that; but at my current job I have existing customers to keep happy and make sure their financial data doesn't get lost or modified.
SOLAR_FIELDS 20 hours ago [-]
> if you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term
A bold claim to make with little to no supporting evidence
delusional 14 hours ago [-]
We are 7 years into a cycle of people baselessly claiming that "AI" will soon develop superhuman abilities and might just take over the world, before killing us all. I don't think there's any way to take part in this discourse without making somewhat unsupported claims.
That is a wild article! Hopefully the recent US open source model initiatives can pan out to cut costs, because I’m not a fan of dependence on Chinese models either, but I’m in a spot where I need to maximize intelligence on minimal cost…
There’s not even a distant comparison US lab rn which is sad
seizethecheese 1 days ago [-]
I'd prefer congress to be asking questions (this is all they are doing so far, based on the article) before doing any legislating.
axus 1 days ago [-]
Why would it matter if foreign companies analyzed DoorDash data? Pizza deliveries to the Pentagon is all I can come up with, but that's publicly available at https://www.pizzint.watch/
I would bet my entire Polymarket balance ($0) that some military contractors have already asked AIs on the public Internet to design software for them.
tosh 8 hours ago [-]
> Using harnesses that are “less chatty” (more token efficient), or tuning existing harnesses to generate less token overhead.
agree, this works, undervalued!
look at minimal agents that protect the context window:
- pi (https://github.com/earendil-works/pi)
- smol (https://github.com/smol-env/smol)
some thoughts on the other tips (for coding):
1) stronger models are more token efficient for open ended tasks because at the limit …
- stronger models can solve tasks that the weaker models can not solve
- stronger models make fewer mistakes, compose things better (cli, abstractions, …)
- navigate the code base better
- are better at removing and simplifying the code base again
that of course is difficult to benchmark, so most attention goes to simple benchmarks that show cheaper models can get similar results on 'closed' tasks with easy to 'eval' results
2) dynamic request and task routing sounds great/obvious but is very very hard
- to benefit from caching you don't want to switch model or inference endpoint
- to _know_ a certain request can be routed to a weaker/cheaper model needs good context and a strong model to get right and often is still unknowable because the active coding session can go many ways and turn from trivial to challenging in a few turns, always in motion is the future, if you get it wrong you are back in the problem space of #1
using cheaper models and auto-routing do work well for 'closed' tasks where you have something repeatable and can evaluate whether a certain quality threshold is reached that you are comfortable with
for open ended coding sessions it is not so easy
that said: cheaper does not have to mean weaker, you want to look at the pareto frontier and stay up to date on new good models
there are many models like deepseek v4 flash and luna that are both cheaper and way better than most other models
dgellow 22 hours ago [-]
What I take from this is that models are already commoditized, and it’s pretty clear nobody has a moat: routing for the models, they can be swapped whenever new models are released, AI labs will have to continue to run on the treadmill non stop or be replaced. Long term I cannot imagine that business will be high margin. Routing for the harness, so anything that differentiate a provider vs another isn’t exposed to the user and isn’t too relevant.
One more datapoint for the thesis that OpenAI and anthropic aren’t viable, sustainable businesses, and cannot justify their $1T valuation and the level of compute commitment (reminder that OpenAI committed to >$750B in infra spending for 2030)
OrangeDelonge 22 hours ago [-]
Do you think Anthropic or OpenAI will eventually try to crack down on routing harnasses? Provide a more vertically integrated experience? They are already trying ro ship hardware products.
gizmodo59 22 hours ago [-]
I will believe there is no moat when the revenues for Anthropic is not 70B. It seems like people want to throw away money and they don’t like switching
thereitgoes456 15 hours ago [-]
There is no evidence that Anthropic's revenue is 70B.
Try, maybe. I don’t know how successful that can be. Will definitely be interesting to follow! Exciting times we are going through
adamddev1 6 hours ago [-]
> "Exponentially growing costs"
I don't think people thing don't think enough about the exponentially growing costs with AI. The cost of tokens, the cost of bugs, the cost of technical debt, the cost of software bloat and slowdown, the cost on the RAM market, on the environment, the costs of the little lies and hallucinations that creep into our codebases, blogs, books, and search results.
What seems like a manageable margin suddenly starts growing and destroying everything.
ianmarcinkowski 8 hours ago [-]
Maybe these larger models are better at producing concise, internally-consistent, non-polluting (of the wider codebase) features.
I just spent 3 days reviewing the code of a vibe coding maximalist at work. They needed to touch 96 front-end files, ~50 back-end files. We now have `is_uk_company` spread throughout the entire codebase every time we need to output a date format. I would say 30% of the code can be replaced by a moderately more-wise approach of "you ain't gonna need this", 20% is actively polluting and/or harmful to the coherence of the feature.
I'm pretty sure this plan came from a braindump of a verbal meeting they had with our product team, using OpenAI's top models. There were 2 initial spreadsheets that were ABSOLUTELY INDECIPHERABLE, and now we have a new one labeled "FeatureX - Phase 2 to 4".
Just the date fields is a hilarious example of why I'm skeptical. Python (babel.date) and Javascript (the browser's locale) both have locale-aware date rendering. Using a library for country-based date and time parsing would have eliminated ~12 country-specific change sites... but instead we got `is_uk` and raw date format strings (`%d/%m/%y`) in-line inside already-enormous functions. Also, at a meta level, the backend code shouldn't usually be formatting the dates (data export may be an exception), we should be sending ISO dates and having the frontend format the code if anyone gives a crap.
I'm ~$500/day and I spent 3 days reviewing this stuff. Our CTO spent ~2 days this week reviewing this feature, I'm not sure what his daily rate would end up being. Just the date-formatting part cost me half a day... that's at least $250. A few hundred dollars on Xhigh tokens burned $2500-3500 in human review effort, and I was delayed from:
1. Getting an entire sentiment analysis feature deployed by helping our interns
2. Getting a load balancer solution working to automate a manual process
3. Improving our deterministic Cypress test suite
4. Getting a demo feature launched for our sales team that would bring in more money
patrickmay 4 hours ago [-]
> Just the date fields is a hilarious example of why I'm skeptical. Python (babel.date) and Javascript (the browser's locale) both have locale-aware date rendering. Using a library for country-based date and time parsing would have eliminated ~12 country-specific change sites... but instead we got `is_uk` and raw date format strings (`%d/%m/%y`) in-line inside already-enormous functions. Also, at a meta level, the backend code shouldn't usually be formatting the dates (data export may be an exception), we should be sending ISO dates and having the frontend format the code if anyone gives a crap.
Did you reject the PR or is that not a career-enhancing approach in your organization?
sandeepkd 1 days ago [-]
I find this funny and interesting at some levels
1. Codex, Claude and others try to switch models being used at their level itself to manage the cost and outcomes
2. Now company like data bricks develops one more layer on the top of it to do the same task, of finding the base harness and applicable model
Companies like Codex and Claude are focussing/investing heavily on to ensure that people are using their harness directly or instead use APIs. Unless Databricks has some agreement in place they are violating the TOS and openly publishing an article about it. Would be interesting if openAi or Anthropic come back and claim for the API usage prices and all the savings go away.
ChoosesBarbecue 1 days ago [-]
... where in the article did they say they were using subscriptions? I'm fairly certain enterprises can't access subscription pricing in any case, they're all API costs (Anthropic doesn't support more than 150 on subscription pricing [0][1]).
Going through their harness (codex, claude) is subscription (app use) which is heavily? subsidized.
Anyone using the enterprise plan are charged the API pricing, however the article is not clear if Databricks is using enterprise plan or not which is why added the following disclaimer
> Unless Databricks has some agreement in place
Anon1096 24 hours ago [-]
Databricks is most certainly getting charged API pricing no matter what harness they are using. OpenAI and Anthropic models are so sought after right now that they set the terms even at the world's biggest companies, there is not a chance to get a special agreement for subscription pricing.
ChoosesBarbecue 23 hours ago [-]
You can use both of those harnesses without going through subscription. That is a native feature in both Codex & Claude Code, even for non-enterprise customers.
therealdrag0 24 hours ago [-]
They certainly have an enterprise plan?
justincormack 1 days ago [-]
Databricks will be using the API anyway, thats all you get with an enterprise agreement.
pinkgolem 1 days ago [-]
Open ai is allowing subscription use, anthropic also paused the effort to stop subscription use.
InsideOutSanta 1 days ago [-]
They did? Is there a source where I can learn more? I'd love to use my Anthropic subscription with opencode.
pinkgolem 19 hours ago [-]
Only if opencode uses the agent sdk/Claude -p
June 15 changes would be the keyword to check, but at least in anthropics case it's... Complicated
copperx 1 days ago [-]
Bans are not in effect?
oh_no 1 days ago [-]
buddy, they're on enterprise plans paying per token
lukasco 4 hours ago [-]
With self-driving agents, the costs stop being evident as you go, and show up after the fact. I've been building governors that slow the agents down, and then also look at odd things some of the harnesses do, such as constantly making mistakes in tool calls.
But overall, it's quite a task, and you really have to decide what you are prioritizing for. Do I want my agents doing lots of work, or (in my case), leaving some of my subscriptions available for me to do work.
nichochar 24 hours ago [-]
Surprisingly pragmatic and info packed article..
Kudos to databricks, I also find it interesting that such different companies (Stripe, Ramp, Databricks) are all building the exact same internal tools.
I think building companies is going to look more generic in the future because intelligence is an API now.
pwendell 23 hours ago [-]
Thank you for the feedback. We wrote this because after discussing with some of our peer companies, I realized everyone was roughly doing similar things. And I thought it would be good for someone to just systematically write down what those are so that others can try out the techniques if they find them useful.
DenisM 21 hours ago [-]
+1 well written, well paced article. Pleasure to read.
Have you tried measuring Gemini? now that you have the router it should be a simple task. Thanks!
desbo 3 hours ago [-]
“agentic coding has measurably improved every velocity metric we track and, in some teams, driven an order-of-magnitude gains in output”
Of course it has. But is that a good enough measure of value? What about the business and customer outcomes from all the extra lines of code and closed JIRA tickets? Was it worth it?
bisonbear 1 days ago [-]
This approach seems fundamentally predicated on being able to evaluate coding agents on your own code by having domain specific evals. With that knowledge, you can trust the routing logic is actually improving/maintaining perf while reducing costs.
Without the insight into agent performance, any changes like this feel like a gamble to save $$ at the cost of developer productivity
I'm actually working on building generic repo-specific benchmarks at https://stet.sh ;)
pwendell 24 hours ago [-]
The difficulty of evaluating coding agents is indeed a really big challenge. We built evals on our own codebase and shared some information about that to allow other companies to replicate. We found our own evals correlated loosely with public generic SWE benchmarks.
In large user populations like at Databricks I think the ultimate answer will come from experimentation instead of offline evals. We are already doing this in small groups, exposing them to new candidate models and then measuring per-developer cost and perceived quality changes.
bisonbear 5 hours ago [-]
Experimentation makes a lot of sense as agents need to be pleasant to work with, and power users are a willing and able cohort to do that differentiation. Agent capability is only part of the equation, as evidenced by the widespread frustration with the experience of working with Opus 5, despite it topping benchmarks and being objectively extremely powerful/competent.
However, as a self-admitted eval true believer, I remain bullish that publicly available eval capabilities will catch up to current model intelligence, and start to be able to capture more and more of the subtle components of the agent experience, such as interactivity and code maintainability. Meta and Scale have some evals in the interactivity space that seem pretty promising https://arxiv.org/abs/2606.29957https://labs.scale.com/papers/swe-interact.
Human usability of the agent should definitely remain the top priority, but IMO we focus too much on the subjective, and not enough on the measurable/objective now, and I think/hope we'll see more companies move in the direction of Databricks
arendtio 12 hours ago [-]
I think the important aspects are making individual developers responsible for their costs and restricting the most expensive models when costs are too high. That way, people have a chance to learn when they tend to use the most expensive models for trivial tasks.
jgilias 10 hours ago [-]
Definitely. I can easily cut my individual AI spend by 50% at least. But I won’t, until someone tells me to. For three reasons:
1. Thinking about how to optimize AI spend is one more thing to think about when I have too many things to think about already;
2. If there’s even a minuscule chance that me selecting down the model results in a task getting delayed because for that particular thing the model happened to be “too dumb”, I’m just not going to do that. Because I’m evaluated for what I ship and saying, “oh, not happening this week because I downgraded to Haiku for the exploration phase, and that resulted in a faulty plan, so it slipped a day” just doesn’t cut it.
3. Why should I spend time thinking about costs, and risk being slower because of that, if the next guy doesn’t?
imilev 8 hours ago [-]
Great article, I think the most important aspect from it is the auto-routing. As humans laziness is in our nature, so having to think if the model is capable enough is not something that most ppl will do - resulting in trying out smaller models which failed our task and then just giving up and running on the bigger model all the time.
Terretta 8 hours ago [-]
DISCLOSURE: I like Databricks. While cosplaying enterprise CTO, I've directed the purchase and heavy integration of their work for over a decade.
That said, this type of post needs to be read with product marketing context in mind.
> I think the most important aspect from it is the auto-routing
On the contrary, in white paper studies auto-routing is shown to destroy the single largest token cost they found, curiously not shown in their opening graphic. As they put it in their own words near the end:
"simple tuning of… caching settings… 50% reduction in … costs, with no observed quality degradation…"
They also aren't showing the harness called ‘pi’ which generally tops results (not only with ‘open’ models), and is consistent with Anthropic's recent "works better when we don't stuff 100k system prompts into context" about Opus 5. Recent models do better with less jank trying to prescribe behavior.
So then one wonders: why is this post written to say you should use a meta router but no mention that's offset by the cache busting, and show developers cost savings but no mention that Anthropic and OpenAI both have developer-facing utilization dashes already?
Perhaps it's so engineers can show enterprise procurement why they want to buy exactly what Databricks happens to have for sale.
aabdi 5 hours ago [-]
If that’s such a large problem then the clear solution is to do sticky sessions?
What’s the problem then?
Use a cheap ds4 or Luna and do the second model net net per one shot best case you save couple dollars per?
yeswecatan 7 hours ago [-]
Do you know they are using pi or are you inferring it?
5 hours ago [-]
thelastgallon 18 hours ago [-]
> nearly every company deploying AI tools at scale has hit the same wall: exponentially growing costs. That curve is unsustainable - left unchecked it will eventually overtake revenue.
But the question remains, AI hasn't shown any explosion in growth/revenue for most companies. The effects seem muted.
williamstein 16 hours ago [-]
If only one company had AI and nobody else did, then that one company would be expected to start seeing an explosion in growth/revenue. As it is, everybody has access to AI, so extensive use of AI should just result in keeping up with the competition. The ROI to spending on AI is "not dying".
skydhash 9 hours ago [-]
AI is sold as a power tool. Even if everyone is using it, you should see at least some absolute improvement in boost, even though they are still relatively the same. No such boost has been observed.
chis 1 days ago [-]
It’s funny how different everyone’s experience is with this stuff. To me the diminishing returns are more around not going crazy with prototyping or running with xmax thinking all the time. I haven’t found it hard to stay under the usage limit of one $200/mo Claude and one $200/mo Codex subscription.
If my company told me yeah we’ve decided you don’t get Fable or Opus 5 because it’s too pricey, you gotta use GLM whatever, I’d be displeased.
pwendell 1 days ago [-]
I authored this - happy to answer any questions.
salmonfamine 23 hours ago [-]
I think there is a lot of dev cope in this thread.
My workflow is very simple:
1. develop requirements for code change
2. take manual notes for implementation, maybe use LLM for some discovery/investigation
3. present notes to frontier LLM
4. develop implementation plan (bulk of work)
5. let LLM rip
6. review diff, manually fixing/refactoring code as necessary, sometimes prompting for revisions
7. get automated LLM review
8. get human review
this reliably produces the work of 2-3 pre-AI senior engineers with a lower bug rate, equivalent performance, robust edge-case consideration, etc.
Does the LLM produce over-engineered solutions? All the time. I stop it from doing that, or manually fix it myself.
Does the LLM always adhere to the best system design? No, not at all. I often have to guide its design into a better, north-star aligned one.
I don't just sit in front of my terminal and say, "Ok Claude, build the app." It is a very iterative process, and not without its potential pitfalls.
But it is very, very productive.
myegorov 3 hours ago [-]
My workflow as well. If I had to gauge the direction of the wind, this produces the work of somewhere between 1-2 pre-AI engineers. My own code quality probably went down, along with my focus and the depth of understanding. Bug rates have increased. I don't have any stats to prove this.
mortalapeman 21 hours ago [-]
This has been my experience as well. The best model I have access to right now is Opus 4.8. It's really good at fixing bugs in an established architecture or adding a similar feature, but it's absolutely mid tier at putting together a simple effective architecture for even common software problems. I can tell that functional programming practices are not a large part of it's training material.
salmonfamine 20 hours ago [-]
Fortunately I’m at a startup with basically infinite Fable 5, and it is legitimately a huge step forward over Opus 4.8. Would recommend. Hopefully some open weight models catch up to its capabilities soon.
wxw 1 days ago [-]
> Rapidly adopting newer, more efficient models delivers the largest cost wins of any technique.
I think the more interesting lever is the fourth they mention: token efficiency.
> By the time costly LLM inference occurs, the user's initial statement accounts for only a negligible fraction of the data fed into the AI system, meaning costs are dominated by context the user did not explicitly include.
I think there’s still lots of low hanging fruit in regards to monitoring and improving agent work. Look at your sessions. Look at how much time and context is being spent on, say, a web search returning dozens of results when one good single-pager doc would’ve been better.
ankitmathur 1 days ago [-]
100% - there's a lot to learn from traces from real-life sessions with coding tools! For example, I found it pretty eye-opening to see how wide the distribution of tasks truly is. There's also subtle things like how a poorly designed MCP API surface can cause a massive amount of token waste from the model just iterating on finding the right way to call it.
gabrigoo 15 hours ago [-]
[dead]
aliasxneo 1 days ago [-]
First time hearing of Omnigent. Anyone have experience using it?
notduckrabbit 1 days ago [-]
I've tested Omnigent superficially, attracted to its thinking around policy, governance, sandboxing, and ui. But it's still alpha at present. I forked its Polly model and got working a somewhat more complex multiagent workflow that I've also modeled in Sandcastle and Gas City but the agent broke after the next update which I would have needed to patch to maintain functionality. Subjectively I also noticed individual models seemed to be performing somewhat worse when wrapped in the platform's framework, presumably due to the extra context introduced (token use was measurably higher). Promising project that I'll revisit when it's further along and I do not doubt the outcomes Databricks claims in committedly dogfooding it.
DenisM 21 hours ago [-]
Double-harnessing distortions seems like a weak point. I wonder if it’s just a temporary measure and long term it’s about writing custom harness going straight to inference APIs across all models.
Similar idea re using the same UI across all models, but the agent can modify the harness config as well as start/schedule sessions, etc. The Kanban board feature can be used to orchestrate agent driven workflows, and the agent itself can modify the Kanban lanes. Basically the agent can do all the same things you use the UI for.
Example prompt: “schedule a session using Opus 5 with max thinking for every feature in the PRD on the canvas. For every session make the prompt instruct the agent to review end to end test coverage for the feature and create a report on the canvas for test gaps that you find. Schedule the sessions overnight tonight spread evenly throughout the night, and have each session set to retry upon token exhaustion.”
I think it’s more meta than other meta-harnesses, but I’m biased because it’s my pet project.
vehemenz 24 hours ago [-]
I've been using it for a week or so. The main draw for me is that I can keep my sessions in one database regardless of the model/provider I use. The webapp can access everything remotely, which is convenient when I'm on my phone.
I haven't gotten a chance to test the multi-agent capabilities, but the DeepSeek Flash prices are so low that I probably will soon.
lubujackson 1 days ago [-]
These seem like the obvious tweaks akin to "using a cheaper hosting platform". I think the real savings come from careful context control for programmatic agents, careful tool awareness and usage to reduce thrashing, distilling workflows into deterministic processes and, moat importantly, adding friction and boundaries for non-technical users who tend to burn tokens making insane asks like "analyze all documents and give me a summary".
behat 23 hours ago [-]
Appreciate the detail in this and the previous post on creating internal benchmarks!
Have you all attempted finetuning smaller OSS models on your repos for coding?
pwendell 22 hours ago [-]
We do this for a lot of our customers (fine tuned to save cost when inference volume is high). Right now for internal coding we are using off-the-shelf models but we are considering fine tuning as well to squeeze more efficiency out.
throwatdem12311 19 hours ago [-]
“use lower cost models”
“use price controls”
Truly revolutionary stuff.
nh43215rgb 13 hours ago [-]
This is interesting. So as of August 6, 2026 the "Databricks recommended models" are glm 5.2, gpt 5.6 sol, and.... Opus 4.8. not opus 5.
XCSme 20 hours ago [-]
As a solo dev, this gives me hope.
I feel like I have an advantage over big companies, if I can use the best models on a subscription and not worry about costs much, when they can't do the same as outlined in the article.
DenisM 22 hours ago [-]
How can Smart Router achieve higher task completion rate compared to any of the base models if all it does is dynamically switch base models based on cost??
vira28 19 hours ago [-]
Reads like an add to Omnigent or whatever harness (wait it’s meta harness?.
shay_ker 1 days ago [-]
how do any of these routing approaches handle kv cache misses? Devin Fusion is the only one that explicitly addresses this, though it does so by switching models during compaction (not sure this isn't still a cache miss though)
ankitmathur 24 hours ago [-]
We're going to do a followup blog detailing our routing approach soon! In short, the router takes in the task description and infers what models and harnesses are available and makes a recommendation up-front. So essentially the routing decision is made when the harness + model is kicked off and it's only changed halfway through if there's a major delta in complexity from the initial judgment. Therefore, most of the time the cache is maintained just as it would be before (this is the advantage of having a meta-harness that is actually planning all the sub-agents centrally)
Maintaining the cache is extremely, extremely important, so we're iterating fast but that's a major factor we track in the router's development. Couple things I'd look at:
1. The cache is generally reset after a compaction - this is the best time to make a switch if you want.
2. In many cases, the max duration of a cache is 1h, so if a session is being resumed after a long time, that's also a good time to re-assess the complexity.
We're iterating fast here and learning a lot! Definitely a lot to think about it in this area.
chris_money202 24 hours ago [-]
The kv cache is wiped as soon as you get your answer, cloud hosts are not going to hold the GPU memory for your entire session. You're probably referring to some agent level cache
semessier 20 hours ago [-]
not sure about the use of exponential and efficiency frontier here, these have formal sides to them but seem to be used rather inflationary and colloquially.
readingrainbow 4 hours ago [-]
Another AI Psychosis/AI Derangement Syndrome post from a shite company. Yay!
nphardon 20 hours ago [-]
good engineer + llm = good engineer.
bad engineer + llm = bad engineer.
sellmethepen 1 days ago [-]
is this opensource or have to buy from Databricks?
Omnigent and OpenRouter are different in the sense that OpenRouter is where you can go to call the actual model but Omnigent is intended to be the place where you go describe the high level task to be done, and work is farmed out to various harnesses and models. Those sandboxes can themselves be using OpenRouter for capacity!
We're calling the layer coordinating harnesses "meta-harness'
jvican 23 hours ago [-]
Omnigent seems to compete more against Orca https://github.com/stablyai/orca
They both went to be the Agent IDE layer, where you come with your tasks and everything is taken care of. I've been using Orca for a handful of tasks and have been largely enjoying it. My default barebones workflow is ghostty + zmx on ssh connections.
mandeepj 22 hours ago [-]
These tools casually like to claim they are orchestrators, but unfortunately, none of them are.
tfrancisl 23 hours ago [-]
Ultimately, Databricks wants your enterprise on their platform. I dont think they particularly care about open source or the little guy.
dyauspitr 24 hours ago [-]
So did we. I just asked my team to get personal accounts that I reimburse them for. It’s just a golden age loop though, the gravy train can’t go on forever unless we start building out thousands of data centers and associated renewable energy.
dude250711 24 hours ago [-]
First the mofos force you to use AI then they become stingy about it.
An AI-edited post by the way.
quikoa 23 hours ago [-]
Well yes, first hit is free.
cyanydeez 1 days ago [-]
Probably coulda got every dev a local model for how much they spent; what a brialliant set of economists
resonance_dev 10 hours ago [-]
[flagged]
Sattyamjjain 12 hours ago [-]
[flagged]
ENTROPY-X 11 hours ago [-]
[flagged]
machinatools 1 days ago [-]
[flagged]
bogota 1 days ago [-]
Really? Because removing it from my company has saved us over 2 million a year and we were able to speed up processing. The chargeback model for databricks is predatory at best.
smt88 1 days ago [-]
I think you’ve misunderstood the article. It’s about how Databricks reduced their own costs, not about how adopting Databricks will reduce anyone else’s costs.
SteveNuts 1 days ago [-]
What did you move to and what type of workload, if I may ask?
skullone 1 days ago [-]
Yawn. Databricks and their half baked overly expensive platform.
GiorgioG 1 days ago [-]
Too bad their AI query generation is next to useless.
dan_q 1 days ago [-]
Quit cold turkey and you can drive down AI coding spend 100%.
- Spend most time prioritizing/discussing what to do.
- Once that's agreed, use Fable 5 High + 5.6 Sol XHigh come up with a design + plan. Agree on the high level plan. (Usually this just comes down to choosing where the change belongs on the spectrum between minimal patch <-> full redesign)
- Use Opus 5 or Sol Med to execute
- Auto-fix bugs and CI until green + thermonuclear review skill x3.
- Manual interrogation of change/nits
- Come up with QA plan and have Codex Computer Use execute on it
- Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc)
I probably spend like $80 a day at least but I produce the output of 3 or 4 2022 engineers and probably at better quality. So it's easily worth it. Would I save money by switching to GLM 5.2 and such...perhaps? IDK. At our scale it's not worth the time spent building the eval harness to actually understand the performance tradeoff.
When I personally want tight code, I have to spend a considerable amount of time adjusting it manually:
- It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating
- I need to consider whether there are solutions based on higher-level assumptions, that AIs typically miss
- I need to check whether there are off-the-shelf solutions - AIs like to reinvent the wheel
IMO, software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
(1) https://simpsons.fandom.com/wiki/The_Homer
> software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
Agreed.
Homer reunites with his long lost brother, who runs car company Powell Motors. Homer is ultimately tasked by his brother with helping to design a car for the “average man” that ultimately bankrupts the company for being wildly overengineered and costing too much ($82,000 in 1991-money).
Here’s the car: https://simpsons.fandom.com/wiki/The_Homer
https://amphetamem.es/meme?id=the-simpsons_02_15_490&text=Th...
For the vast majority of software out there, this quality gap simply doesn't matter. Not everything is big tech, needing internet scale. Most software built by developers at non-tech companies are simple line-of-business apps used by a handful to a few dozen or maybe a few hundred people to do a thing a bit better than the Excel spreadsheet that was used before.
And AI has essentially automated the development of this kind of software. Or rather, it has dramatically increased the ability of skilled developers to produce larger amounts of these kinds of systems.
Excel hardest hit.
Respectfully, I disagree. When you have 5 users using an app sandboxed within the corporate firewall that just does some CRUD and basic workflows, these problems you're calling out really just don't matter much. Some do some of the time, but not much.
Obviously not all software falls into this category. But so, so much does. Do you know how many thousands of tiny applications are maintained by big all non-big-tech corporations? I've seen many of them up close -- the code is usually complete garbage (seriously, it's bad; and alas, most was written before the age of AI coding agents), and yet, the business continues to operate and use the software.
You don't mean the majority of paid for software can behave non-deterministically? That's the value add of the human in the loop.
You will really have to weigh the cost of making the software against the expected revenue.
In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change.
Even before LLMs, you could've commissioned a half-assed clone of any app you wanted from a 3rd world consultancy for a few thousand dollars. LLMs are basically Bangalore-as-API.
I remember hearing a story that in the past movies were so technically difficult to make that any movie that got made had a good chance to be a profitable hit. But as movies got cheaper to make, more movies got made. Nowadays movie studio execs have to really calculate out the audience and expected revenue for any new movie and balance that against the budget and the cost of the studio's failed movies.
I think a similar dynamic may happen in software
That might be part of it, but I think it also has to do with the reality of replicating and scaling. Hardware or physical goods simply don’t scale like digital goods. There can be hundreds of knock-off physical products that have lower quality and lower cost but serve 90% of the same purpose, because physical capacity for raw materials, construction, labor, shipping, etc. have scaling limits in each market and economy. Digital goods are just so much easier to replicate and scale, so it often doesn’t make sense to buy software at lower quality and lower price if it doesn’t do most of the job. There are still limits of course, and different from physical goods, but I think this is a key reason why software is seen as winner-take-all.
As the LLM's keep getting better at writing software actual coding skills required will decrease which will result in a huge influx of software crashing prices. You can see this in already in open source projects in the last 6 months to a year Jellyfin had a huge influx of custom clients made by different people for fixing their own itches about the official software.
I.e. because software was hard to make and complex to copy you would tend to have "natural monopolies" that were hard to compete with. Who wants to try to build a new desktop OS to compete with Windows? Or a web browser from scratch? Or a new search engine? Etc.
Those and other pieces of software were complex and hard to make. The cost to copy and compete was very high. So one winner took most because that winner was the company who could figure that software out.
But as we can see with Kimi Work and other such things, software is now much easier to copy. Let's say it took $1 billion to make a copycat piece of software with people but now takes $100 million or $10 million with AI. Suddenly a copy and compete tactic makes much more sense than before
For example, with AI it might make financial sense to build a Chinese Native OS instead of Windows. Similarly for Russia, Iran, the EU, and a whole bunch of other places. All of a sudden, Windows might not be the winner take most OS, we might have lots of Operating Systems, with smaller markets and lower profits, which require much more careful financial analysis to stay profitable.
This would be just like Movies, TV Shows, Books or Music. When something works, people relentlessly copy it and different regions put their own spin on the idea. After Iron Man succeeded we had so many super hero movies. Etc. So there is not really a winner take most dynamic in these other digital products. Software may be moving that way
This could not be further from the truth. It might be true if the software works without supervisors in a B2C scenario where everything is self explanatory and nothing can go wrong, movie streaming services for example, and even these have tons of variants and multiple big players racking up billions in market share. So even there "winner-takes-all" does not apply. It gets even worse in B2B software, as there is SO MUCH software thats commonly used and needs tons of supervising by experienced devs. If we had a "winner-takes-all" market a lot more software would already be dead. And a lot of companies would have way less options when adopting new software for something.
It's not like restaurants where for example you can open a "Larry's Pizzeria" and still build a valuable business despite a Domino's nearby.
Fairly sure software dev was always an iceberg. Most software and most software devs aren't working on horizontal software, but on vertical software, in cost centers. Sales for that kind of software don't scale as much.
When we were making brick houses we needed a few brick factories with thousands of people for million of bricklayers.
[1]: https://pluralistic.net/2024/04/24/naming-names/#prabhakar-r...
Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade.
I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.
I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases
I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
Maybe currently not. But we will never be able to know, as models are undeterministic and benchmarks are kind of scams. When you cannot prove that something gets worse, rest assured companies will to it.
Which ones don't?
I used to play Wing Commander II. It was 30mb. Today, the software to adjust the colored lights on my keyboard requires literally a thousand times as much memory.
For who?
The public? The public has never liked buying software at any price.
Businesses? Businesses need higher quality software when it's relevant to their core competencies, so they hire people instead. Buying competing SaaS or depending too much on AI is throwing the baby out with the bathwater.
Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be better abstracted when that's actually needed.
This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI.
[0] https://www.salaryexpert.com/salary/job/software-developer/u...
Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so.
Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.
Pay hits a ceiling, and that ceiling is moving lower regardless of experience. That has nothing to do with AI, but what the market will bear. Hiring counts of humans must increase no matter what. Moving some of the spend to AI reduces the risk of hiring less qualified employees they might have rejected a decade ago.
Wages at the top end are stagnating to subsidize this. That's undeniable.
And this is how I find out I'm woefully underpaid.
We're entering a market of lemons in software. It's going to be interesting indeed.
"Will I benefit from this code being minimalist before [date]", where [date] is whenever you think the agent will be good enough to come back and make the corrections you would make today.
Even as a self-contained unit, you can't step in the same river twice, and on [date] some important details may have seriously faded, both in terms of text that can be mined and also in terms of human "why did we do that" and "what was the reason we did it this way and not that way" etc.
How does a “small startup” end up with a multi million line “legacy” codebase? Something not mathing
Easy! The output of 6 months ago Opus! Which seemed so wonderful at the time.
My observation has been:
- Initial greenfield work by an LLM is fast and very effective with minimal or no human oversight.
- Subsequent work ends up being over engineered and very verbose. Assumptions are made that aren't suited to the problem at hand (for example I find Fable is extremely regex happy where structured data would work much better from a readability perspective.)
- Once code bloats beyond a certain point due to unguided LLM usage, complexity is high enough that only LLMs can operate on the codebase with any economical amount of time.
- Rinse repeat and your code ends up unclear about any state that's not explicitly being tested and verified in QA loops
For some of our products this has been fine, for others it's been problematic. An understanding of your size and reliability requirements will help make the conversation more productive.
Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage?
> any state that's not explicitly being tested and verified in QA loops
As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?
You didn't say anything positively or negatively regarding this so I made an assumption that you were using the LLM relatively unguided (e.g. a bit of oversight, not the kind of thing that heavy code reviews used to involve pre-agents.) Feel free to add clarity on your actual usage loop.
> As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?
In my experience, most engineers are quite good at reasoning about code behavior for non-QAed code paths. Obviously things fall through the cracks. But I've been in the ground floor of plenty of Big Techs in their early stages before agents and, yes, a lot of initial development had spotty test coverage and yet most of the engineers had good mental models of what was happening. It used to be a very valuable skill to wrap your head around a torrid piece of code with few or no tests but was nonetheless a core piece of your application. Conversely, agentic development can bring cognitive debt [1].
===
This isn't a fight. We aren't sparring over what's right and wrong. I'm just curious how other people use agents in their work as someone who is also now in a startup that uses LLM agents heavily and has no limitations on spend.
[1]: https://martinfowler.com/fragments/2026-02-09.html
I feel like this statement betrays your lack of advanced experience coding with LLMs.
OP's elaboration of the steps they are going through (planning, agreeing on plan, getting one LLM to draft execution plan, approving it, then executing with a separate LLM, then reviewing/testing) made it super obvious to me that they are guiding their LLMs quite considerably as part of their work.
Anyone making blanket statements about LLMs producing garbage is just telling on themselves about not having proper SDLC practices in place.
I find for my own work that I need to read the diff the LLM produces then offer feedback on the diff in its own loop before I am satisfied, and this is after all the unattended QA steps through Codex Computer or Claude MCPs happen. Then auto reviewers come in and then reviewers come in. Of course, at our stage, we rarely have this luxury and it's only reserved for the very core of our codebase.
This is still much less guidance than we used to do for code before agents became popular. Even at Series A companies, before agents, we used to socialize tech specs, get buy-in from multiple engineers, create test plans, etc etc.
> Anyone making blanket statements about LLMs producing garbage is just telling on themselves about not having proper SDLC practices in place.
> I feel like this statement betrays your lack of advanced experience coding with LLMs.
Are we in school debate club? I don't know what's going on lol, I'm just curious how people are using LLMs! Is it just that irresistable to take a cheap shot at each other?
Not that I know of but that's the conclusion I drew from your statement.
It's not a cheap shot unless you took it personally?
I suppose I could have said "the fact that OP's explanation of how they work did not lead you to conclude they were in fact guiding their LLM usage quite a bit tells me that perhaps you have not been working with LLMs in any advanced capacity".
For the SDLC comment I admit it was a broader statement (based on observing people generalizing that "LLMs produce bad outputs") and not specifically aimed at you, and I didn't make that clear, so my bad.
Yes, this is literally what that means.
What exactly does "unguided" mean to you, then?
You might be surprised at how much benefit you get from the model council[0] pattern.
The common pattern I've seen is "build with Claude, review with GPT", or vice-versa.
It's been proven to be very effective.
0: https://www.google.com/search?q=model+council+pattern&udm=14
Design is too expensive, we do agile. QA too expensive, we fire all of them, and claim devops is the now, which allows us to fire the Ops team too, 100% ownership from deisng to ops on devs.
One person with an agent can replace all these teams. Yeah mo profits.
What is the point of working at a startup if you’re dealing with millions of lines of legacy code ? Isn’t the whole point of startups to create & innovate with a clean slate and modern tools?
I’ve never seen a startup with a multi million line legacy codebase.
>”Have you worked at many startups?”
In response to a question about a legacy codebase at a startup. That implies that they think whatever they are doing is common. And forking a multi million line codebase and heavily developing it isn’t common for startups.
OP might just be working at a small software company or for one that broke from a bigger one and is now "startup" like?
And if you're saying (based on your other comments) that a 6 month window is enough to create a legacy codebase...that indicates a serious lack of experience or understanding as to what a legacy codebase is, or why they exist.
Accept that other people may ascribe different meanings/interpretations to words than you, and that if your reading of their statement doesn't make sense to you, perhaps you are simply reading it wrong.
Trying to hold someone else to your definition of words suits what purpose exactly? Are you just trying to "win" ?
But who cares? The point is any codebase over a few years old with lots of customers and a big surface area has lots of code, much of it "legacy" from the standpoint of a guy in 2026.
15 people is a pretty small startup, what if this is a 50-person startup?
Doesn't seem like that much to me, depending on what you're building and the size of your team.
15 devs putting out 400k LOC a year into the same codebase is not normal at all. I’ve never seen anything close to that kind of rate of growth across that number of people.
And 50 engineers is not even remotely close to a small startup. I have worked at a startup that had 50 engineers after a few years, but it was a multi billion dollar unicorn.
This whole thread around loc is depressing. It speaks volumes of some peoples inexperience working on actual legacy code. Legacy code is not just age or size but that the technical foundation is dated in a fundamental way. A giant monolith running on a now defunk framework using a database only one guy in canada knows about.
Case and point in my day job. The org that owns XMM development does not know how to recover a physical bench that is bricked because everyone who knew how has left. So now they just use simulators…
Interestingly, AI figured out some of this pretty easily for me. But the org has the exact same AI as i do. At the same time another org is close to a year into a greenfield rewrite that has been developed via agentic swarms. Absolute trainwreck.
AI doesnt make bad engineers good. Anyone who says they are doing 4 eng work likely would be without ai too. Those that claim otherwise, are the bad engineers.
Yes of course there are words that have very clear meaning, and that's not the case here.
"startup" and "legacy" - used in the context of this discussion - are not words like "left" and "right" where everyone has a universal agreed-upon understanding of their meaning.
A lot of my high costs is because I just throw Sol at everything. If I were more selective and brought in Luna or v4 Flash every once in a while, I think I'd be more like ~$400/month. That's why I'm not aligned with the notion that "tokens are subsidized so that's why people are using so much": its not that I'll have to adjust to using less, its just that I'd need to think before I prompt a bit and be more judicious. I could easily see my raw token counts doubling or tripling in the coming months. I don't think that will change as subsidization subsides; though maybe lab revenue will; intelligence per dollar is getting cheaper every week. Its solely a function of adaptation to process, which takes time.
The productivity gains per token are the single most asymmetrical thing I've ever seen in engineering. The engineers on our team are pretty effective with tokens; easily that 2x-4x output as you're seeing, spending $20-$200/day. Some of our security folks have also started contributing more-and-more code, and they're on the other side: they'll spend hundreds a day running in circles, eventually producing these +/-30k loc pull requests that take ages to get merged and are littered with issues. They weren't writing much code before, so arguably they're more productive by some multiplier greater than 1, but I think the drag on the rest of the team, and potential issues with what they produce, has overall created a net-negative situation. Inversely, some other company functions have produced a few one-off websites for things like sales processes, and those have been a huge win. The asymmetry is wild. There's almost a valley of incoming skill where if you know nothing about code, you'll leverage it well; if you know just a little bit, it makes you super dangerous; if you know a lot, you're the biggest winner. Really difficult situation to navigate.
- Suggest a better approach that makes the AI say, “That’s much simpler. And you’re right. My original plan was over-engineered.”
I have had some $3,000 token days - even without Fable. I don't see how this is sustainable.
My personal 20x plans get so much usage for so cheap. The consumer subsidies are crazy, but alas I can't use them for work.
It actually gives me quite an uncanny feeling, bulldozing over years of human optimization work with a newer, "perfect" design. Like bringing an AK-47 back to the middle ages.
it's so hard to square such totalizing statements with my day to day experience with fable and sol, (every possible, improvement, really?? they are NOT omniscient) arguing with them/my colleagues' agents that no they have slowed down the system 200x with their terrible change, doing string operations on millions of db rows, trying to get it to understand that I don't care that it's calling it a "cache" if a cache hit is slower than what we had before.
These agents do let you learn codebases quickly, and produce code way faster. I don't look at IDEs all that often. But literally multiple times every single day I catch them doing something stupid.
I don't think its impossible that we could get better performance from the agents. I know ive tried all sorts of workflows and skills, few of which seem to have much effect on the things the models struggle with. I think a big part of it is encoding enough context for large codebases, and providing it with all the tools it needs to make it successful, things to automatically check its work, etc. But that's not automatic, in fact its generally a terrible judge of what it needs or what its bad at
but the revolution is it doesn't take that long. in like 15 minutes you can chat with fable and get to the meat of whatever the issue is with repeated questioning. and then it does the solution for you. so it's not magic but it's still like a 100x speedup.
Opus and I wrote a parallelized test harness and labeled groundtruth in around 2 hours.
In 2022 that would've likely been all I did for a couple sprints
That sense that you did something better in a few days than you would have in a month 5 years ago. It's like buying a table saw for wood working.
One crazy thing I think about often is how there are so many correctness and testing harnesses that would have taken weeks to build in the past so we simply never would have. We'd just do our best then wait and see what comes to the surface. This is a huge part of what makes it possible to actually make better software with LLMs in my opinion. It isn't just 'LLM codes better than I ever could' (that's often untrue still) but 'LLM enables me to make assertions about the program to degrees that would have been absurdly impractical in the past'. It's huge
In this case the design was also AI generated, and there were limited wins to be found because the design was already superb.
In my experience at the begining of the full agentic coding loop in our company we were more hands on with the codebase and had better judgement over the plans. Now it is quite often that the inital plan after executed needs more refinement and that made the plan review somewhat obsolete for us.
I'm in an infra team and for the last 2 weeks or so I've been trying to understand whether a particular workload will catch fire if a switch is flicked. I'm also new to the team so partly it is me wearing training wheels, familiarizing myself with the telemetry etc, but I will state that I'm not completely lousy at this stuff.
No model in my experience can do anything remotely comparable to the work "what happens to the workload if this switch is flicked" needs. They can't even design a reliable quick experiment to answer what cast should be applied to the binary trace_id in table A for the join to table B to work. They will happily do something idiotic and then conclude that the join does not work.
But that's also why it's now easy to justify the cost of an Nvidia or Intel inference server with Kimi K3 locked and loaded :)
you should probably be doing this discussion work along with Fable 5. It will give good feedback if you're working on the correct things.
> Come up with QA plan and have Codex Computer Use execute on it
QA plan should be part of the above "design + plan", not after it. The implementer needs to be able to fully test before publishing a PR. This is true whether humans or agents are writing the code.
> Manually spot check the final result (usually a sizable diff, thousands of lines, complete feature E2E, etc)
unfortunately this is not really scalable with amount of code agents can produce, so you need independent (fresh context) agent reviewers to help. Ideally they only escalate to a human when really stuck.
> I probably spend like $80 a day at least
at a small startup you should be on the $200/month plan(s).
(and mid-to-long-term, often also short-term end up cheaper than weaker models)
this might change soon if we are reaching a certain capability threshold
but right now that's still the case
unless you are working on throw-away trivial stuff where iteration speed and trying many speculative things might give you an edge
Gotta love this loop, I have it running while I'm asleep all the time.
Possibly, but the output of a 2022 engineer is about 1/10th of the output of a 2010 engineer, so it's an extremely low bar.
I would love to see what these tools create but outside slop there's never: This works, is in production, here's the code.
Any day now.
It was all generated in the last 4 months, so don’t come at me with the “but did they use a model from the last 6 months” nonsense.
Are you at least conversational in the subject matter? You're gonna have a good time just by paying attention and adjusting your workflow. If you're getting a lot of back and forth with it, its asking a lot of planning type questions, stop, step back, rethink the whole feature, and start again from the beginning with everything more fleshed out.
If you are in a brand new field, there's no way to bridge that divide. The issue is you don't know what is good or bad, or whether what you have learned is good or bad. You're in a sports car and you don't know how to drive much less what's track and what's field.
You can spend a lot of effort getting good at prompting towards writing tests and E2E tests to at least verify your app does what you expect it to, regardless of experience.
I actually have no doubt that I could replace my Opus 5 Low/Medium subagent profiles with Grok 4.5/GLM 5.2/Deepseek v4 Flash and perf would probably be pretty similar.
On top of that - highly recommend adding accurate cost counters to your statusline. You can't improve what you don't measure! (Or even have any intuition about).
mate, if youre not using subscription then youre spending waaaaaaay more. the plan itself with fable/sol will most likely have already cost more then $80 -- ime thats more like 500-2k/day of usage.
most harnesses let you see the usage in the status bar, i encourage you to enable it
> I probably spend like $80 a day
This doesn’t sound like “unlimited”, I spend more than this out of pocket per day and I have a strict budget.
Wait what? I don't understand these numbers. I spend $1k/day
Your story about being told to use AI for everything I was expecting you to be well over that
no one’s AI spent will be in question working a single terminal with carefully planned out and executed process you do
On what planet do people start paying for things without keeping an eye on the costs and no-one notices until you have spent a crazy amount? I don't understand. You are either paying a fixed amount which you are happy about in-advance or you are PAYG in which case you would ballpark how much it costs.
Otherwise it reads a bit like a fake problem, because it didn't really happen, you just foresaw it (as you should) and added a few guide rails.
1. Insanely discounted starter plans. Claude $200/mo plan is like $5k-$8k of API rate usage.
2. Very limited cost visibility, they make it hard to figure out where you spent money (unless you're on the enterprise plan which is for people with unlimited money).
3. Nobody, not even the model provider, knows what your request will cost before it returns. You're writing a blank check every time you hit enter.
4. When you run out you run out very suddenly and disruptively. It's very hard to tell a developer on the 28th of the month "sorry, code by hand until the 1st of next month" so you tend to grant exceptions.
5. The price is changing all the time. New models come in, old models come out, prices change, caching behavior changes, harnesses change, etc. The cost of doing a single task is not predictable even if the task does not change.
6. Basically no volume discounting. Anthropic offered us 2% off for committing to $1M+ per year at API rates.
I manage AI spend for my team at work and I try really hard to keep costs under control but it's absolutely herding cats. Much harder than any other spending I've ever had to manage at work.
Happy to give you (or anyone here) some trial credits if interested! Email address in my profile.
In addition, we're seeing people applying AI to more and more use cases, so token growth is very significant. Paired with consumption pricing, it's brought this problem to the forefront very quickly for lots of companies.
The AI rollout has in large part been about giving management an opportunity to say "fuck you" to engineering.
Engineering payroll needed to be slashed and the C-Suite was happy to let their managers get some lashes in on the uppity workers.
The point of AI is:
1. It's bad engineering and everyone knows it.
2. Management is pushing it through anyway.
3. Deal with it, asshole.
The ROI is besides the point. It's about breaking the spirit of engineering labor with irrational decisions made outside of their control.
Edit: It is just as much about affirming the exalted status of management in comparison to labor. It's a bitter reinscription of the status hierarchy in tech; Management is good and labor is bad; Management is considerate and labor is ignorant; Management is strong and labor is weak; Management is superior and labor is inferior.
It is true that this problem can be mostly managed by the techniques we mention here. Those are actually pretty difficult to set up at scale, so many companies (including us) we only really did this in earnest once we started to see those large cost oscillations.
The main reason we shared this here is to maybe help other companies get infrastructure in place before massive cost swings rather than after.
This is how AWS made its fortune.
Initially, you picked the Multi-ZA RDS db.t3.2xlarge instance because you figured "eh i have credits anyway". Two years later, someone looks at this and says "hey, this is expensive and I bet we can do everything we need on a machine half the size". But then they think "if i downsize it and that works, i'll get a thumbs up emoji on a slack thread. If i downsize it and it causes problems, i'll draw the ire of the whole team. I better leave it alone." And the truth is... by the time your company hits the end of those credits, you're probably at the point where that savings isn't gonna do much. Or maybe you are out of business.
And that is how almost every successful company that uses AWS eventually ends up paying six-figures or more annually.
What didn't happen is any analysis of cost/benefit up front. Many of the corporate decisions around AI have seemed characterized by companies blindly copying each other.
They’re not saying they regret doing it, or that it was a mistake.
They’re just saying they’ve gained experience and have leveraged the tools to an extent their usage can be optimized.
Pretty standard business or life iteration.
If you have a 500k line codebase and even > 50% is written by agents, you are in a world of pain that won’t justify the costs longer term.
Now of course, there are products that just involve lots of code but are not actually complex. This is generally the project with like hundreds or thousands of features but most of the features are separate and don’t actually interact in complex ways. Think a task management app with hundreds of features like calendar, email integration etc. there I think agents gives you more bang for the buck. Just my thought, using agents at work.
Velocity and quality is through the roof, "10x" would be a regression at this point, the bottleneck is only figuring out what is relevant/good to deliver, the implementation is the output of the machine.
Have you ever listened to a politician talk for an hour and you can't remember what they talked about?
If some VC wanted to fund my crazy ideas for building personalized private versions of the internet because people are sick of the nightmare, I would vibe code the hell out of that; but at my current job I have existing customers to keep happy and make sure their financial data doesn't get lost or modified.
A bold claim to make with little to no supporting evidence
There’s not even a distant comparison US lab rn which is sad
I would bet my entire Polymarket balance ($0) that some military contractors have already asked AIs on the public Internet to design software for them.
agree, this works, undervalued!
look at minimal agents that protect the context window:
some thoughts on the other tips (for coding):1) stronger models are more token efficient for open ended tasks because at the limit …
that of course is difficult to benchmark, so most attention goes to simple benchmarks that show cheaper models can get similar results on 'closed' tasks with easy to 'eval' results2) dynamic request and task routing sounds great/obvious but is very very hard
using cheaper models and auto-routing do work well for 'closed' tasks where you have something repeatable and can evaluate whether a certain quality threshold is reached that you are comfortable withfor open ended coding sessions it is not so easy
that said: cheaper does not have to mean weaker, you want to look at the pareto frontier and stay up to date on new good models
there are many models like deepseek v4 flash and luna that are both cheaper and way better than most other models
One more datapoint for the thesis that OpenAI and anthropic aren’t viable, sustainable businesses, and cannot justify their $1T valuation and the level of compute commitment (reminder that OpenAI committed to >$750B in infra spending for 2030)
I don't think people thing don't think enough about the exponentially growing costs with AI. The cost of tokens, the cost of bugs, the cost of technical debt, the cost of software bloat and slowdown, the cost on the RAM market, on the environment, the costs of the little lies and hallucinations that creep into our codebases, blogs, books, and search results.
What seems like a manageable margin suddenly starts growing and destroying everything.
I just spent 3 days reviewing the code of a vibe coding maximalist at work. They needed to touch 96 front-end files, ~50 back-end files. We now have `is_uk_company` spread throughout the entire codebase every time we need to output a date format. I would say 30% of the code can be replaced by a moderately more-wise approach of "you ain't gonna need this", 20% is actively polluting and/or harmful to the coherence of the feature.
I'm pretty sure this plan came from a braindump of a verbal meeting they had with our product team, using OpenAI's top models. There were 2 initial spreadsheets that were ABSOLUTELY INDECIPHERABLE, and now we have a new one labeled "FeatureX - Phase 2 to 4".
Just the date fields is a hilarious example of why I'm skeptical. Python (babel.date) and Javascript (the browser's locale) both have locale-aware date rendering. Using a library for country-based date and time parsing would have eliminated ~12 country-specific change sites... but instead we got `is_uk` and raw date format strings (`%d/%m/%y`) in-line inside already-enormous functions. Also, at a meta level, the backend code shouldn't usually be formatting the dates (data export may be an exception), we should be sending ISO dates and having the frontend format the code if anyone gives a crap.
I'm ~$500/day and I spent 3 days reviewing this stuff. Our CTO spent ~2 days this week reviewing this feature, I'm not sure what his daily rate would end up being. Just the date-formatting part cost me half a day... that's at least $250. A few hundred dollars on Xhigh tokens burned $2500-3500 in human review effort, and I was delayed from:
1. Getting an entire sentiment analysis feature deployed by helping our interns
2. Getting a load balancer solution working to automate a manual process
3. Improving our deterministic Cypress test suite
4. Getting a demo feature launched for our sales team that would bring in more money
Did you reject the PR or is that not a career-enhancing approach in your organization?
1. Codex, Claude and others try to switch models being used at their level itself to manage the cost and outcomes
2. Now company like data bricks develops one more layer on the top of it to do the same task, of finding the base harness and applicable model
Companies like Codex and Claude are focussing/investing heavily on to ensure that people are using their harness directly or instead use APIs. Unless Databricks has some agreement in place they are violating the TOS and openly publishing an article about it. Would be interesting if openAi or Anthropic come back and claim for the API usage prices and all the savings go away.
[0]: https://support.claude.com/en/articles/9797531-what-is-the-e...
[1]: https://support.claude.com/en/articles/9266767-what-is-the-t...
Anyone using the enterprise plan are charged the API pricing, however the article is not clear if Databricks is using enterprise plan or not which is why added the following disclaimer
> Unless Databricks has some agreement in place
June 15 changes would be the keyword to check, but at least in anthropics case it's... Complicated
But overall, it's quite a task, and you really have to decide what you are prioritizing for. Do I want my agents doing lots of work, or (in my case), leaving some of my subscriptions available for me to do work.
Kudos to databricks, I also find it interesting that such different companies (Stripe, Ramp, Databricks) are all building the exact same internal tools.
I think building companies is going to look more generic in the future because intelligence is an API now.
Have you tried measuring Gemini? now that you have the router it should be a simple task. Thanks!
Of course it has. But is that a good enough measure of value? What about the business and customer outcomes from all the extra lines of code and closed JIRA tickets? Was it worth it?
Without the insight into agent performance, any changes like this feel like a gamble to save $$ at the cost of developer productivity
I'm actually working on building generic repo-specific benchmarks at https://stet.sh ;)
In large user populations like at Databricks I think the ultimate answer will come from experimentation instead of offline evals. We are already doing this in small groups, exposing them to new candidate models and then measuring per-developer cost and perceived quality changes.
However, as a self-admitted eval true believer, I remain bullish that publicly available eval capabilities will catch up to current model intelligence, and start to be able to capture more and more of the subtle components of the agent experience, such as interactivity and code maintainability. Meta and Scale have some evals in the interactivity space that seem pretty promising https://arxiv.org/abs/2606.29957 https://labs.scale.com/papers/swe-interact.
Human usability of the agent should definitely remain the top priority, but IMO we focus too much on the subjective, and not enough on the measurable/objective now, and I think/hope we'll see more companies move in the direction of Databricks
1. Thinking about how to optimize AI spend is one more thing to think about when I have too many things to think about already;
2. If there’s even a minuscule chance that me selecting down the model results in a task getting delayed because for that particular thing the model happened to be “too dumb”, I’m just not going to do that. Because I’m evaluated for what I ship and saying, “oh, not happening this week because I downgraded to Haiku for the exploration phase, and that resulted in a faulty plan, so it slipped a day” just doesn’t cut it.
3. Why should I spend time thinking about costs, and risk being slower because of that, if the next guy doesn’t?
That said, this type of post needs to be read with product marketing context in mind.
> I think the most important aspect from it is the auto-routing
On the contrary, in white paper studies auto-routing is shown to destroy the single largest token cost they found, curiously not shown in their opening graphic. As they put it in their own words near the end:
"simple tuning of… caching settings… 50% reduction in … costs, with no observed quality degradation…"
They also aren't showing the harness called ‘pi’ which generally tops results (not only with ‘open’ models), and is consistent with Anthropic's recent "works better when we don't stuff 100k system prompts into context" about Opus 5. Recent models do better with less jank trying to prescribe behavior.
So then one wonders: why is this post written to say you should use a meta router but no mention that's offset by the cache busting, and show developers cost savings but no mention that Anthropic and OpenAI both have developer-facing utilization dashes already?
Perhaps it's so engineers can show enterprise procurement why they want to buy exactly what Databricks happens to have for sale.
What’s the problem then?
Use a cheap ds4 or Luna and do the second model net net per one shot best case you save couple dollars per?
But the question remains, AI hasn't shown any explosion in growth/revenue for most companies. The effects seem muted.
If my company told me yeah we’ve decided you don’t get Fable or Opus 5 because it’s too pricey, you gotta use GLM whatever, I’d be displeased.
My workflow is very simple:
1. develop requirements for code change
2. take manual notes for implementation, maybe use LLM for some discovery/investigation
3. present notes to frontier LLM
4. develop implementation plan (bulk of work)
5. let LLM rip
6. review diff, manually fixing/refactoring code as necessary, sometimes prompting for revisions
7. get automated LLM review
8. get human review
this reliably produces the work of 2-3 pre-AI senior engineers with a lower bug rate, equivalent performance, robust edge-case consideration, etc.
Does the LLM produce over-engineered solutions? All the time. I stop it from doing that, or manually fix it myself.
Does the LLM always adhere to the best system design? No, not at all. I often have to guide its design into a better, north-star aligned one.
I don't just sit in front of my terminal and say, "Ok Claude, build the app." It is a very iterative process, and not without its potential pitfalls.
But it is very, very productive.
I think the more interesting lever is the fourth they mention: token efficiency.
> By the time costly LLM inference occurs, the user's initial statement accounts for only a negligible fraction of the data fed into the AI system, meaning costs are dominated by context the user did not explicitly include.
I think there’s still lots of low hanging fruit in regards to monitoring and improving agent work. Look at your sessions. Look at how much time and context is being spent on, say, a web search returning dozens of results when one good single-pager doc would’ve been better.
Similar idea re using the same UI across all models, but the agent can modify the harness config as well as start/schedule sessions, etc. The Kanban board feature can be used to orchestrate agent driven workflows, and the agent itself can modify the Kanban lanes. Basically the agent can do all the same things you use the UI for.
Example prompt: “schedule a session using Opus 5 with max thinking for every feature in the PRD on the canvas. For every session make the prompt instruct the agent to review end to end test coverage for the feature and create a report on the canvas for test gaps that you find. Schedule the sessions overnight tonight spread evenly throughout the night, and have each session set to retry upon token exhaustion.”
I think it’s more meta than other meta-harnesses, but I’m biased because it’s my pet project.
I haven't gotten a chance to test the multi-agent capabilities, but the DeepSeek Flash prices are so low that I probably will soon.
Have you all attempted finetuning smaller OSS models on your repos for coding?
“use price controls”
Truly revolutionary stuff.
I feel like I have an advantage over big companies, if I can use the best models on a subscription and not worry about costs much, when they can't do the same as outlined in the article.
Maintaining the cache is extremely, extremely important, so we're iterating fast but that's a major factor we track in the router's development. Couple things I'd look at:
1. The cache is generally reset after a compaction - this is the best time to make a switch if you want.
2. In many cases, the max duration of a cache is 1h, so if a session is being resumed after a long time, that's also a good time to re-assess the complexity.
We're iterating fast here and learning a lot! Definitely a lot to think about it in this area.
bad engineer + llm = bad engineer.
https://www.databricks.com/blog/introducing-omnigent-meta-ha...
https://github.com/omnigent-ai/omnigent
We're calling the layer coordinating harnesses "meta-harness'
An AI-edited post by the way.