The hidden costs of AI (and who ends up paying them)

Listen on:

AI budgets are escalating rapidly, yet few teams measure the value of their spend – and who ends up paying, anyway? Steve Jones, Grant Fritchey, Kellyn Gorman, and Pat Wright discuss that and more. Also on the agenda: LLMs, data centers, regulation, and what happens to the next generation of engineers if they never learn to solve problems the hard way…

Listen or watch to every episode of the Redgate Simple Talk podcast at 
https://www.red-gate.com/simple-talk/podcasts/

Don’t have time to listen in full? Here are the 10 key takeaways from the episode.

The real cost of AI: 10 key takeaways

1. AI budgets have a way of landing on the database team

A fully cloud-embedded PostgreSQL shop pushed hard into AI from 2023 and, when total AI spend eventually outgrew the budget, nobody asked for less AI. Instead, the database team was asked to find savings to pay for it. Here, the cost sits in the cloud – not in PostgreSQL licensing. Nothing is free.

2. The “just use it” era is ending

Many companies are told to move as fast as possible on AI, so maximize discounts and free AI token allowances until they run out. After that, the monthly bill becomes painful.

3. Token counts are a poor measure of productivity

Some organizations ‘gamify’ token usage, treating it as a proxy for how much work people get done. Then comes the whiplash: “stop using so many tokens, it’s too expensive!” The highest token users at Redgate are sales reps (and it’s not even close) – proving that heavy AI token usage doesn’t map neatly onto any one role.

4. Few teams weigh token spend against value

The Advocates rarely see anyone deciding that a task is worth what it costs in tokens. For example, are AI-generated daily reports – costing about 80 cents each – worth keeping? Save AI for work it can genuinely accelerate, and use a plain search for quick questions.

5. If you can’t show the return, consider a simpler approach

Company executives increasingly want to see ROI (return-on-investment). If a feature isn’t innovative, build it the traditional way – deterministic or rule-based. If you’re just shifting your development cost onto the customer, you’ve already failed.

6. Large language models (LLMs) may not be the only answer

The Advocates expect wider use of deterministic and machine learning (ML) approaches. One example: an open-source project that reportedly cut its daily cost from $19,000 to under three cents. The Advocates are also skeptical of tightly-controlled vendor demos, and note that small and medium language models are already common. Making anything truly deterministic is a bit like training a junior colleague: it takes effort.

7. Cost pressure is really about people

Many executives think they can only pay for AI by cutting staff. Spending an extra half million on AI in the hope of doing ten million dollars more work is a hard sell when most companies struggle to sell the work they already have.

8. Data centers and regulation are a thorny mix

Fear-driven arguments may lead to tighter regulations that lock-in today’s big players and squeezes out newcomers. Large data centers create real externalities, such as power demand, that others end up paying for. And infrastructure investment has a history of skipping rural communities. Regulation is important but must be executed carefully.

9. AI has real upside where traditional methods fall short

AI has enhanced medical research, including simulating tests in areas where women have historically been under-studied, as well as work protecting animals and the environment. However, the upsides can’t be separated from the downsides – including costs that could put AI tools out of reach for students and researchers.

10. The skills gap is the hidden cost

Those who lean on AI without learning the fundamentals could be setting themselves up for a skills/knowledge gap in five or six years’ time. Mathematicians, for example, who object to AI solving problems because working through them is how people learn. These skills gaps may be filled, but how they’re filled is yet to be determined.

There’s also the view that curiosity and judgment are human problems, not age problems, and that AI can either erode or sharpen them depending on how you use it. One idea for building good habits: a hackathon where you can use AI, but only within a budget.