The Evangelist and the Critic

Transcript

This is the first higher Ed hot take, and boy is it a doozy. I'm your host, Joel Goodman.

Here we are, talking about AI again. There's so much blind trust in these products floating around higher ed, and I have felt the need to bring a critical lens to assessing these things. I’ve been saying it a lot lately — I’m a critic, not a skeptic. From here on out, I’m going to be using the term LLM because the way we use AI today is purely as a marketing term taken from another technology and slapped onto machine learning by Sam Altman and OpenAI.

What you use every day and call AI is still just machine learning, and we've had that for decades. And it's very useful, but doesn't sound as flash when you decouple it from the chat products you've been duped into believing is AI.

Did you know that OpenAI lost $5 billion after revenue in 2024? The company that owns ChatGPT and powers most of the other products your institution is probably paying five- or six-figure sums for lost $5 billion, even with all the revenue they bring from ChatGPT subscriptions and API contracts. According to financial reporting, it costs OpenAI $1 billion more than all of its revenue to run its software.

This is a problem. OpenAI loses money on every call to its API, every question you ask ChatGPT, and every run of a custom GPT you make. Worse still, they lose more money the more customers they acquire.

And the same is true for Anthropic (the company behind Claude). That means that the price you’re paying for LLM-powered software is well below the cost it takes to actually run it — somewhere around 125% less than the actual costs to run them. These products have been backed by venture capital based on little more than smoke and mirrors.

With news that Microsoft (who runs the datacenters powering OpenAI’s GPTs) is pulling back on its datacenter build-outs, there is a very real chance this bubble we’ve been ignoring is about to burst.

If I could generalize a bit, there are two kinds of AI thought leaders and only one gets listened to.

First, there’s the Evangelist who knows enough about how to use these systems and the general framework for how the models are constructed. The evangelists are all-in either because they’re super optimistic, or super opportunistic.

Either they are happy to ignore all the warning signs and continue hyping ChatGPT as a solution despite the very real possibility reality with catch up with them, or they see the potential for a short-term cash return and are pretty cynical about society. If they can get rich off of your lack of knowledge, they will.

AI evangelists fall into two camps. There are the ones who started or shifted their businesses to take advantage of this new wave of LLM dominance. They built chat interfaces, CRMs, code and search frontends that are wrapped around OpenAI or Anthropic. They feed your data back to these companies and have pinned their business to this bubble. They don’t actually own the tech. They rent GPU cycles and create marketing buzz.

Then there are the so-called experts who played with the early ChatGPT web app and saw an opportunity to own a thought leadership lane. You’ve seen them at conferences trying to help you figure out how to use LLMs in your own workflows. We’ve all learned a ton from them. Maybe they’ve written a book about “AI” already. They are genuinely trying to help us stay ahead of the curve.

Then there's AI critics, and this is me. I've been using these tools longer than most of the evangelists.

I'm always an early adopter and I'm always critical. When I lived in Austin and was plugged into the startup scene, I used to tell people I'd be a terrible VC because I thought everyone's ideas were bad. I'm a critic and I do keep a watchful eye.

Critics aren't skeptical though. We count data scientists, machine learning engineers, marketers, journalists, and plenty of others among our ranks.

We don't believe the hype, but we test it against reality. If the bubble is about to burst, we need to sound the alarm.

So on to the problem. First, do you really think that chatting with a computer is a good experience and super useful as a general, every day activity? Like, the most efficient way to interact with LLMs is to spend brainpower trying to figure out how to prompt it best and physically typing that into a chat box?

We've done that before. It's called programming and it's why we have graphical user interfaces. BECAUSE IT'S FASTER TO USE AN INTERFACE.

Like, I get it for doing deep research, but the chat part of ChatGPT is not the product. In fact, LLMs shouldn’t be the product, even though they’re sold like they are. LLMs should be part of the infrastructure making products better, and the best all of these supposedly brilliant people can come up with is chat.

Get real. Come on. I mean, really.

Meanwhile, university websites are still slow and institutions are adding more AI chatbots and search and RFI forms that pop up at supposedly just the right time. It's, it's just junk. It's just junk that slows down our website and makes them worse to use while search engines are eating our traffic for breakfast. You've seen those AI summaries, right?

People aren't getting to your websites anymore and you're making them heavier and making it harder so that the ones that even get there don't use them.

What are we doing? Seriously. How is any of this strategic? Your strategy is to skip past doing the actual work and expect half-baked, consistently hallucinating predictive algorithms to make everything better? Seriously?!

This is me saying it louder for everyone. AI tools are not a replacement for content strategy. They're not a replacement for UX design. They're not a replacement for brand strategy. AI tools are not a replacement for admissions counselors and they're not a replacement for human kindness.

But we're still out here paying lip service to the actions we know we should be taking while dumping cash we don't have into middleware that ruins the environment, guts the arts, and steals from the very people that we educate. How do you square that with the mission of higher education?

So why are you spending money on chatbots and targeted ad-style contact forms? Why are you giving money to a company that will either fail bringing an entire industry down with it, or raise its prices so high you'll never be able to recover?

I'm begging you to pause and think bigger about LLMs, think bigger about what value they have, but also think more sustainably.

What I've taken away from the AI craze is this: The billions and billions of dollars set on fire by these companies and the possibly irreparable damage to the planet that they've done have brought the promise of big data analysis to our personal computers.

And when you clear past the hype and marketing and evangelists, this is an incremental gain.

The biggest advance is that every institution now has the power to analyze all of its data, find trends, do market analysis, and reduce the workload of its already overworked staff.

The bloated, expensive operational side of running a college or university can be made more efficient, freeing up the messy humanity needed side of the house to provide better care to your prospective students and donors. You know what doesn't do that? Chat bots, especially ones that give the wrong answers, site search that nobody uses—6% of your traffic and most of that's internal—and hard sells that cover up broken UX.

There are other ways. There are smarter, more effective ways to implement this stuff. And right now, can you really afford to be betting the house on someone else's failing business?