Disclosure: The company I work for is very heavily invested in generative AI, specifically developer tools. As a result, I have financially benefited from recent AI popularity and I’ve had involvement with building software products that incorporate modern generative AI. These are my opinions, but my closeness to AI has surely influenced these opinions.
I’ve been seeing a lot of dread and fear from my peers about generative AI and the worries that it’s a threat to our industry and our livelihoods.
If this is a thing that keeps you up at night, let me give you a few salient points that might help put your mind at ease:
- Have you ever actually tried leaning on generative AI to do anything nontrivial for programming? It largely can’t, and it certainly isn’t going to help you with big refactorings on large codebases because the context window would need to be way too large.
- Have you tried troubleshooting something complicated by relying only on the AI? AI can’t troubleshoot because it can’t interact with the system it’s allegedly helping you write. It can’t go too far past platitudes. They’re somewhat personalized platitudes, sure, but writing software is a discipline where you get backed into super specific corners.
- Have you tried using an LLM-based tool with your large codebase to ask questions about the codebase itself? It gets hit and miss really fast.
- Have you tried asking an LLM to do anything remotely creative? It falls flat.
- Have you ever asked the LLM for a code sample on how to accomplish something, only to find that it generates code that references APIs that don’t actually exist but sound plausible? That’s not really a bug; it’s core to how LLMs are designed; they don’t have a notion of whether APIs exist.
- Have you tried leaning on AI tools to make your writing great? At best you get text with this milquetoast voice and sameness to it. It’s forgettable and completely lacking in character.
LLMs as we know them today aren’t capable of logical reasoning (at best, we have future ones that pretend they can reason about things). They’re incapable of interacting with the outside world as part of their operation. All they do is receive input and generate statistically plausible sounding output using that input plus their training data.
And these deficiencies of LLMs aren’t mere kinks that can be ironed out. Even if you added several orders of magnitude more tokens to prompts and could feed it your entire company’s code and the code for all of your dependencies the LLM would still fall short because all that extra context still isn’t helping the LLM understand what it’s helping you write because it doesn’t understand anything in the first place.
To build an AI that can make software would require building a fundamentally different set of tools altogether.
When ChatGPT came onto the scene a couple years ago it felt like it came out of left field, but ChatGPT is actually the result of decades of incremental research. It feels like it’s evolved fast in the last 2 years but that’s with billions being poured into improving it, and at the end of the day ChatGPT and its peers are all still fundamentally LLMs, and–say it with me: all LLMs do is receive input and use that input with their training to generate statistically plausible output.
Now, that’s not useless! LLMs can be really handy while programming. They can be great at helping you start a project. You can ask them high-level questions to help you learn new things (assuming that you’re asking about something it has a lot of training data for). They can often cut through the toil of certain tasks, like writing tests.
But the moment you try to really lean hard on the LLM to help you with something (programming or not), you realize the LLM’s “knowledge” is ultimately very shallow, and they get really hand-wavy as soon as you’re asking about anything nontrivial.
I understand why LLMs feel scary. They make for remarkable demos. We’ve been conditioned for decades to think of computers as being really rigid and picky about how you communicate with them, and ChatGPT suddenly comes along and breaks that assumption with uncannily human-seeming responses. But the corollary of that is that we also assume the output of ChatGPT is highly reliable and consistent like a computer but it’s not; it’s giving output that’s just as loosey-goosey as the input we get to give it. It looks plausible and in many cases it will give factually correct output, but it’s never correct because it knew anything; the math just worked out in its favor.
You can’t do knowledge work without knowledge.
I’m not suggesting we be complacent. Complacent industries get disrupted all the time. But so much of the fear I see from engineers about generative AI is based on this assumption that it’s capable of things it can’t actually do. If you look past the hype and try to really understand what today’s AI can can’t can’t do, you’ll realize that AI’s about as capable of stealing your engineering job as Alexa or Siri.
A much bigger worry for me (and something that we saw happening in ’22–23) is companies getting into a panic about their spending and deciding to lay off engineers, usually saying they’re overstaffed. The companies probably aren’t actually overstaffed, but when you fire people you see those cash savings on your balance sheet instantly, and you won’t actually start seeing the effects of firing a chunk of your team until months or years down the road, and leadership doesn’t really learn anything.
Software engineers are one of the only roles who have enjoyed decent leverage negotiating compensation in the last 20 years, and companies resent them for that.
Companies can lay off their engineers regardless of whether LLMs can help fill in the gap or not (and they do), but reality eventually catches up with you.
Leave a Reply