This year has been the breakout year for Large Language Models (LLMs). But developers are still at the early stages of working out how best to leverage their power.

When I was told that a potential interview candidate had been caught using ChatGPT while answering web-based questions, I had mixed feelings. It felt a little like when students first had access to hand held calculators. Today, that is no longer considered a problem in math exams — they can be used freely. Ultimately, arithmetic is just a small part of mathematics, and a calculator is definitely just a tool. Yet using ChatGPT to supplant your own skills and knowledge seems both unnecessary and forlorn. But the evidence this year is that for developers, AI hits hardest in the form of tools or assistants.

The heart and soul of the LLM is that first word: Large. Only when a lot of data has been ingested do they perform so surprisingly well. My simple attempt to use similar techniques but without Neural Network reinforcement did not produce any decent poetry, but did show the process of ingesting a corpus of text.

Image via Stable Diffusion; prompt: “a massive robot shooting lasers out of its eyes, with everyone running in terror”

ChatGPT’s ability to consolidate information in response to a query is golden. (Although for answering many queries, you might find Perplexity.ai a better experience) And many developers have been charmed by the way they can quickly get a fully explained code example into their projects very quickly. But their strength as tools is when they can increase the efficiency of the developer with the least friction.

AI Developer Tooling

This year has largely been about developer tooling, usually including AI. Rust has helped with building faster full screen tools, with both Zed and Warp as strong examples. Zed is a fully featured “multiplayer” editor, built for speed. Zed2 should be coming out soon, and I note their work on Tailwind completions. Warp continues to innovate with the command line.

AI adoption in tooling has been central to many efforts. The first big waves were from Copilot in Visual Studio. I also looked at Ghostwriter for Replit, as well as CodiumAI’s test generator. Cursor had an AI-first editor.

Many of these are doing no more than AI wrapper requests for code examples. But Copilot was able to complete a class method seeing just a signature, if the method name was reasonably canonical. I’ve been more impressed with features like this that enter code directly within the code window itself, not text in a separate window. Right now Microsoft seems to have the better direction, but that will change as other projects mature.

CodiumAI’s test generator definitely proved that AI can help directly in the development cycle, by producing sensible unit tests based on existing coded methods. There are plenty of ancillary tasks around coding that AI can serve.

But What about the Full Development Cycle?

This is where we need to address the different needs of the code dabbler and the professional developer, and why the current designs don’t quite work for either. My honest opinion is that the full development cycle is still too tricky to truly be accessible to the dabbler and AI tools don’t change this yet.

Remember, one missing quote can still render an entire file of code uncompilable. But on the other hand, people who code every day are looking for lots of small continuing assists, not large chunks of code from the web. The wisdom for these can be derived by LLMs, but it is iron control of the editor interface that is vital. If it throws me out of my flow, no advice is useful.

This year we saw platforms that help developers work with LLMs indirectly. I think it is too early to talk much about them, given that further advances from OpenAI can easily undermine their utility. One example I got to grips with is Fixie, which uses an agent approach to leverage LLMs.

Should Devs Be Worried about Their Jobs?

So what about LLM tools writing entire projects? Is the developer’s job at risk? Do we all have to cheat now?

The developer has two skills that they use daily to progress work: making connections, and understanding transitions. These are hardly inaccessible to LLMs, but for now, these remain specifically human traits.

Project transitions need the understanding of the staff, the organization’s financial position, the business environment, etc. Changing from a relational database to a Redis key store and then to something new on the cloud are changes ChatGPT could recommend, in theory, but no one would trust that opinion entirely — after all, you can’t sack or demote a website.

Making connections is something that requires daily observational living, but is not technically beyond AI. The WIMP visual interface from Xerox PARC came about because the desktop metaphor came into the designer’s mind. Right now, LLMs are mainly responsive in nature — they don’t suddenly have ideas in the bathtub.

I have a little sympathy for people who think that AGI and calamity are just around the corner, simply because the strength of LLMs was such a surprise. But in reality, there is little evidence for this. Pattern recognition is very important for us because it is how we navigate our world. But language comprehension in silicon is just a tool for us. As Professor Michael Wooldridge says: “It’s not aware of the world in any meaningful sense.”

I’ll write about what we might see trending next year in my next post.

David has been a London-based professional software developer with Oracle Corp. and British Telecom, and a consultant helping teams work in a more agile fashion. He wrote a book on UI design and has been writing technical articles ever since….

Read more from David Eastman