For a long time, I’ve wondered how AI could better enable software developers to do their jobs.
2023 has been a pivotal year for AI in software development, with tools such as GitHub’s Copilot evolving to help developers build applications.
For at least a decade I have intermittently posed that question to my tech buddy Grady Booch, chief scientist for software engineering at IBM Research and an IBM Fellow. Early on, he said he believed there would be uses for AI in software development, but he was skeptical of how much.
And he remains so. Late last year, Booch had a Twitter (now X) confrontation with Amjad Masad, CEO of Replit, about what AI could bring to developers. Masad wrote that as AI models progressed, at some point, “Everyone in the world will be at least John Carmack-level software capable.” Carmack is a renowned game developer (co-creator of Doom and other games) and AR/VR guru who was former CTO of Oculus VR and former consulting CTO at Meta after its acquisition of Oculus.
Well, we’re nowhere near that point yet. But 2023 proved to be huge in terms of AI for developer productivity.
“2023 is indeed a seminal year for AI-assisted coding and software development,” said Lian Jye Su, chief analyst for applied intelligence at Omdia, via email to The New Stack. “Copilot and other generative AI tools have demonstrated a high degree of accuracy, automation, and, most importantly, flexibility when used as a developer tool. Developers can use natural language to express their intent, and focus on the creative and evaluation aspects as the tools take care of the mundane part.
“As such, some developers generate nearly 40% of their code using these tools. Aside from Copilot, other key tools include OpenAI Codex, Replit, Tabnine, Codacy, and Durable.”
A recent JetBrains study on the state of the developer ecosystem showed that given the opportunity, 56% of respondents would let an AI assistant write code comments and documentation.
The Evolution of Copilot
Meanwhile, 46% of the survey respondents said they use Copilot.
GitHub launched Copilot, its code auto-completion/pair programming tool, in June 2022. It now has more than 1.5 million users, Inbal Shani, GitHub’s chief product officer, told The New Stack in an interview. Moreover, the percentage of new code written using Copilot has grown from 35% to 60% and is expected to reach 80-90% in the coming years, she said.
AI-powered developer tools like GitHub Copilot are seeing massive growth and having a transformative impact across software development, with positive effects on productivity, quality and job satisfaction, Shani noted.
The adoption of AI tools by developers has grown exponentially. Ninety-two percent of developers now use some form of AI in their work, according to GitHub. This is largely due to the rise of GenAI models that have democratized access to AI.
AI-assisted tools like GitHub Copilot can now generate full code blocks, explanations of code, documentation and answers to developer questions. This helps onboard new developers more quickly, Amanda Silver, corporate vice president of product, design, user research, and engineering systems, in Microsoft’s developer division, told The New Stack in a video interview.
In addition, Copilot can optimize and refactor code, troubleshoot issues, suggest security fixes, aid debugging, and more – thus reducing the need for deep expertise in multiple domains, she explained.
Internally, Microsoft engineers would “dogfood” Copilot features as they built their own products, providing feedback to GitHub to improve the technology. Indeed, Microsoft engineers used Copilot to perform mass migrations of internal systems, which also helped test its bulk refactoring abilities, Silver said.
Whispering to Code
While Amazon also introduced its Copilot competitor, CodeWhisperer, last year, much like Copilot and other tools, 2023 saw it emerge as a tool for everyday use. Amazon CodeWhisperer is a machine learning (ML)-powered service that helps improve developer productivity by providing code recommendations based on developers’ natural comments and prior code.
Amazon Web Services had a big year in generative AI, with the launch of CodeWhisperer in generation availability, Bedrock for building custom AI apps, and PartyRock, currently in preview, for no-code AI app building. At re:Invent in late fall, the company also announced Amazon Q, a GenAI assistant that is specifically for work and can be tailored to an organization’s business.
CodeWhisperer has shown 50-60% developer productivity gains in Amazon studies and features like customization boost it further, Doug Seven, director of software development and general manager of Amazon CodeWhisperer and Amazon Q at AWS, told The New Stack in an interview.
“Amazon is competitively positioned in AI given its work across the stack from chips optimized for AI workloads up to developer tools leveraging AI,” Seven said.
The Field at Large
Up until 2023, for the entire history of digital computing, the core activity of programming has been a human being writing lines of code. Some AI tools had come along to augment this capability, but human coding has always been at the center.
That’s changed, Jason Bloomberg, an analyst at Intellyx, told The New Stack via email. “Today, the central programming activity moving forward is having AI (generative AI and neural networks, primarily) create the applications,” he said. “Instead of coding, humans create and manage the models and the data sets that train the AI.”
Hand coding still takes place, of course — but Bloomberg said he believes it has moved into a secondary position. “Any organization, vendor or enterprise, who doesn’t realize this point is doomed to becoming less competitive and eventually irrelevant,” he said.
Indeed, as GitHub’s Shani said, “AI is now table stakes for software development.” And GitHub is looking to cover more of the overall software development life cycle (SDLC) with its tools.
“Obviously, Copilot stands out because of time to market and volume, but other tools like Tabnine, CodeStory, Codium, etc., are all expanding” the GenAI space, said Thomas Murphy, an analyst at Gartner, via email. Cloud providers, in addition to AWS and Google, such as GitLab and Atlassian also are expanding into and beyond code generation and broader AI assist features for the SDLC, he said.
JetBrains is another prominent dev toolmaker that has been building its own AI assistant feature and released it in its latest product updates, focused primarily on the flagship IntelliJ IDEA integrated development environment (IDE). The company’s new AI Assistant plugin introduces different capabilities — it can “suggest code refactoring at a higher, more abstract level rather than just fixing specific patterns,” Matt Ellis, a JetBrains developer advocate, told The New Stack.
For the future, Ellis said, the company wants to improve the assistant to make it “smarter” so it can better understand code context using its existing internal code indexes and metadata and explore enterprise hosting options for its models.
“Tools like Copilot are good at local code suggestions but lack application context,” Elizabeth Lawler, CEO of AppMap, told The New Stack. “New players could integrate observability data, architectural info, etc. to make recommendations more accurate.”
However, 2023 made AI infrastructure and models accessible enough that now smaller companies can leverage customizable AI, she said. This will enable new value-added services, Lawler noted. Moreover, she added, quality issues still exist with AI-generated code, but it forces developers to critically evaluate recommendations.
Meanwhile, from a venture capital perspective, Nick Adams, founder and managing partner at Differential Ventures, said he is seeing opportunities for companies to use GenAI to assist business analysts who often write repetitive queries to extract data.
“AI tools can help by taking natural language questions from business users, generating a query in SQL or Python as a starting point, and allowing analysts to refine the code,” Adams told The New Stack. “This speeds up the process and avoids writing queries from scratch each time.”
Migrating legacy code like COBOL is another opportunity, he noted. This is something IBM is doing with its watsonx GenAI effort to migrate old COBOL code to Java.
“Lots of old systems are near retirement written by now-retiring developers,” Adams said. “Electrical engineering also has a lot of soon-to-retire expertise built up over decades that will need to transfer to a new generation with modern tools and training.”
GenAI and Low Code
I used to think GenAI would somehow subsume low-code, no-code development, but Don Schuerman, CTO and vice president of product marketing at Pegasystems, set me straight.
GenAI is having a big impact on the software industry, including low-code platforms, because tools that auto-generate code could disrupt the low end of the market.
“However, enterprise-scale low-code is more focused on architecture, integration, DevOps, security, reusability, etc. This complexity isn’t just about generating code snippets,” Schuerman told The New Stack.
Pegasystems is using GenAI to auto-generate low-code artifacts to accelerate developers, e.g. standard workflow templates, test data, process documentation, data mapping and more.
Amazon’s Seven said that while CodeWhisperer assists professional developers who work in an IDE, Party Rock is aimed at other groups, such as subject matter experts, to build apps with little or no code.
“I don’t think GenAI will make low code, no code obsolete,” he said. “I think it will make low code better.”
Addressing the issue of whether AI will replace developers, Schuerman said, “I don’t see generative AI fully replacing developers. It will make them more productive by providing starting points and suggestions. Developers rarely write everything completely from scratch anyway.”
The JetBrains study showed that 60% of respondents said they think AI coding tools will radically change the job market, and 51% said they think these tools will increase demand for professional software developers. Yet, there is a consensus that AI will never fully take over the job of writing code for developers.
The Year of the Studio
In November, Microsoft launched a preview of its Azure AI Studio, a new platform that empowers developers of all abilities and preferences to innovate with AI and to explore, build, test, and deploy using the latest AI tools and ML models, grounded in responsible AI practices, John Montgomery, corporate vice president of program management, AI patform at Microsoft, told The New Stack.
“We believe AI is the ultimate amplifier,” he said. “This latest shift to generative AI has completely changed the world. It’s a game-changer for applications. I’ve been at Microsoft for 25 years; I’ve never seen customers adopt a technology this quickly. We have more than 18,000 customers that are using Azure OpenAI in the year, less than a year, that it’s been available to make their own stuff.”
As one example: Instacart is using GPT-4 Turbo with Vision to enable their users to take a picture of a handwritten shopping list and to generate a virtual shopping list in their app, he said.
“You can think of AI studio as the ultimate toolkit for generative AI,” Montgomery said. “It brings together the right models, data and responsible AI systems so that customers can be confident their solutions will be safe, secure and scalable.”
Microsoft announced support for its own latest large language models (LLMs), as well as the latest models from OpenAI, Meta, Nvidia, Mistral AI and Hugging Face.
In addition, the company announced Azure AI models as a service in the form of Azure AI Services, “which is going to make it even easier for customers to deploy and run cutting-edge models,” Montgomery said. And Microsoft also launched its prompt engineering tooling called prompt flow. Prompt flow is an advanced prompt engineering, evaluation and deployment system, which is a key part of Azure AI Studio and Azure Machine Learning.
Despite OpenAI’s importance in this space and its close partnership with Microsoft, a company spokesman told The New Stack that the team was unavailable for an interview.
Meanwhile, not to be outdone, Google recently announced Google AI Studio. Google AI Studio is a free tool that enables developers to quickly develop prompts and then get an API key to use in their app development. It’s free to use right now, within limits, and it will be competitively priced, Google said.
During a press briefing, Josh Woodward, vice president of Google Labs, demonstrated how to sign into Google AI Studio with a Google account and take advantage of the free quota that allows 60 requests per minute. Woodward also demonstrated how a developer can simply click on “Get code” to transfer their work to their IDE of choice.
Internally, Google has been dogfooding its AI technology for code completion, documentation, chatbots and a lot more, Jeanine Banks, vice president and general manager, Developer X and DevRel at Google, told The New Stack. “Applying this technology internally,” she said, “really gives us a lot of learnings” that the company can bring back into the products.
YOUTUBE.COM/THENEWSTACK
Tech moves fast, don’t miss an episode. Subscribe to our YouTube
channel to stream all our podcasts, interviews, demos, and more.
