The best AI for coding in 2025 (and what not to use)

I’ve been around technology for long enough that very little excites me, and even less surprises me. But shortly after Open AI’s ChatGPT was released, I asked it to write a WordPress plugin for my wife’s e-commerce site. When it did, and the plugin worked, I was indeed surprised.

That was the beginning of my deep exploration into chatbots and AI-assisted programming. Since then, I’ve subjected 11 large machine models (LLMs) to four real-world tests.

Also: The five biggest mistakes people make when prompting an AI

Unfortunately, not all chatbots can code alike. It’s been 18 months since that first test, and even now, five of the 10 LLMs I tested can’t create working plugins. 

In this article, I’ll show you how each LLM performed against my tests. There are two chatbots I recommend you use, but they cost $20/month. The free versions of the same chatbots do well enough that you could probably get by without paying. But the rest, whether free or paid, are not so great. I won’t risk my programming projects with them or recommend that you do until their performance improves.

I’ve written a lot about using AIs to help with programming. Unless it’s a small, simple project, like my wife’s plugin, AIs can’t write entire apps or programs. But they excel at writing a few lines and are not bad at fixing code. 

Also: I bought an iPhone 16 for its AI features, but I haven’t used them even once – here’s why

Rather than repeat everything I’ve written, go ahead and read this article: How to use ChatGPT to write code: What it can and can’t do for you.

If you want to understand my coding tests, why I’ve chosen them, and why they’re relevant to this review of the 10 LLMs, read this article: How I test an AI chatbot’s coding ability – and you can too.  

Let’s start with a comparative look at how the chatbots performed:

Next, let’s look at each chatbot individually. I’ll discuss ten chatbots, even though the above chart shows 11 LLMs. The results for GPT-4 and GPT-4o are both included in ChatGPT Plus. Ready? Let’s go.

Chatbots to avoid for programming help

I tested 11 chatbots, and six passed most of my tests. The other chatbots, including a few pitched as great for programming, each only passed one of my tests — and Microsoft’s Copilot didn’t pass any.

I’m mentioning them here because people will ask, and I did test them thoroughly. Some bots do just fine for other work, so I’ll point you to their general reviews if you’re just curious about how they function.

Meta AI

Meta AI is Facebook’s general-purpose AI. As you can see above, it failed three of our four tests. 

Also: 15 ways AI saved me time at work in 2024 – and how I plan to use it in 2025

The AI did generate a nice user interface but with zero functionality. And it did find my annoying bug, which is a fairly serious challenge. Given the specific knowledge required to find the bug, I was surprised it choked on a simple regular expression challenge. But it did.

Meta Code Llama

Meta Code Llama is Facebook’s AI designed specifically for coding help. It’s something you can download and install on your server. I tested it running on a Hugging Face AI instance.

Also: Can Meta AI code? I tested it against Llama, Gemini, and ChatGPT – it wasn’t even close

Weirdly, even though both Meta AI and Meta Code Llama choked on three of four of my tests, they choked on different problems. AIs can’t be counted on to give the same answer twice, but this result was a surprise. We’ll see if that changes over time.

Claude 3.5 Sonnet

Anthropic claims the 3.5 Sonnet version of its Claude AI chatbot is ideal for programming. After failing all but one test, I’m not so sure.

If you’re not using it for programming, Claude may be a better choice than the free version of ChatGPT. 

My ZDNET colleague Maria Diaz reports that Claude can handle uploaded files, process more words than the free version of ChatGPT, provide information roughly a year more current than GPT-3.5, and access websites.

Gemini Advanced

Gemini Advanced is Google’s $20 pro version of its Gemini (formerly Bard) chatbot. I expected the tool to do better than one out of four. Interestingly, it passed the one test that every AI other than GPT-4/4o failed — knowledge of that fairly obscure programming language produced by one programmer in Australia.

So, if it knew that language, why couldn’t it handle basic regular expressions or other first-year programming student problems?

Microsoft Copilot

You’d think the company with the “Developers! Developers! Developers!” mantra in its DNA would have an AI that does better on the programming tests. Microsoft produces some of the best coding tools on the planet. And yet, Copilot did badly.

Also: What are Microsoft’s different Copilots? Here are the differences and how you can use them

The one positive thing is that Microsoft always learns from its mistakes. So, I’ll check back later and see if this result improves.  

But I like [insert name here]. Does this mean I have to use a different chatbot?

Probably not. I’ve limited my tests to day-to-day programming tasks. None of the bots has been asked to talk like a pirate, write prose, or draw a picture. In the same way we use different productivity tools to accomplish specific tasks, feel free to choose the AI that helps you complete the task at hand. 

The only issue is if you’re on a budget and are paying for a pro version. Then, find the AI that does most of what you want, so you don’t have to pay for too many AI add-ons.

It’s only a matter of time

The results of my tests were fairly surprising, especially given the big investments of Microsoft and Google. But this area of innovation is improving at warp speed, so we’ll be back with updated tests and results over time. Stay tuned.

Have you used any of these AI chatbots for programming? What has your experience been? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Keep your entertainment at your fingertips with the Amazon Fire TV Stick 4K! Enjoy streaming in 4K Ultra HD with access to top services like Netflix, Prime Video, Disney+, and more. With an easy-to-use interface and voice remote, it’s the ultimate streaming device, now at only $21.99 — that’s 56% off!

With a 4.7/5-star rating from 43,582 reviews and 10K+ bought in the past month, it’s a top choice for home entertainment! Buy Now for $21.99 on Amazon!