Is AI making our code stupid?

Coding tools like GitHub Copilot can help developers author code 55% faster, according to GitHub. Hurray! Right? Well, maybe not. GitClear analyzed 153 million lines of changed code between January 2020 and December 2023 and now expects that code churn (“the percentage of lines that are reverted or updated less than two weeks after being authored”) will double in 2024. In other words, “Code generated during 2023 … resembles [that of] an itinerant contributor,” as if you hired a short-term contractor more concerned with cranking out verbose code than ensuring its maintainability. 

Bloated code creates security nightmares, among other issues, as Bert Hubert captures. This isn’t to suggest that developers shouldn’t use AI assistants. Rather, it’s an indication that care must be taken to ensure over-reliance on AI doesn’t make our machines, and the code that runs them, stupid.

Everyone is doing it

There wouldn’t be much need for concern had generative AI (genAI) not exploded into use last year. With that adoption comes both good and bad, as Datasette creator Simon Willison highlights. “In the past 24–36 months,” he writes, “our species has discovered that you can take a giant corpus of text, run it through a pile of GPUs, and use it to create a fascinating new kind of software.” That software can significantly improve our lives, he continues, if we learn to master it. Fortunately, large language models (LLMs) are “actually quite easy to build,” requiring just a few hundred lines of Python code. 

The tricky part of LLMs is not the code used to write them, but rather the data used to train them. In 2023 we learned new ways to keep them from going off the rails (“hallucinating”) with retrieval-augmented generation (RAG), and other means. That’s the good.

On the bad side of the ledger, we’re also realizing just how flaky they can be. After all, knowing how to prompt an LLM is still a dark art (with no consistency). As Wilison notes, “The things you sometimes have to do to get the models to behave are often incredibly dumb.” This is less true of creating software than of, say, asking ChatGPT to write a term paper. But it’s still true that getting an LLM to do any particular work is an exercise in fiddling with knobs (prompts) to get what you want. 

The worst part of AI-generated software—and of today’s software in general—is bloat. “The more [code] you have … the more risks you run,” argues Hubert. In some ways, we’ve tried to route around this by making it someone else’s problem to run our software. “Software is now (rightfully) considered so dangerous that we tell everyone not to run it themselves,” says Hubert. The key is to write less code. Leaner software is safer software. “The world ships too much code,” Hubert declares, “most of it by third parties, sometimes unintended, most of it uninspected. Because of this, there is a huge attack surface full of mediocre code.” 

Copyright © 2024 IDG Communications, Inc.

Source :

Leave a Comment

SMM Panel PDF Kitap indir