When Meta released Llama 2, a powerful artificial intelligence model similar to the one behind ChatGPT, last month, it made it possible for developers, startups, and researchers to play with the kind of AI that has enthralled the world for nearly a year.
Today, Meta is following up with the release of Code Llama, a version of the model that has been tuned for programming tasks. The release could mean more developers getting a taste of AI-assisted coding. It could also inspire new ways of embedding AI into software. And it could help further establish Meta as the preeminent supplier of “open” AI tools.
“It’s exciting that they’re releasing the weights to the community,” says Deepak Kumar, a postdoctoral researcher at Stanford who has studied AI coding, referring to the parameters of the neural network at the core of the model.
Kumar says the release of Meta’s regular language model Llama 2 led to the formation of communities dedicated to discussing how it behaves and how it can be modified. “It gives us a little bit more flexibility to play with what exactly is going on under the hood, compared to these closed-source models from Google or OpenAI.”
Kumar says developers are likely to build new kinds of applications using Code Llama. For example, it could be possible to create a programming assistant that performs various additional safety checks before recommending a chunk of code, says Kumar, whose own research has explored how AI assistance can sometimes lead to less secure code. Kumar adds that the release could inspire the creation of assistants specialized for particular kinds of coding.“You can build all sorts of tooling on top of the model,” he says.
Talia Ringer, an assistant professor at the University of Illinois Urbana-Champaign who researches programming, says that Code Llama will be valuable for academic research. I already have students using Llama models for research, and I could see those students being extra excited about a code model given the nature of our work,” she says. But Ringer adds that, ideally, the data used for training would also be released. “That’s often the missing piece for making sense of research on LLMs,” she says.
Programming is one area in which recent advances in AI have already had a considerable impact.
In May 2021, GitHub, a subsidiary of Microsoft, launched Copilot, a plug-in for coding programs that auto-completes sections of code based on the first line or a comment typed by the user. Copilot uses a version of Open AI’s GPT, the large language model behind ChatGPT. That model is trained further using code that GitHub stores for developers, as well as, reportedly, by contractors who are paid to annotate their own code.
GitHub faces a lawsuit for using some open source code in its training data, and Masad says Meta is likely to have limited the training data to avoid such complications. Copilot costs $10 per month for individuals and $19 per month, per user, for businesses.
