Delange said that open source language models are improving rapidly and can be better than OpenAI’s market-leading GPT-4 for some specialized tasks. But he noted that many of the best open source models have come from outside the US, saying that 01.AI could be positioned to benefit from innovations that spring up around its model. “US companies have become a little bit less open and transparent,” he said at the briefing. “But there’s this interesting dynamic with AI where the more a company releases open source, the more the ecosystem develops, and so the stronger they become at building AI.”
Meta’s Llama 2 is a rare example of a top open source model from a US company and is the social media giant’s challenge to OpenAI, Microsoft, Google, and other major tech rivals investing heavily in generative AI. Meta chose to release its AI language model under a license that allows commercial reuse, with some caveats.
Yi-34B and Llama 2 appear to have more in common than just being leading open source AI models. Not long after the Chinese model was released, some developers noticed that 01.AI’s code had previously included mentions of Meta’s model that were later removed. Richard Lin, 01.AI’s head of open source, later said that the company would revert the changes, and the company has credited Llama 2 for part of the architecture for Yi-34B. Like all leading language models, 01.AI’s is based on the “transformer” architecture first developed by Google researchers in 2017, and the Chinese company derived that component from Llama 2. Anita Huang, a spokeswoman for 01.AI, says a legal expert consulted by the company said that Yi-34B is not subject to Llama 2’s license. Meta did not respond to a request for comment.
Whatever the extent to which Yi-34B borrows from Llama 2, the Chinese model functions very differently because of the data it has been fed. “Yi shares Llama’s architecture but its training is completely different—and significantly better,” says Eric Hartford, an AI researcher at Abacus.AI who follows open source AI projects. “They are completely different.”
The connection with Meta’s Llama 2 is an example of how despite Lee’s confidence in China’s AI expertise it is currently following America’s lead in generative AI. Jeffrey Ding, an assistant professor at George Washington University who studies China’s AI scene, says that although Chinese researchers have released dozens of large language models, the industry as a whole still lags behind the US.
“Western companies gained a significant advantage in large language model development because they could leverage public releases to test out issues, get user feedback, and build interest around new models,” he says. Ding and others have argued that Chinese AI companies face stronger regulatory and economic headwinds than their US counterparts.
Speaking at the World Economic Forum in Davos last week, Lee argued—perhaps hoping the message would travel back home—that the open approach would be crucial for any country to take full advantage of AI.
“One of the issues with one or a few companies having all the most power and dominating the models is that it creates tremendous inequality, and not just with people who are less wealthy and less wealthy countries, but also professor researchers, students, entrepreneurs, hobbyists,” Lee said. “If there were not open source, what would they do to learn; because they might be the next creator, inventor, or developer of applications.”
If he’s right, 01.AI’s technology—and applications built on top of it—will put Chinese technology at the heart of the next phase of the tech industry.
Source
