OpenAI boss Sam Altman admitted a computer chip shortage is hindering ChatGPT’s progress during an off-the-record meeting in London last month – the details of which surfaced after one attendee mistakenly published a blog post about the event.
Altman’s candid remarks to a roomful of app developers and startup founders were detailed in a post late last month by Raza Habib, the CEO of London-based AI firm Humanloop.
During the discussion, Altman purportedly said OpenAI lacked enough graphic processing units, or GPUs – the ultra-powerful chips required to train and run AI software — to make fast improvements to ChatGPT.
“A common theme that came up throughout the discussion was that currently OpenAI is extremely GPU-limited and this is delaying a lot of their short-term plans,” Habib wrote in the blog post.
“The biggest customer complaint was about the reliability and speed of the API. Sam acknowledged their concern and explained that most of the issue was a result of GPU shortages,” Habib added.
OpenAI is just one of many firms impacted by the industrywide chip shortage. In April, The Information reported that AI startups were experiencing major difficulties due to server shortages at key cloud providers such as Amazon, Microsoft, and Google.
Demand for the chips is so strong that Nvidia, the leading US manufacturer, recently attained a $1 trillion market cap for the first time. The company’s shares are up by a whopping 171% since the start of the year.
Habib quickly took down the blog post, but archived versions spread quickly on social media and other platforms. The blog post sparked a pair of lengthy threads on the forum Hacker News, where users dissected Altman’s purported roadmap for OpenAI.
The original post now displays a message that reads, “This content has been removed at the request of OpenAI.”
When reached for comment, Habib confirmed that he had taken down the blog post after someone at OpenAI told him the event was meant to be “off the record.”
“We posted our notes from the developer session, they messaged me to say they had intended it to be off the record and apologized that they weren’t clear,” Habib said. “I took it down because we’re close partners and we want to support them as they support us.”
Habib did not respond to a request for further comment. The Post has reached out to OpenAI for comment.
According to the blog post, Altman reportedly complained that a lack of computing power has blocked OpenAI from implementing larger “context windows” for ChatGPT. Simply defined, context windows are the amount of information the chatbot can process when responding to a specific user prompt.
Without an improved context window, ChatGPT and other chatbots are limited in how much information they can “remember,” including past user prompts, and lack the capacity for more complex assignments such as complex coding.
“The main takeaway is indeed the GPU bottleneck, which is caused by a huge gap between demand and supply, that must be solved as soon as possible,” Omri Geller, the CEO, and co-founder of Israel-based Run:ai, told The Post. “This goes for OpenAI, but it’s true for the whole ecosystem.”
Altman also reportedly laid out his immediate plans for OpenAI and how to improve the ChatGPT experience for professional customers.
This year, Altman reportedly said that OpenAI would focus on creating a “cheaper and faster GPT-4” and “longer context windows” for the chatbot, as well as improvements to the API meant to streamline the experience for developers.
Altman also purportedly reassured worried developers that OpenAI had no plans to “release more products beyond ChatGPT.”
“Quite a few developers said they were nervous about building with the OpenAI APIs when OpenAI might end up releasing products that are competitive to them,” Habib wrote.
Fortune was the first to report on the deleted blog post.
Source
