[ad_1]
Right now, major companies are acquiring GPUs to train their AI models. Because of this, Nvidia is one of the most valuable companies in the world. Well, there’s a company named Groq that’s just burst on the scene, and this company is bringing a special kind of chip that might make other companies reconsider what they’re doing. Groq introduced the LPU and, based on an early example, it’s pretty fast.
To clear things up, many people on social media and across the web are referring to Groq as an AI model. However, Groq isn’t actually an AI model. A more accurate description of it would be an “AI chip system.” Any examples you see of Groq generating text is through a chatbot called GroqChat. The company employs open-source models like Llama 2 and Mixtral to showcase examples of how quickly AI models can run on the Groq AI chip interface. So, the text being generated isn’t being made by Groq. Think of Groq as the conveyor belt moving the text along.
Groq, with its LPU technology, might shake the AI chip market
The company became an overnight success once videos of it producing text surfaced. However, the company was actually founded back in 2016, the same year that OpenAI was founded. So, it’s not really a newcomer.
If you’re curious about what an LPU is, so is most of the world. The LPU stands for Language Processing Unit, and it was designed by Groq Inc. itself. It’s a chip specially designed for the tasks it’s performing.
side by side Groq vs. GPT-3.5, completely different user experience, a game changer for products that require low latency pic.twitter.com/sADBrMKXqm
— Dina Yerlan (@dina_yrl) February 19, 2024
We’ve seen examples of GroqChat versus ChatGPT, and they’re pretty wild. One example shows Groq generating a large wall of text in about 3 seconds. It took ChatGPT around 20 seconds to do the same thing. In fact, the company states that Groq is able to process up to 500 tokens per second. This is massive compared to GPT 3.5’s 40 tokens per second.
Why is Groq so fast? Well, why are Apple products so fast?
What Groq has going on is seriously impressive, and it could usher a massive paradigm shift for AI chips going forward. Groq Inc. posted a video on X explaining a few aspects of the company, and one quote stuck out. “We’re faster because we designed our chip & systems”.
This points to the fact that these chips are highly optimized from the ground up to work with Groq’s own systems. When it comes to other AI companies, they purchase their GPUs from companies such as Nvidia.
If this sort of dynamic sounds familiar, then you know about Apple versus well, the rest of the world! Apple designs its chips and builds its hardware and software from the ground up to work seamlessly with one another. This is why its devices are so insanely fast and reliable. Well, this is probably what we’re seeing now with Groq.
The company is building its chips and the interface running on top of them, which is Groq. So, this sort of optimization definitely lends to the speed and performance we’re seeing in the videos.
We’re not certain if other companies are going to emulate this model. However, if Groq proves to be as much of a disruptor as it looks, then other companies are sure to follow suit
[ad_2]
Source link