GPT-4 and the Latest Developments in AI
In the world of artificial intelligence, there were 11 major developments this week, and each one probably deserves a full video. But just for you guys, I'm going to try to cover it all here. From scaling GPT-4 100x to stable Beluga 2.2 Senate testimony, let's dive into the latest developments in AI.
RT2: Understanding the World
RT2, which as far as I'm concerned could have been called R2D2 or C-3PO, is starting to understand the world. In this demonstration, RT2 was asked to pick up an extinct animal, and as you can see, it picked up the dinosaur. Not only is that manipulating an object that it had never seen before, but it's also making a logical leap that, for me, is extremely impressive. It had to have the language understanding to link "extinct animal" to this plastic dinosaur.
Robots at Google and elsewhere used to work by being programmed with a specific highly detailed list of instructions. But now, instead of being programmed for specific tasks one by one, robots could use an AI language model, or more specifically, a vision language model. The vision language model would be pre-trained on web-scale data, not just text but also images, and then fine-tuned on robotics data. It then became what Google calls a visual language action model that can control a robot. This enabled it to understand tasks like "pick up the empty soda can," and in a scene reminiscent of 2001: A Space Odyssey, robotic Transformer 2 was given the task of hammering a nail. It then picks up the rock, and because its brain is part language model, things like chain of thought actually improved performance when it was made to output an intermediary plan before performing actions. It got a lot better at the tasks involved.
Mustafa Suleiman's Prediction
In an interview with Barons, Mustafa Suleiman, the head of Inflection AI, said that they are about to train models that are 10 times larger than the cutting-edge GPT-4 and then a hundred times larger than GPT-4. That's what things look like over the next 18 months. He went on to say that it's going to be absolutely staggering, and it's going to be eye-wateringly different. This is an idol speculation, but Inflection AI has 22,000 h100 GPUs, and because of a leak, Suleiman would know the approximate size of GPT-4. Knowing everything he knows, he says he's going to train a model 10 to 100 times larger than GPT-4 in the next 18 months.
Runway Gen 2
The rapid development of AI video is also worth noting. This is Runway Gen 2, and let me show you 16 seconds of Barbie Oppenheimer, which Andre Carpathy calls filmmaking 2.0.
Sam Altman's Research
Sam Altman and his researchers made it clear in 10 different ways that they pray to the god of scale. They want to keep going bigger to see where this paradigm leads. They think that Google is going to unveil Gemini within months, and they say, "We are basically always prepping for a run," and that's a reference to GPT-5.
Real-Time Speech Transcription
Real-time speech transcription for deaf people is now available at less than $100. Subtitles for the real world are now possible using a device that can actually see captions for everything you say in your field of view in real-time while also getting a good sense of your lips, your environment, and everything else around you.
AI Voices Can Whisper
AI voices can now whisper. Ladies and gentlemen, hold on to your hats because this is one bizarre sight.
Stable Beluga 2
Stable Beluga 2 is based on the Llama 2 70 billion parameter foundation model. By combining the Orca methodology, albeit with only 10% of the data set size, and the Llama 2 models, the results are quite extraordinary. As you can see, on quite a few benchmarks, Stable Beluga 2 is competitive with GPT-3.5.
Universal Jailbreak for Large LLMs
Researchers published a universal jailbreak for large LLMs, allowing you to create a virtually unlimited number of such attacks. They were built to target open-source LLMs like Llama 2, but they found that the strings transfer to many closed-source publicly available chatbots like Chat GPT Bard and Claude.
Bio Risk
AI could empower a much larger set of actors to misuse biology. Anthropics is concerned that AI could contribute to the misuse of biology. Today, certain steps in bioweapons production involve knowledge that can't be found on Google or in textbooks and requires a high level of specialized expertise. This being one of the things that currently keeps us safe from attacks. We found that today's AI tools can fill in some of these steps, albeit incompletely and unreliably. In other words, they are showing the first nascent signs of danger.
Conclusion
In conclusion, the latest developments in AI are both exciting and concerning. From RT2 understanding the world to Mustafa Suleiman's prediction, AI is advancing at an unprecedented pace. Real-time speech transcription, AI voices that can whisper, and stable Beluga 2 are just a few of the latest developments. However, the risks associated with AI, such as bio risk, are also increasing. It's important to stay informed and to be aware of the potential risks associated with AI.
FAQ
Q: What is RT2?
A: RT2 is a robot that is starting to understand the world.
Q: What is Stable Beluga 2?
A: Stable Beluga 2 is a language model based on the Llama 2 70 billion parameter foundation model.
Q: What is the universal jailbreak for large LLMs?
A: The universal jailbreak for large LLMs is a tool that allows you to create a virtually unlimited number of attacks on language models.
Q: What is bio risk?
A: Bio risk is the risk associated with the misuse of biology, such as bioweapons production.
Q: What are the risks associated with AI?
A: The risks associated with AI include bio risk, the potential for AI to be misused, and the potential for AI to be used to create nefarious content.
Resources
- [Barons](https://www.barrons.com/articles/ai-could-spark-the-most-productive-decade-ever-says-inflection-ceo-51631600000)
- [The Atlantic](https://www.theatlantic.com/science/archive/2021/10/sam-altman-openai/620276/)
- [MIT Technology Review](https://www.technologyreview.com/2021/10/14/1041429/ai-jailbreaks-are-coming-for-the-worlds-largest-language-models/)
- [Senate Testimony](https://www.youtube.com/watch?v=JQJlmFqUA8w)
