Recent breakthroughs in AI: A brief overview | Aravind Srinivas and Lex Fridman

Full podcast episode: https://www.youtube.com/watch?v=e-gwvmhyU7A
Lex Fridman podcast channel: https://www.youtube.com/lexfridman
Guest bio: Arvind Srinivas is CEO of Perplexity, a company that aims to revolutionize how we humans find answers to questions on the Internet.

With certain things like ChatGPT I think that language is more modular than other things and easy to work with on a computer. Language is kind of like coding where you can copy whole sections of code and have it work the same in different places most of the time. Real things are different. Like cars are modular but that's not optimum. If I select an exhaust for my honda it's not necessarily the perfect size or people select giant ones that actually make it lose horsepower and they don't know the difference. Music is also another thing that you see the limits of written music vs reality. The computer copying notes from music and mixing them up is not the same as playing music.

Language is already filtered reality. People have saying such as "a picture speaks a thousand words". It's already digital pretty much or modular. In calculus for example the numeric way of solving problems turns turns the integrals into little modules. That's what the square waves of computers are and that's not real. I think the AI has a lot of hype and you're building a super mcdonalds register.

✨ Summary:

– Attention mechanisms, such as self-attention, led to breakthroughs like Transformers, significantly improving model performance.

– Key ideas include leveraging soft attention and convolutional models for autoregressive tasks.

– Combining attention with convolutional models allowed for efficient parallel computation, optimizing GPU usage.

– Transformers marked a pivotal moment, enhancing compute efficiency and learning higher-order dependencies without parameters in self-attention.

– Scaling transformers with large datasets, as seen in GPT models, improved language understanding and generation.

– Breakthroughs also came from unsupervised pre-training and leveraging extensive datasets like Common Crawl.

– Post-training phases, including reinforcement learning from human feedback (RLHF), are crucial for making models controllable and well-behaved.

– Future advancements might focus on retrieval-augmented generation (RAG) and developing smaller, reasoning-focused models.

– Open source models can facilitate experimentation and innovation in improving reasoning capabilities and efficiency in AI systems.

https://waleedsaima.blogspot.com/2024/07/the-top-10-breakthroughs-in-artificial.html?m=1

Over half of population obese or overweight, take Ronaldo’s advice and get that Coca-Cola out of there 😉

Crystalized history.

what in the actual ef is this guy talking about we don’t all have pHDs