Yannic Kilcher — transcripts.

1NeurIPS 2023 Poster Session 3 (Wednesday Evening)this gve W that is a representation of a graph we I'm not a gra well you are I mean if the graph given to us by the teacher model is good enough and that's what we are showing… 2TransformerFAM: Feedback attention is working memoryhello there today we're going to look at Transformer F feedback attention is working memory by group of researchers out of Google this paper adds what they call feedback attention… 3[Paper Analysis] The Free Transformer (and some Variational Autoencoder stuff)Hello there. Today we are looking at the free transformer from France Fur at Farret Meta. This transformer is extending the classic decoderbased transformer with uh a series of… 4Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper)hello how is everyone doing long time no see today we're going to look at this paper called scaling llm test time compute optimally can be more efficient effective than scaling… 5[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)Hello there. Today we're going to look at on the theoretical limitations of embedding based retrieval. And this paper is really interesting uh because it proves something about… 6I BUILT A FULLY AUTOMATIC MANSPLAINERHello. Have you ever been in a situation where you're overhearing conversation and it dawns on you this person well intended is wrong about something minuscule but important to… 7I created an AI-powered Social Networkare you tired of using words in order to transmit information are you tired of uploading actual pictures and then have other people see every single Pixel of that picture but this… 8Titans: Learning to Memorize at Test Time (Paper Analysis)Hello. Today we're going to look at Titans learning to memorize at test time. This is a paper by Google research and has been pushed as part of their Nurips publications. So… 9TiDAR: Think in Diffusion, Talk in Autoregression (Paper Analysis)Hello. Today we're looking at tidar thinking diffusion talk and auto regression by researchers at Nvidia. And this is a really cool paper because it effectively um makes good you… 10[ML News] Chips, Robots, and Modelshello hello uh today we're going to look at a few happenings in Industry new models new data sets and eval sets coming out and a few new tools that have been released or that… 11Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentionhello there today we're going to look at leave no context behind efficient infinite context Transformers with infinite attention so the specific techniques we're going to look at… 12[ML News] Devin AI Software Engineer | GPT-4.5-Turbo LEAKED | US Gov't Report: Total Extinctionhowdy didly do welcome to ml news it's a beautiful Monday and we're diving into everything that happened this week the big big Talk of the day is a thing called Devon which I'm… 13Art @ NeurIPS 2023we have our complete sliders to change the mood okay that within or the context within which what you're saying is going to and then I use that with that to get that okay that… 14Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Paper Explained)hello there today we're looking at hallucination free assessing the reliability of leading AI legal research tools this is by a set of researchers out of Stanford and Yale as you… 15Mixtral of Experts (Paper Explained)hello today we're going to look at Mixr of experts this model has been out for a while it has there's been blog posts and so on but this is the paper about the mixol 8x7 B mixt of… 16On the Biology of a Large Language Model (Part 1)Hello. Today we're taking a look at on the biology of a large language model which is a sort of paper published by anthropic on their kind of series on transformer circuits where… 17Byte Latent Transformer: Patches Scale Better Than Tokens (Paper Explained)hello there today we're looking at the paper bite lat and Transformer patches scale better than tokens this paper in a sense does away with classic fixed vocabulary based… 18[ML News] Llama 3 changes the gameH hello we are witnessing the Llama Revolution um so today I don't have big fancy intros or anything like this it's just pretty raw me um because we have to talk about llama 3… 19Were RNNs All We Needed? (Paper Explained)hello today we're going to look at where rnn's all we needed this is a collaboration of Mila and Borealis Ai and puts into question a couple of these modern takes on kind of RNN… 20Context Rot: How Increasing Input Tokens Impacts LLM Performance (Paper Analysis)Hello, today we are looking at context rot how increasing input tokens impacts LLM performance. This paper looks at what happens if with these long context models now we are just… 21AGI is not coming!Hello. I'm out of my usual room today. As you can see, I'm flying to the US for work, but I wanted to give some quick updates on OpenAI's new model launches the past week. So… 22LLaMA Pro: Progressive LLaMA with Block Expansion (Paper Explained)hello there today we'll look at llama Pro Progressive llama with block expansion this paper takes a llama large language model specifically a llama 7B and adds some layers to it… 23TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters (Paper Explained)hello there yeah it's cold here um we cannot be dissuaded from reviewing papers today we're going to look at token forer rethinking Transformer scaling with tokenized model… 24[ML News] Grok-1 open-sourced | Nvidia GTC | OpenAI leaks model names | AI Acthello it is another beautiful Monday wherever you are it's Monday now and we're talking about news in the ml space welcome lot of stuff happening this week first and foremost open… 25NeurIPS 2023 Poster Session 4 (Thursday Morning)subscriber oh very cool thank you very much can I take a picture of you sure sure sorry yeah I mean I know you everyone knows you hi I'm hi hi yeah your videos here we go another… 26Another Hit Piece on Open-Source AIhello I feel we have to talk about this report right here this is a report coming out of the Stanford internet Observatory cyber policy Center which for all I can tell is a center… 27[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelshello there today we're looking at Deep seek math pushing the limits of mathematical reasoning in open language models now this paper is a bit older um it's from as you can see… 28Hugging Face got hackedhello how's everyone doing today we're going over another set of news that I call the fun category the first one isn't so much fun as it is just interesting and that is a blog… 29AlphaGeometry: Solving olympiad geometry without human demonstrations (Paper Explained)cool hello there today we're going to look at solving Olympia geometry without human demonstrations this is a paper that introduces the alpha geometry model it's by Google… 30Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (Paper Explained)hello how's everyone doing I hope you're having a great summer I am um back from vacation and we're ready to dive into some new papers this paper is called privacy back doors… 31On the Biology of a Large Language Model (Part 2)Hello and welcome back to our analysis of on the biology of a large language model by anthropic. This is a blog post that anthropic has published uh investigating what they call… 32Energy-Based Transformers are Scalable Learners and Thinkers (Paper Review)Hello, how's it going? Today we're going to look at this paper right here. Energy- based transformers are scalable learners and thinkers. This paper combines the uh general… 33Until the Litter Endall right we're out of money that's it uh I have to shut down the litter social network that I made a few videos ago it was uh very fun to see what people submitted but it was… 34Lumiere: A Space-Time Diffusion Model for Video Generation (Paper Explained)hello today we're talking about Lumiere A Spacetime diffusion model for video Generation by Google research this paper is it's pretty insane so you put in text and you get out… 35Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping (Searchformer)hello there today we're going to look at Beyond AAR better planning with Transformers via search Dynamics bootstrapping by fair at meta this paper teaches a language model how to… 36[ML News] Jamba, CMD-R+, and other new models (yes, I know this is like a week behind 🙃)helloo everyone I hope you're having a wonderful Monday today we're going to dive into some new models that came out in the last 2 weeks it's an exciting time and the first one is… 37[ML News] Devin exposed | NeurIPS track for high school studentshey how's everyone doing it's beautiful Monday and this is a few selections of news around the release of llama 3 and 53 uh around these big announcements so some of this stuff is… 38No, Anthropic's Claude 3 is NOT sentientno the new anthropic model is not conscious or sented or anything like this it's not AGI it's not oh my God the world is going to change so much uh and upend everything it's a… 39Gemini has a Diversity ProblemGoogle almost had it they almost did it they were almost back into the prime Spotlight on on the good side of everyone and then they screwed it up what do I mean by that Google… 40[ML News] Microsoft to spend 100 BILLION DOLLARS on supercomputer (& more industry news)hello and welcome back to ml news today we're going to go over several things that happened in Industry during the last week the first one Microsoft planning to spend a100 billion… 41GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelshello there today we're going to take a brief look at GSM symbolic understanding of the limitations of mathematical reasoning in large language models this paper is out of apple… 42Scalable MatMul-free Language Modeling (Paper Explained)hello everyone today we're going to look at scalable mapol free language modeling by researchers of UC Santa Cruz sucha University UC Davis and Loxy Tech this is a paper that… 43Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Paper Explained)hello today we're going to look at Muma linear time sequence modeling with Selective State Spaces by Albert goo and Tre da I know I'm a bit late on this paper uh but I still… 44V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video (Explained)hello today we're going to look at revisiting feature prediction for learning visual representations from video also known as the paper that introduces the model v jeppa v JEA is… 45Flow Matching for Generative Modeling (Paper Explained)hello there today we're going to look at flow matching for generative models by people from meta Ai and whitesman Institute of science this is a bit more technical but it's the… 46[ML News] OpenAI is in hot waters (GPT-4o, Ilya Leaving, Scarlett Johansson legal action)we have to talk about open AI there's so much been going on in the entire world of AI and today we're going to have a full plate with just open AI I know there's tons of other… 47ORPO: Monolithic Preference Optimization without Reference Model (Paper Explained)hello there today we're looking at orpo monolithic preference optimization without reference model this is a paper by researchers of kaist AI and largely deals with alignment… 48[Video Response] What Cloudflare's code mode misses about MCP and tool callingHello, this is in many ways a response video to Theo or T3 GGG's video called MCP is the wrong abstraction and it is also a video about this Cloudflare article which that video is… 49xLSTM: Extended Long Short-Term Memoryhello there today we're going to look at xlsm extended long short-term memory by maximilan Beck corbinian Pepple and a team around them especially the corresponding author here is… 50Safety Alignment Should be Made More Than Just a Few Tokens Deep (Paper Explained)hello there today we're going to look at safety alignment should be made more than just a few tokens Deep by Anonymous authors so this is a paper that had been submitted as far as…