Jeff Dean: Trends in Machine Learning [video]

https://www.youtube.com/watch?v=oSCRZkSQ1CE

by belter • 2 years ago

339 112 2 years ago

youtube.com

adamnemecek5• 2 years ago

All machine learning is convolution.

Legend24403• 2 years ago

Only certain types of neural networks use convolutions. It's not universal.

CamperBob22• 2 years ago

What are some examples of types that don't?

adamnemecek5• 2 years ago

No, ALL machine learning architectures are convolutions https://grlearning.github.io/papers/11.pdf

dartos2• 2 years ago

All computers do is add.

tambourine_man2• 2 years ago

And conditional jumps

c0pium• 2 years ago

Random forest would like a word.

rvnx3• 2 years ago

At 1:00:18 it's claimed, "more than just publications" and that products are live for the public to use.

And they mention something that sounds cool. It's called DermAssist, it's to find similar images related to skin-diseases.

However, when checking the website, you are supposed to join a waitlist, but then it 404s.

Is the tool already shutdowned or it was never released ?

realprimoh• 2 years ago

It's not shutdowned I'm pretty sure. Looks like a bug. Hopefully it will be fixed soon.

rvnx3• 2 years ago

Let's hope so, because it seems really useful.

In the meantime, I found an open-source equivalent https://modelderm.com/en.html but if we could see the certified + tested tool it'd be nice :)

dr_kiszonka• 2 years ago

If you are in the US, "DermAssist is not available in the United States"(https://health.google/consumers/dermassist/)

htrp• 2 years ago

thats a new one... so where is it available?

nsoldiac• 2 years ago

Hi, my partner worked in technology strategy for a large healthcare system where she saw many derm AI-based applications evaluated over the last decade. (Dis)Incentives aside, they all followed a similar\ story arc overpromising in their research findings and underdelivering in actual care delivery. They've been around for longer than you'd imagine. I hope they reach their potential, but suggest approaching them with healthy skepticism.

JohnCClarke• 2 years ago

For an example of medical AI that delivers take a look at https://cydarmedical.com/

Real-time augmented reality for keyhole aortic surgery using machine vision to understand patients' individual anatomies.

Full disclosure: I was the original CTO.

nomilk6• 2 years ago

I see from comments I'm far from the only one using AI to summarise videos before deciding whether to watch them.

Reminds of the meme "why spend 10 minutes doing something when you can spend a week automating it". i.e. "why spend an hour watching a talk when you can spend 5 hours summarising it with AI and debating the summary's accuracy".

This sounds silly but potential gains from learning AI summarisation tooling/flows are large, hence why it warrants discussion. Learning how to summarise effectively might save hours per week and improve decisions about which sources deserve our limited time/attention.

63• 2 years ago

and debating the summary's accuracy

Perhaps consider simply reading the description for an accurate summary.

From the description:

Abstract: In this talk I’ll highlight several exciting trends in the field of AI and machine learning. Through a combination of improved algorithms and major efficiency improvements in ML-specialized hardware, we are now able to build much more capable, general purpose machine learning systems than ever before. As one example of this, I’ll give an overview of the Gemini family of multimodal models and their capabilities. These new models and approaches have dramatic implications for applying ML to many problems in the world, and I’ll highlight some of these applications in science, engineering, and health. This talk will present work done by many people at Google.

nomilk6• 2 years ago

In this case the video description contains a useful Abstract. AI summaries can offer additional value though, going into more/less detail (as desired), and allowing you ask follow up questions to drill into anything potentially of interest.

swyx• 2 years ago

sure its an accurate summary, but is it at a granularity or specificity that you want? LLM summaries lets you move around the latent space of summaries and you probably dont agree with the one chosen for youtube descriprtions.

bmitc2• 2 years ago

I feel like I'm missing some boat, but I'm not sure what boat it is. These "AI" systems seem very superficial to me, and they give me the same feeling as VR does. When I see VR be some terrible approximation of reality, it just makes me feel like I'm wasting my time in it, when I could go experience the real thing. Same with AI "augmentation" tooling. Why don't I just read a book instead of getting some unpredictable (or predictably unpredictable) synposis? It's not like there's too much specific information there. These tools are just exploding the amount of unspecific information. Who has ever said: "hey, I have too much information for building this system or learning this topic"? Basically no one.

It's just going to move everything to the middle of the Bell curve, leaving the wings to die in obscurity.

nomilk6• 2 years ago

If you know a book’s worth reading, going ahead and reading it works well. But for a lot of books/talks there’s competition for time - eg my bookshelf has 20 half read books (this is after triaging out the ones that aren’t worthy of my time) - any tooling that can help better determine where to invest tens or hundreds or hours of my time is a win.

Regarding accuracy, I think we’re at a tipping point where ease of use and accuracy is starting to make it worth the effort. For example Bard seems to know about youtube videos (just a couple of months ago you’d have to download it -> audio to text -> feed into a LLM). So the combination of greater accuracy and much easier to use make it worth considering.

saalweachter• 2 years ago

I was thinking the other day: Star Trek computers make a lot of sense if they are working with our current level of AI.

You can talk to it, it can give you back answers that are mostly correct to many questions, but you don't really trust it. You have real people pilot the ship, aim and fire weapons, and anything else important.

mrbonner• 2 years ago

I just read “Robust Python” book. My overall reaction is that book could have been written with half the length and still be valuable for me. I can't stop thinking if I could ask LLM to summarize each chapter for me, I still could "read" the whole book in the manner the author outlinea but save a tons of time.

JKCalhoun• 2 years ago

I had a conversation with a friend where he suggested that he had had a broad range of experiences just from gaming. I think the context was a conversation about how experiences in life can expand you — something like that.

The whole premise bothered me though.

I can remember a bike ride where I was experiencing the onset of heat stroke and had to make quick decisions to perhaps save my life.

I remembered decades ago lost in Michigan's upper peninsula with the wife, on apparently some logging road and the truck getting into deeper and deeper snow as we proceeded until I made the decision to instead turn around and go back the way we came lest we become stranded in the middle of nowhere.

I remember having to use my wits, make difficult decisions while hitchhiking from Anchorage, Alaska to the lower 48 when I was in my early twenties....

The actual world, the chance of actual death, strangers, serendipity ... no amount of VR or AI really compares.

hn_throwaway_99• 2 years ago

Why don't I just read a book instead of getting some unpredictable (or predictably unpredictable) synposis? It's not like there's too much specific information there.

I'm trying to understand this comment, because I couldn't disagree more. It is the absolute explosion of available data sources that has me wanting to be much more judicious with where I spend my time reading/watching in the first place.

Your comment was interesting to me because I feel like I agree with one of its main sentiments: that AI generated content all kinda "sounds the same" and gives a superficial-feeling analysis. But that is why I think AI is a fantastic tool for summarizing existing information sources so I can see if I want to spend anymore time digging in to begin with.

tmaly• 2 years ago

100 percent on things moving to the center of the curve.

For now, that’s not a bad thing if you need to know what the average information is.

As time goes by it might not be a good thing.

ren_engineer• 2 years ago

I've A/B tested this with webinars and the tools I've tried tend to miss some really valuable/interesting stuff even when I give it the full transcript. Same goes for when I try to use ChatGPT or other tools for full interactive analysis, even when I basically hand it what I'm looking for as if I hadn't watched the video it will leave out the critical information

tomrod• 2 years ago

What is your workflow for this, if you don't mind me asking?

nomilk6• 2 years ago

My approach wasn't fancy, just asked bard (aka gemini). I was drawn to bard/gemini for this since the source video is on youtube, so figured google would better support its related service (although that was an arbitrary hunch)

https://imgur.com/a/psb64IP

sammyatman3• 2 years ago

Try out www.askyoutube.ai!

jerpint2• 2 years ago

If you’re interested I did a YouTube video and short blog post about it

https://www.jerpint.io/blog/yougptube/

https://www.youtube.com/watch?v=WtMrp2hp94E

name_nick_sex_m• 2 years ago

What tool do you use to summarize video?

nomilk6• 2 years ago

(since it's a youtube video) I used bard/gemini: https://imgur.com/a/psb64IP

I have no idea if it's the best (or even a good) tool. Other commenters suggest some other tools (for both text summaries and condensed video summaries - a sort of 'highlights reel'):

https://news.ycombinator.com/item?id=39435930

https://news.ycombinator.com/item?id=39435964

kirill5pol2• 2 years ago

(Little self-plug) I made a tool that’s pretty relevant

https://www.platoedu.org/videos/oSCRZkSQ1CE/watch

It's not really giving summaries but gives topic/section timestamps and highlights what was discussed.

(Main focus is actually making mini-courses off of YouTube videos but I found the section summaries really useful for figuring out which parts to watch)

sammyatman3• 2 years ago

This is exactly why I built https://www.askyoutube.ai. It helps you figure out if a video has the answer you want before you spend time watching it. It does this by aggregating information from multiple videos in one-go.

I don't think it completely replaces watching videos in some cases but it definitely helps you skip the fluff and even guides you to the right point in the video.

esafak• 2 years ago

Do you transcribe the videos or use the captions, because GPT4 can already do the latter?

castles• 2 years ago

Learning how to summarise effectively might save hours per week and improve decisions about which sources deserve our limited time/attention.

If you like summaries, you'll probably love de-summaries (WIP): https://socontextual.com/

munificent• 2 years ago

1. Author spends a week producing a video when writing an article would have taken a day.

2. Viewer spends hours summarizing the video to an article so they don't have to watch it.

P R O G R E S S

kirill5pol2• 2 years ago

I made a tool that might be interesting for people here!

https://www.platoedu.org/videos/oSCRZkSQ1CE/watch

It's not really giving summaries but gives topic/section timestamps and highlights what was discussed

(for example: The Transformer Model (21:06 - 24:48) - Introduction of the Transformer model as a more efficient alternative to recurrent models for language processing)

The main focus is actually creating Anki-like spaced repetition questions/flashcards for videos and lectures you watch to retain knowledge, but I found the section information quite helpful for finding which parts of the video contain the info relating to topics/concepts

theGnuMe• 2 years ago

What ai tool do you use to summarize?

paxys• 2 years ago

Parts of it were good, but it mostly felt like he was reading through a slideshow created by the Google marketing team.

babl-yc2• 2 years ago

Agreed -- given the talk is titled "Exciting trends in Machine Learning" it feels pretty incomplete to gloss over ChatGPT blowing up.

OpenAI bet big on (1) emergent behaviors as you scale language models and (2) RLHF / fine-tuning to follow instructions.

Both those topics were lightly covered but very much from the "Google did this" perspective (word2vec, step-by-step reasoning).

lern_too_spel• 2 years ago

The original paper on emergent behaviors in LLMs is from Google: https://arxiv.org/abs/2206.07682

The difference between OpenAI and Google isn't due to different research directions between the two firms. It is a difference in ability to execute.

elevatedastalt• 2 years ago

The research breakthroughs all happened at Google, so it's not surprising.

godelski3• 2 years ago

What I hate about the ML community is that papers have become ad pieces now. I'm all for them releasing technical papers, but do we have to fuck up the review system for it. We don't need to railroad our research community and doing so is pretty risky. I don't buy the scale is all you need side. But if you do, doesn't matter if I'm wrong, we'll just get there when Sam gets his $7T. But if I'm right, we need new ideas if we're going to keep progressing without missing a step.

_t89y8• 2 years ago

Domain adaptation across verticals is the only driver of innovation. Check out the NeSy computation engine. Its ``semantic parsing'' is domain parsing and its symbols are numeric. Scale if you want. It works for images. That's all that matters, right?

Mapping language onto the latent space of images gets you crude semantic attributes sometimes. If you have to push for multimodal out of the gate maybe start with articulatory perception. These LVMs aren't going to cut it.

ML research isn't meant to further your understanding of anything. You can't separate it from corporate interest and land grabbing. It's the same thing re-hashed every year by the same people. NLP is pretty much a subfield of IR at this point.

I love how fast my comment was blacklisted to the bottom of this thread. lol

throwaway4good• 2 years ago

I watched some of this after it was promoted to me on YouTube.

It is very much: Look at this other cool blackbox that Google has made - I think it has a helpful personality.

And sure: Applied AI / ML is a part of computer science but I had hoped it would be more of a walk-through of what has happened in terms of advances in theory, algorithms, architecture, training methodology, and perhaps some sort of explanation of why does it work giving this general purpose model a bunch of medical data? (Is it merely an elaborate fuzzy search, how does it extrapolate, is there actually reasoning going on and how does emerge from a neural net etc...)

knbknb• 2 years ago

Maybe this was a review talk, designed for an audience of undergraduate students from various majors.

npalli• 2 years ago

Seeing a lot of people talk about auto-summary tools which reminds me of a joke.

“I took a speed-reading course and read War and Peace in twenty minutes. It involves Russia.” ― Woody Allen

Mistletoe• 2 years ago

This is such a great comment about so many things right now. Thank you.

idkdotcom• 2 years ago

It's a nice high level overview about the state of the art in machine learning.

If you have watched his other talks, Jeff Dean generally does a very good job explaining things from a high level point of view.

Here is a controversial view: I think that the current neural networks driven approach coupled with massively distributed computing has plateaued.

For the machine learning field to move forward, it will need a different, less data/compute hungry paradigm.

zyklonix• 2 years ago

Summary: https://www.videogist.co/videos/jeff-dean-google-exciting-tr...

nliang86• 2 years ago

Thanks for the post - I put together the tool above. I tried to strike a balance between being concise but also capturing all the important details. For that reason, the tool is hit or miss on longer (> 45 min) videos - the summary on this video is good but I've seen it omit important details on other long videos. The tool also captures relevant screenshots for each section.

Hopefully it's helpful. You can summarize additional videos by submitting a youtube URL in the nav bar or the home page. Also, feedback welcome!

crudalex• 2 years ago

Are you using LLM to summarize? If yes can you share the prompts used?

denfromufa• 2 years ago

The highlight of this event was running with Jeff at Rice University before his talk:

https://x.com/JeffDean/status/1756319820482592838?s=20

opisthenar84• 2 years ago

I'm surprised he didn't mention much about vector search

H8crilA• 2 years ago

It is a finished thing, no? So many systems/products have it built in for quite a few years now.

_t89y8• 2 years ago

There's a typo in the title of the talk but don't worry I fixed it: Trends in Computer Vision

``In recent years, ML has completely changed our view of what is possible with computers''

In recent years ML has completely changed our view of what's possible with computer vision.

``Increasing scale delivers better results''

This is true for computer vision.

``The kinds of computations we want to run and the hardware on which we run them is changing dramatically''

Optimizations on operations for computer vision isn't exactly dramatic change. Who is We?

Trends in Machine Learning, 2010: semantic search for advertising. Trends in Machine Learning, 2024: semantic search for advertising, short-form video content.

[link redacted]

_t89y8• 2 years ago

lol. Three computer vision researchers dislike this comment. Do any of you want to respond to it?

karmasimida3• 2 years ago

Not to shade on anyone, but even Jeff Dean isn't good at predicting the future.

The pathways model/architecture aren't exactly where things are moving towards, LLM is.

https://blog.google/technology/ai/introducing-pathways-next-...

knowriju• 2 years ago

Care to share some topics where the things are moving towards? I understand diffusion, GaNs and Mamba is in vogue these days, but those are different logical architecture. I am unsure where the next level ML physical architecture research is moving towards.

karmasimida3• 2 years ago

I think at this rate, everything is moving towards Transformer based models(text/audio/image/video), as Sora has shown, there isn't really anything Transformer can't do, it can generate both real life quality photo and video. Its ability to fit ANY given distribution is beyond compare, the most powerful neural network we have ever designed, nothing else is even close.

GANs are on the contrary, not hot any more in industry, diffusion models have achieved high fidelity in image generation, hard to see how GAN can make a comeback. It is faster, but it image generation in terms of quality is done, the wow factor is no more.

This might be a hot take, but I think architectural changes is going to die down in industry, Transformer is the new MOS transistor. As billions of dollars pumping into making it runs faster AND cheaper, other alternative architecture is going to have a hard time compete.

wslh• 2 years ago

I think this is the time where the idiom "it's good fishing in troubled waters" makes completely sense.

This is a moment where it is wise to wait and see. Investing in building AI/AGI startups now is like being a fish more than a fishermen. An outlier could win market share but will be only one within zillions. Google is catching up OpenAI but their competition is fruitful for all. Typical oligopoly. A single startup showing good traction will be immediately acquired.

From the business perspective it is time to focus on "go to market execution" and less or nothing on research. The research results are coming alone, except if you are one of the top scientist teams in the world or someone alike Ramanujan.

osigurdson• 2 years ago

One thing I think should be fairly clear is Google will actually try to compete in this space (vs quitting if it isn't an immediate success as they usually do). Traditional web search will be fairly uncommon in the long run I think.

Jeff Dean: Trends in Machine Learning [video]

Related Stories