GPT-4 is coming next week – and it will be multimodal, says Microsoft Germany

https://www.heise.de/news/GPT-4-is-coming-next-week-and-it-will-be-multimodal-says-Microsoft-Germany-7540972.html

by georgehill • 3 years ago

126 113 3 years ago

oidar• 3 years ago

Kenn explained what multimodal AI is about, which can translate text not only accordingly into images, but also into music and video.

Interesting that they are broadening instead of deepening the text side of GPT - maybe they are running into problems on that side and are building the feature set with low hanging fruit on adjacent areas.

xen2xen1• 3 years ago

Or it matches their search needs?

thatguyman• 3 years ago

it is interesting. i believe they may be hitting a wall as well

flangola72• 3 years ago

A paper from a week ago found that models trained on multiple data modes perform an order of magnitude better than text-only models of the same or even larger size.

lumost3• 3 years ago

There is substantial evidence that cross training on multiple tasks improves performance.

ChatGPT struggles with intuition about the world, if it has image/video knowledge than it can potentially reason about what happens when you flip a plate upside down.

mortenjorck• 3 years ago

I wonder if there’s any reason this wouldn’t also apply in reverse: could a corpus of textual training data from which the role of a hand can be derived (including the fact that it naturally has five fingers) help DALL-E reliably render hands correctly?

WoodenChair• 3 years ago

It doesn’t “reason.” It predicts.

jamilton2• 3 years ago

Cross-training doesn't always help, I think the architecture is still being worked out. I can't remember the name of it but Google had some multimodal model they released a paper for, maybe six months or a year ago, and it did slightly worse than SOTA ("negative transfer" is the term they used for doing worse from additional modalities), and I recall the paper talking about how reaching near-parity with single mode models was an improvement.

One of their new models, PaLM-E, does show some positive transfer, so it's definitely a potential path for increased capabilities.

inconceivable• 3 years ago

i think it's likely that they hired a diversified AI team with expertise in multiple areas and they're all working on their own areas. parallel efforts.

fizx• 3 years ago

There's no more text to train on.

Also, amazingly, transfer learning works better than anyone could have hoped! As an analogy, imagine that you could show that football players learned musical instruments 50% quicker than non-athletes. That would be amazing!

But LLMs can write music and training them on music makes them better writers. It's almost like intelligence & skills are completely general!

CuriouslyC• 3 years ago

That's just not true. There's no more super cheap data, but there's still probably about 5x the data we are currently feeding these models just waiting to be scanned and OCRd, and most of it is higher quality (if less topical).

osigurdson7• 3 years ago

The fact that AI requires so much data does make me think that we are somehow on the wrong track. Evolution didn't scan every atom on earth in order to produce a brain. FSD is a good example of this. It is ridiculous to me that we need to show the car videos of every possible situation in order for it to know how to drive.

highwayman47• 3 years ago

If this is true, wouldn't it be reported elsewhere? Also, what does this mean: GPT4 is coming to what?

MichaelMoser123• 3 years ago

Once GPT-4 has been released, how long would it take OpenAI to start using the new language model in ChatGPT?

MattRix5• 3 years ago

They might not do it right away if it’s slower and more expensive to run. ChatGPT has undergone a lot of efforts to optimize it.

theRealMe• 3 years ago

I’d love to read that article, but I don’t speak German and so have no idea how to get rid of that cookie modal.

Reptur• 3 years ago

https://webcache.googleusercontent.com/search?q=cache:janNff...

bhaney• 3 years ago

Thankfully your ad-blocker's element hider should still use your system language!

djmips• 3 years ago

I don't read German and it wouldn't let me ablehnen (reject) so eventually my only choice was zustimmen (agree). oh well.

est2• 3 years ago

You can ask New Bing to summarize it for ya.

etaioinshrdlu• 3 years ago

OpenAI is doing an amazing job fueling hype and intrigue. It is hard to separate what is real and what is imagined.

Previously, I heard that Bing Chat was based on GPT4. Is this true or not? Is it publicly known, or not? Certainly the model is far more powerful than what is in ChatGPT.

If Bing is GPT4, did they remove its multimodal capabilities?

If Bing is not GPT4, what is it? Does there exist a GPT3.5-mega with vastly improved reasoning capabilities and larger context windows?

I find the current moment in tech quite confusing.

htrp2• 3 years ago

gpt3 is referred to as davinci-003

chatgpt is what they call got3.5-turbo-0301 (current month checkpoint)

bingGPT is a allegedly finetuned version of gpt3.5 turbo that is better with search results.... microsoft internally calls it prometheus. You may also hear it referred to as Sydney.

i suspect GPT4 is just a bad translation

famouswaffles2• 3 years ago

bing is definitely not just a finetuned turbo. and microsoft have said as much.

MattRix5• 3 years ago

minor correction: The original davinci is GPT 3 but davinci-003 is GPT 3.5

irthomasthomas• 3 years ago

chatgpt is what they call got3.5-turbo-0301 (current month checkpoint)

Then why does the chatgpt endpoint show as text-davinci-002-render-sha? It's very confusing.

famouswaffles2• 3 years ago

There are many multimodal models that take an existing text only llm and do some visual training. Blip-2, Fromage, prismer and palm-e are examples. Multimodal isn't necessarily multimodality from scratch

IIAOPSW4• 3 years ago

Have I missed something? Last I heard Bing chat was off the rails, "vastly improving" the ability to rationalize contradictions rather than to reason. There was that screenshot going around where, facing a logical inconsistency, it resolved it by insisting to the user the current year is 2021.

arcanemachiner• 3 years ago

It was 2022.

Watching the Bing Chat escapades is both hilarious and mildly unsettling. It reminds me _so strongly_ of the sarcastic evil AI that runs the levels in the _Portal_ games.

nl3• 3 years ago

Last I heard Bing chat was off the rails

That was at least 2 weeks ago....

They've taken steps to limit it which have been mostly effective. There are some jailbreaks but they are getting harder.

It's a beta product. Give it 6 months and then judge it.

gr4yb34rd• 3 years ago

It always just gives me the hand emoji and tells me it doesn't want to talk if I correct it. That was like a week ago. Not sure what it's doing today.

woeirua3• 3 years ago

If even half of what we're hearing about GPT 4 comes to fruition then we're going to know in the next year or two tops if the scaling hypothesis is true or not.

If it's true, then we're going to have something very similar to the Star Trek computer by the end of the decade at the latest.

If it's false, then we're probably going to enter another AI winter this decade.

smy20011• 3 years ago

2 AI winter so far!

siva7• 3 years ago

AI winter isn't an issue anymore and won't be - the breakthrough in public perception already happened for its first time at this scale.

russdill2• 3 years ago

I am curious if AI is successful if an AI backlash is coming. Perhaps one of the existing political parties or a new political party taking up the mantle of regulating or outright banning of AI. Or even just attacks on AI researchers and institutions.

ethanbond8• 3 years ago

Why is that a perception question rather than a delivery question? We still don’t really know how/if the current crop of AI is going to deliver.

Even if there are a lot of great applications for these things (I think there are), it’s not clear they’re going to be truly transformative?

dangero• 3 years ago

Public interest wanes quickly if the technology doesn’t continue to be magic

https://en.m.wikipedia.org/wiki/Gartner_hype_cycle

spion• 3 years ago

Its not the first time. Chess was the first time.

BulgarianIdiot3• 3 years ago

What kind of a "winter" would this be when literally the world already runs on AI?

Search engines run on AI, stock markets trade on AI, social networks distribute content by AI, your photos and content are sorted on your phone by AI, vaccines and medicines are produced by AI, precedent law is implemented increasingly via AI, our weapons are AI (drones) and so on and so on and so on.

And all this up there was BEFORE GPT and DALL-E.

No, no winter. We're in the singularity.

anticensor• 3 years ago

Technological singularity is the point no one is able to predict what happens after the point from what comes before. We have not arrived that yet. And yes, strong AI will pave road to technological singularity.

woeirua3• 3 years ago

You're using the term AI pretty generously there. Let's not delude ourselves into believing that deterministic algorithms are general artificial intelligence.

midlightdenight• 3 years ago

This has always made me curious about multi modality and especially Google’s Palm. At least how google presents Palm in diagrams.

The way I’ve interpreted the scaling hypothesis is that we will see emergent intelligence with larger nets through automated training alone. If we want a model to learn images we throw a larger net at it with training data.

The way I’ve interpreted some of these newer techniques to multi modality is that they are stitched together models. If we want a model to learn images we decide this and teach one model images and then connect it to the core model. There’s not a lot of emergent behavior due to scaling in this scenario.

With that perception I don’t see how gpt4 says anything about the scaling hypothesis. However I am not in this field and would be grateful to learn more.

concinds2• 3 years ago

The brain is composed of distinct regions that specialise in specific tasks. It's reasonable to assume AGI would be the same.

So the goal should be: we've created a "language module" (LLMs) and a "visual perception module" (computer vision), but we also need to add a "logic module", a "reasoning module", an "empathy module", etc, while continuing to improve each.

I just don't see how you could get an LLM, no matter how advanced, to recognize a car. Even if it can describe cars (wheels, windshield, doors) it doesn't know what any of those components look like. It's like that old joke about philosophers being unable to define a chair beyond "I'll know it when I see it".

osigurdson7• 3 years ago

I recall watching a video where Sam Altman was downplaying expectations for GPT4 so I'm not expecting Star Trek. However, I don't think we are due for another AI winter anytime soon. Even if technology were to plateau at GPT3.5 for several years it would not constitute a typical AI winter as I don't think we've found the edges of the current technology yet.

tpoacher• 3 years ago

surely you mean GPT3.1

followed by GPT95, 98, XP, etc

woeirua3• 3 years ago

We can have an AI "winter" without actually having a solid freeze in terms of development if expectations fail to match the reality. Right now, the hype around ChatGPT is unwarranted in my opinion, and if nothing else comes from it we will end up in at least a cooling off period as people realize that.

zeofig• 3 years ago

Spoiler: it's false

concinds2• 3 years ago

I've looked for papers, but can't figure this out: do our current neural networks (LLM, vision) qualify as complex systems?

Are they composed of autonomous agents that exhibit: - diversity (in behavior) - high connectedness - interaction among agents - adaptation (agents changing their behavior based on the behavior of other agents)?

Because if current NNs don't fit that definition of complexity, they it seems unreasonable to expect them to be intelligent, no matter their scale. Intelligence will likely be an emergent property of the above four criteria.

If anyone knows of relevant books or papers, would love to hear it.

BTW: "AI winter" refers to a collapse in funding; but I think we're well past that point. Previous winters only happened because academia led AI research, and there were no commercial opportunities. Now, the economic benefits are far more tangible. And if funding doesn't dry up, there's no reason to think innovation will dry up either.

pksebben2• 3 years ago

they're gonna. I've seen a lot of material on wiring these models together to have them do more interesting cross-functional work. Microsoft is about to put out a model you can talk to that makes and recognizes images - and there's already multi-model frameworks that do things like speech-to-text to LLM to text-to-speech, which means you can hold a verbal conversation.

Once the continuous learning problem is solved, it'll be pretty trivial to just tell your compound model to "go learn everything you can about x" and have it autonomously do so.

If it were me, the first thing I'd ask it to do was learn how to make other AI, then give it some hardware and the directive to build the best thing it could and test it until it can build a model better than itself.

jokoon• 3 years ago

Artificial intelligence will always be artificial.

If science can neither have a better definition of intelligence, or analyze and understand the resulting black box of a neural net, I don't see how AI would improve or get nearer general intelligence.

I have much more curiosity about neurology and how psychology define intelligence, than how programmers are trying to build things that digest data.

Proper science requires understanding. ML is not about understanding, it's only "sophisticated statistical methods". It is still artificial, thus but genuine.

nl3• 3 years ago

I don't really get the obsession with if the MS models is some variant of GPT-3.5 vs GPT-4.

It's just a version number, and there aren't really meaningful reasons why something is a 0.5 increment vs a 1.0 release increment, other than "there are lots of changes" where lots ~= something.

chessgecko• 3 years ago

The gap in quality between gpt2 and gpt3 was much much larger than the gap between gpt3 and 3.5. If they could get the gap between 2 -> 3 again when moving to 4 it would be crazy

MattRix5• 3 years ago

It is a big deal because the major version numbers usually represent a new model trained from scratch, vs just a base model that has been given extra training (which is what happened from GPT 3.0 -> 3.5)

nl3• 3 years ago

InstructGPT (which ChatGPT/GPT-3.5) is based on was trained from scratch:

We train three model sizes (1.3B, 6B, and 175B parameters)[1]

It does use the same model architecture as GPT-3 though so I guess there is a bit of reasoning there maybe.

[1] https://arxiv.org/pdf/2203.02155.pdf

justanotheratom• 3 years ago

I call bs

bob1029• 3 years ago

In the leaked openai foundry pricing there is a davinci 32k model. I wonder if this is effectively GPT4. 32K tokens is a lot of context.

musicale• 3 years ago

Old news, I'm already on GPT-6.

GPT-4 is coming next week – and it will be multimodal, says Microsoft Germany

Related Stories