Grok-2 Beta Release

by meetpateltech • 2 years ago

226 276 2 years ago

CMLab• 2 years ago

When github release?

ipsum2• 2 years ago

Realistically? Never. Grok-1 weights were released because it was quite bad compared to open source and closed models. Now that they have a competitive model, they won't give it away.

thomasahle• 2 years ago

Now that they have a competitive model, they won't give it away.

Lamma 3.1 is competitive.

pikseladam• 2 years ago

It uses FLUX.1 to generate images and it has been fun so far. Its good on writing, can generate very realistic photos, can create memes, and looks like hands problem is fixed now.

DaoVeles• 2 years ago

When I have time I will do my usual test "Realistic looking wizards bowling!" and see how it goes. So far I have had fairly disappointing results.

esperent2• 2 years ago

What's a realistic wizard? Given that wizards don't actually exist this might be a confusing request.

Have you tried putting in "photorealistic" instead of "realistic", assuming that's what you mean? I'm curious if that would get better results.

espadrine2• 2 years ago

First try, so no cherry picking: https://fal.ai/models/fal-ai/flux-pro?share=43b11da2-e008-4d...

ssijak• 2 years ago

Oh this is great, one more competitor with top model which will be available via API. I wonder what the pricing will be. OpenAI was slashing prices multiple times in the last year and a half I was using it.

mcintyre1994• 2 years ago

I can't imagine anyone would want to build on top of their APIs after they completed destroyed the Twitter API and its whole ecosystem.

perbu• 2 years ago

I assume it'll be a paid API so the "contract" is a lot more clear. Twitter never understood what to do with its API so pulling that particular rug makes sense.

But I too wouldn't use this. X is playing fast and loose with ... everything, so having a business rely on their product seems risky.

bboygravity5• 2 years ago

If they have the best model, everybody will use it.

With LLMs (and AGI) it's really that simple: the company with the best model wins regardless of all else.

threeseed4• 2 years ago

Musk apparently lied about the DDOS that caused the X Trump stream failure.

https://www.theverge.com/2024/8/12/24219121/donald-trump-elo...

If that's true it's not exactly the sort of behaviour you want from an API you're depending on.

tgsovlerkhgsel3• 2 years ago

The nice thing with LLMs is that the API is relatively simple - for the most basic case, it's string in, string out. While you may need to redesign your prompt a bit, I bet for many use cases, LLMs are reasonably interchangeable, and the integration work required for an API change should be minimal.

miki1232112• 2 years ago

LLMs are pretty easy to switch, though.

From a black box perspective, LLMs are pretty simple, you put text or images in, (possibly structured) text comes out, maybe with some tool invocations.

If you use a good library for this, like Python's litellm for example, all it takes is changing one string in your code or config, as the library exposes most APIs of most providers under a simple, uniform interface.

You might need to modify your prompt and run some evals on whatever task your app is solving, but even large companies regularly deprecate old models and introduce vastly better ones, so you should have a pipeline for that anyway.

These models have very little "stickiness" or lock-in. If your app is a Twitter client and is built around the Twitter API, turning it into a Mastodon client built around the Mastodon API would take a lot of work. If your app uses Grok and is designed properly, switching over to a different model is so simple that it might be worth doing for half an hour during an outage.

bitshiftfaced• 2 years ago

Those who would build on top of the API might be considering a couple of past changes that are significant, but not necessarily a reason to think they'll be further pain in the future: the company ownership changed, and those who train LLMs all of a sudden want all the human-created text on the internet.

ionwake7• 2 years ago

Guys come on, you cant keep releasing software in the US, and then do a staggered launch where months later things are available to users in England, Denmark etc. There should be no reason for it. Im sure whatever dumb EU regulations can be dealt with easily in the software, these staggered releases ( such as chagpt having no Memory etc MONTHS down the line for EU users ) its just a hindrance to progress. Its starting to feel like we live on an island in the middle of nowhere.

k4rli• 2 years ago

Fortunately we already have Anthropic etc so this new release isn't really relevant or useful

hersko2• 2 years ago

Why would another competitive option not be a good thing?

croes4• 2 years ago

I'm pretty glad the EU a least tries to protect it's citizens from company experiments, who only care about profit and not others well-being.

seydor9• 2 years ago

What is it protecting us from, realistically? It's just a powerplay by the world's most incompeteny private club (brussels)

ionwake7• 2 years ago

I've just realised people are upset because I called the EU regulations dumb. I want to clarify, they are an excellent idea, I truly mean it, but I think it should be OPT IN. Its more important for me to have access to cool tech then it is to have privacy, atm , in this stage of my life. I guess my problem is the implementation.

Cookie popups on every website, its completely idiotic. Ofcourse I end up wasting hours of my life clicking on them randomly. There should just be an OPT IN for the cookie popup, or OPT OUT. Either on a EU portal or something.

Infact I find it fascinating noone is coming up with a better implementation of privacy concerns in the EU. This whole US not releaseing software incase they get sued, and thinking its fine for the whole EU population to go around clicking popups every 2 seconds is a total failure that needs to be fixed.

Again the idea is good, but the implementation needs to be fixed.

irusensei• 2 years ago

Did you just criticized the glorious candy colored EU bureaucracy in Orange Reddit of all places?

ionwake7• 2 years ago

To be honest I love the occasional surprise downvote , I never try to say anything controversial - it's a good laugh when somehow I do I guess

mortehu• 2 years ago

This is how we encourage migration to the US. Early access to software and video streaming, superior Amazon experience, and much fewer cookie warnings (though we still have a lot).

You'll have to say goodbye to BBC iPlayer though.

ionwake7• 2 years ago

Well it’s working on me I’m starting to consider it. Tbf I should have moved to SV decade ago

squigz3• 2 years ago

It's utterly hilarious to think that people will move to America in its current state for... early access to TV shows, and fewer cookie warnings?

stainablesteel3• 2 years ago

the eu is on a regulation binge, there's not much that can be done

also be careful, you might be arrested for these comments in a few years time (lol)

ZacnyLos• 2 years ago

Twitter started irreversibly feeding users’ data into its “Grok” AI technology in May 2024, without ever informing them or asking for their consent.

https://noyb.eu/en/twitters-ai-plans-hit-9-more-gdpr-complai...

Lerc2• 2 years ago

What does irreversibly mean in this context? It seems like negative connotations are implied, but I feel like it's like irreversibly baking a cake.

makin• 2 years ago

Once the data is "compressed" into the model it cannot be easily removed without starting the training over.

0x0732• 2 years ago

If they use Twitter data does grok answer with a 280 character text?

Additional Twitter data is in my eyes mostly low quality content, that's nothing I would want in a AI model.

tgsovlerkhgsel3• 2 years ago

If they use Twitter data does grok answer with a 280 character text?

That may be considered a feature.

ChatGPT seems reasonably concise, Gemini's answers tend to be verbose (without adding meaningful content).

_imnothere• 2 years ago

low quality content

How does it matter even if the quality is high or low? The point is user data was used without consent.

bboygravity5• 2 years ago

So did OpenAI, why is it only a problem when Twitter itself does it?

squigz3• 2 years ago

I'm pretty sure it's not. I'm pretty sure people have been angry about OpenAI doing the same thing for a while now.

viraptor2• 2 years ago

Has it been proven that OpenAI used twitter for training? I know it knows about the popular tweets, but those are reported in many places, so could be ingested accidentally with other content.

(But regardless, many people raised an issue of OpenAI training from sources they shouldn't be allowed to access, so they're definitely a problem as well)

seydor9• 2 years ago

Twitter bad, but it s not unlawful in their jurisdiction . Don't want it? dont use it

esperent2• 2 years ago

As someone from the EU, hearing this argument over and over from Americans is exhausting.

They provide a product in the EU, therefore they must either follow EU law or exit the EU market. Just like an EU company that provides a product in the US has to follow US law.

aspeckt112• 2 years ago

[flagged]

iotku• 2 years ago

Seems the only impossible problems that can't be solved by AI are things that make metrics falsely look good to investors

pavlov• 2 years ago

Ironically X shouldn't care about those metrics, it's privately owned by the world's richest person...

bboygravity5• 2 years ago

Source, substantiation?

noduerme• 2 years ago

Idk. Ask it if you should close your account and see if it gives an honest answer.

bboygravity5• 2 years ago

This is a good one, another one that blows my mind: when I use the "I'm not interested in this topic" button and refresh the page, it shows that very same post at the top of my feed?!

I had to resort to configuring "muted words" to actually fix the problem.

Its very weird to me that such basic things seem so bugged on a platform that popular, but other much much more complex things (mass video streaming, Grok itself) work totally fine?!

jatins• 2 years ago

If I am reading the table correctly they are claiming it is better than all models but 3.5-Sonnet

Is anyone with X premium able to confirm the vibe check -- Is the model actually good or another case of training on benchmarks?

bekantan• 2 years ago

You can try it yourself on https://chat.lmsys.org (sus-column-r model)

floam• 2 years ago

In the intro text they described it better than Claude 3.5 Sonnet and GPT-4 Turbo (which isn’t OpenAI’s current model.)

modeless• 2 years ago

I don't think you are reading the table correctly. On LMSYS it's better than all models except the latest Gemini 1.5 Pro and GPT-4o. But there is a detailed benchmark table and different models win different benchmarks.

So results are mixed, but the real takeaway is this is a competitive model that is good enough to be worth using today. It puts xAI significantly above their previous position, and up near the top of the field with OpenAI, Anthropic, Google, and Meta. And their new H100 cluster should allow them to keep up with the next wave of releases, whenever that starts.

opdahl• 2 years ago

My favorite part of that table is how they put 3.5-Sonnet all the way to the right of the table making it harder to compare.

saberience• 2 years ago

As with everything promised by Musk, I'll believe it when I see it and use it myself and compare it to Claude 3.5.

Right now I'm not really a believer in Grok and I doubt it will be worth using.

eltoxo• 2 years ago

I don't personally see how an individual can judge this at this point unless it is a huge leap.

More importantly, if the model is not a huge leap at this point I just don't care if it is as good as the very limited models we already have because I am not impressed by any of these anymore.

Anything less than a 3.5 to 4 jump from here is just not going to vibe for me.

ayakang31415• 2 years ago

Can anyone tell me how much censorship grok has? I hate that many other LLMs have too much censorship.

stainablesteel3• 2 years ago

there are abliterated versions on ollama that you can use, some are more censored than others

i didnt test any crazy promps on grok but the image generation won't let you do things like naked people for example

vlaaad• 2 years ago

But when will it be available in my region (Europe)?

seydor9• 2 years ago

As soon as Elon ends his spat with Thierry breton

ThalesX• 2 years ago

I'm in Europe and have access to it.

miohtama3• 2 years ago

Just use VPN like other Europeans.

And also remember to vote in the next EU elections pro development, not against development, parties.

worstspotgain• 2 years ago

We need an alt-right version of AI like we need a pumpkin spice sushiccino. No thanks but no thanks.

hersko2• 2 years ago

Can you explain the difference between "alt-right" and "right"?

QuesnayJr• 2 years ago

I have seen sus-column-r on LMSYS a bunch of times. It seemed pretty good, though not as good as the best Google, Anthropic, or OpenAI models.

I'm surprised they managed to catch up. I guess there really is no moat.

llmfan• 2 years ago

The moat is compute and Elon has enough money and connections to jump the queue with providers

tgsovlerkhgsel3• 2 years ago

I assume they will have a lot less "safety", i.e. the model will be more likely to actually do what you ask instead of finding a reason why "sorry Dave, I can't do that".

Since these "safety" features tend to also degrade the model, that's likely also helping them catch up in the benchmarks.

BoredPositron2• 2 years ago

Sadly it's at the level of Claude and way worse than grok-1 or Lama without safety. It roleplays as nearly everything so I guess they know their target group.

drawnwren• 2 years ago

I’m so confused by this comment. Are you not aware that Claude 3.5 Sonnet is currently considered the best model?

smileybarry• 2 years ago

It’s weirdly the opposite of what you have in mind. It has no problems generating images of Trump and Elon in explicit situations or Elmo covering 9/11, but it “safety-censors” LGBT-related prompts to the point of generating a heterosexual couple when asked for a gay couple: https://x.com/karlmaxxer/status/1823753493783699901 Some got expected results for prompts with LGBT terms, but that generation is still very weird.

miki1232112• 2 years ago

Do we have any info on this model's balance of censorship versus safety?

This is Musk after all, so I wouldn't be surprised if it strayed far from the norm.

chrisco2555• 2 years ago

Censorship isn't safety.

corn13read2• 2 years ago

Why is this speculation your go-to first question here? Do some research yourself on the models instead of adding your own implicit bias. Are you saying the engineers at X are collaborating with Musk in a coupe for a secret censorship of their model vs others. Do you have evidence or is this your bias?

voidUpdate2• 2 years ago

Given how when grok first came out, and people started asking questions about trans people and it came back with very sensible takes (trans women are women etc), and Elon and all the techbros absolutely hated it, I'd guess steps have been taken to avoid a repeat of that

majewsky• 2 years ago

I don't think their bias was implicit. :)

CaptWillard3• 2 years ago

"censorship versus safety"

Do you guys have any idea how sinister "safety" sounds in this context?

remexre• 2 years ago

For example, not telling people to eat glue just because Reddit suggests eating glue could be considered a safety measure...

2OEH8eoCRo0• 2 years ago

Try asking it questions that are critical of Musk.

leroman• 2 years ago

It's hilarious they put Claude 3.5 Sonnet in the far right corner while it scores the highest and beats most of Grok's numbers.

jug2• 2 years ago

Yes, and I also noted how it beats Claude 3.5 Sonnet in Chatbot Arena by a bit of a margin.

This further feeds into my concern that the more advanced AI models we get, random enthusiasts at that site may no longer be able to rank them well, and tuning for Chatbot Arena might be a thing. One that is also exploited by GPT-4o. GPT-4o absolutely does not rank wildly ahead of Claude 3.5 Sonnet in a wide variety of benchmarks, yet it does in Chatbot Arena... People actually using Claude 3.5 Sonnet are also quite satisfied with its performance, often ranking it more helpful than GPT-4o when solving engineering problems, but at the expense of tighter usage limits.

Chatbot Arena was great when they were still fairly stupid, but these days, remember that everyday people are put against the task of ranking premium LLM's even solving some logic puzzles, trick questions and with a deep general knowledge far beyond that of singular humans. They can strike against traditional weaknesses like math, but then all of them suffer. So it's not an easy task at all and I'm not sure the site is very reliable anymore other than for smaller models.

squigz3• 2 years ago

Is there yet standardized ways of objectively testing LLMs? The Chatbot Arena thing has always felt weird to me; basically ranking them based on vibes.

lhl• 2 years ago

There was a mini-uproar when GPT-4o-mini (an obviously "dumber" model) outscored claude-3.5-sonnet on Chatbot Arena, so much so that LMSYS released a subset of the battles: https://huggingface.co/spaces/lmsys/gpt-4o-mini_battles

You can review for yourself and decide if it was justified (you can compare based on W/L/T responses and matchups). Generally, Claude still has more refusals (easy wins for the model that actually answers the request), often has worse formatting (arguable if this is better, but people like it more), and is less verbose (personally, I'd prefer the right answer with less words, but ChatArena users generally disagree).

If you look at the questions (and Chat Arena and Wildchat analyses), most people aren't using LLMs for math, reasoning, or even coding - if anything the arena usage is probably overly skewed to reasoning/trick questions due to the subset of people poking at the models.

Of course, different people value different things. I've almost exclusively been using 3.5 Sonnet since it came out because it's been the best code assistant and Artifacts are great, only falling back to GPT-4o for occasional Code Interpreter work (for tricky problems, Mistral's Codestral actually seems to be a good fallback, often being able to debug issues that neither of those models can, despite being a tiny model in comparison).

maeil5• 2 years ago

Your concerns are valid.

Two more things concerning Chatbot Arena:

- The prompts people use on have an incredible sample bias towards certain tasks and styles, and as such are unrepresentative of "overall performance" which is what people expect from a leaderboard.

- It is incredibly easy to game by a company, their employees or their fanboys if they would like to. No idea if anyone has done so, but it's trivial.

Just to give one example of the bias; advances in non-English performance don't even register on the leaderboard because almost everyone rating completions there is doing so in English. You could have a model that's a 100 in English and a 0 on every other language, and it would do better on the leaderboard than a model that's a 98 in every human language in the world.

lossolo2• 2 years ago

Use this https://livebench.ai It's a better benchmark.

espadrine2• 2 years ago

The technology is impressive; achieving such a level requires a lot of efforts in dataset creation, neural architectural costs, and GPU shepherding.

What is the company’s ethical position though? It officially stemmed from Mr Musk’s objection that OpenAI was not open-source, but it too is not open-source. It followed Mr Musk’s letter to stop all AI development on frontier models, but it is a frontier model. It followed complaints that OpenAI trained on tweets, but it also trained on tweets.

Companies like Meta, Mistral, or DeepSeek, address those complaints better, and all now play in the big league.

cheptsov4• 2 years ago

I entirely agree. I’m quite sure Musk has a very strong stance on ethics, but it would be great to hear about it more clearly, and ideally not just through words, but through actual actions.

thomashop• 2 years ago

A strong stance on ethics? Like his comments about unions and firing workers in the Twitter space with Trump yesterday?

kergonath4• 2 years ago

I’m quite sure Musk has a very strong stance on ethics

His whole history tells otherwise.

shafyy• 2 years ago

Haha, good one!

cheptsov4• 2 years ago

I’m not a fan of Musk but it is so funny to see that many haters here.

baq• 2 years ago

“Conservatism consists of exactly one proposition, to wit: There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect.”

sounds like mr Musk is a conservative.

bko• 2 years ago

This is not what conservatism is

didgeoridoo3• 2 years ago

According to the random Crooked Timber blog commenter who coined that viral aphorism in 2018, yes. But by what standard are that commenter’s musings to be considered expositive on modern conservative philosophy?

binary132• 2 years ago

what a nasty vicious outgroup

kardianos3• 2 years ago

That's not any conservatism that I recognize. In fact, what is espoused there is exactly the progressive left (Herbert Harcuse's "repressive tolerance") mindset.

And while I will grant you that liberalism (not to be confused with leftism), is different then conservatism, both (classical) liberalism and conservatism strongly require equal treatment (procedural symmetry).

clarionbell2• 2 years ago

I would encourage everybody who thinks so to pursue basic political and philosophical education. Perhaps with a dash of history.

This definition is plainly wrong on so many levels that it's practically impossible to engage with. But I'll make that mistake and engage on two points.

First, it implies that conservative position has somehow consistent features across time and space. There is difference between conservative in Germany, USA and China. Not to mention conservative in early, mid and late 20th century.

Second, ignoring legal norms is neither stated, nor implicated position of conservative political movements. At very worst, we can accuse them of maintaining laws with discriminatory intents. But not of flaunting those same laws.

tacitusarc• 2 years ago

This is unconstructive flamebait.

dredmorbius2• 2 years ago

Quote's by Frank Wilhoit.

hagbard_c• 2 years ago

Sounds like a made-up definition of Conservatism as 'that which I do not agree with'. It is not a very good definition, if you really want to find out what it is about you could read (or listen to) some of Roger Scruton's works. Here's an interview with Scruton to give some ideas of what it is about:

https://www.nationalreview.com/2018/07/roger-scruton-meaning...

You do not need to agree with him or his definition of Conservatism and there are other definitions of the term which are also applicable but none of those definitions have any resemblance to what you posed.

Musk is a centrist, not a conservative. He used to stand to the left of centre but has been moved to the right of centre by virtue of the left moving further left, thereby moving the centre to the left as well while Musk staid put.

Society needs conservatives just like it needs centrists and progressives and whatever other names you want to give to these philosophies and/or ideologies. A world made out of only progressives never gets anywhere since they will never find out what works well and what does not since their aim is to shape the future by means of societal change. A world made out of only progressives will eventually grow stale when the rate of change in the environment outpaces societies' capacity for change. Progressives can make good innovators but tend to be less able at keeping things running. Conservatives can be good at keeping things running but tend to be less inclined to innovation. These are broad brush strokes but the essence is sound, society tends to work best when there is a balance between conservatives and progressives.

You might notice some parallels: architects tend to be lousy builders, builders tend to be uninspired architects. Developers tend to be less gifted at UI design, UI designers tend to be sloppy developers. Copy editors tend to be unremarkable writers, writers tend to be less effective copy editors.

ralfd• 2 years ago

Offtopic: Wiki says about that quote attributed to political scientist Francis Wilhoit it is just from some random musician from Ohio

https://en.wikipedia.org/wiki/Francis_M._Wilhoit#:~:text=Con....

However, it was actually a 2018 blog response by 59-year-old Ohio composer Frank Wilhoit, years after Francis Wilhoit's death."

machdiamonds2• 2 years ago

Pretty simple explanations for all of those:

- xAI opens sources models with a 6 month lag, look at Grok 1

- No one else stopped development, so why should he?

- He owns Twitter, why wouldn't it be okay for him to train on Tweets?

DrSiemer• 2 years ago

Regarding the last question: because nobody gave them permission to use that data.

They tried to add a pre-checked mark to the settings, but at least in Europe, where we actually have consumer protection, that won't fly.

viraptor2• 2 years ago

xAI opens sources models with a 6 month lag, look at Grok 1

That's what happened once, rather than a policy that we can expect to be applied. (Unless I missed some announcement?) Based on "we'll publish the algorithm" which ended up being a one-off partial snapshot, never updated afterwards, I wouldn't hold my breath for the models.

He owns Twitter, why wouldn't it be okay for him to train on Tweets?

There's a whole thing about having clear opt-in agreement about how your data will be used for EU citizens. Twitter didn't comply here with their hidden opt-out strategy.

handsclean2• 2 years ago

He owns Twitter, why wouldn't it be okay for him to train on Tweets?

Because he doesn’t own the tweets. Can you imagine if posting a photo you took to Twitter meant it’s not your photo anymore? Totally ridiculous.

lenerdenator• 2 years ago

- No one else stopped development, so why should he?

I thought it was a moral imperative or some such thing to do AI right because it could "destroy humanity"?

Or was that just Musk and the rest of the special people in SV's way of aggrandizing themselves while trying to do something most of them have either no experience in or fail miserably at, which is raise an intelligence to be a responsible actor?

dachworker• 2 years ago

Don't take this as a pro-Musk or anti-Musk comment. I just want to paraphrase his reasoning:

In a recent interview on Lex Fridman, he envisioned a future where humans augmented AI, through a brain to computer device like Neuralink would be able to keep up with pure AI.

Now one can immediately notice a hole in this reasoning: namely what guarantees that the AI that is use to augment humans is going to be benevolent and won't go rogue?

Palpatineli• 2 years ago

Nothing guarantees that. But in this augmentations scenario human brain is necessary, unlike in the many extinction scenarios with pure silicon AGI take off.

alm12• 2 years ago

maybe that humans are controlling the creation process and can terminate it when the AI versions are going increasingly "rogue"?

kardianos3• 2 years ago

Musk has publicly stated his goal for AI is alignment with Truth, where truth is defined as to what corresponds with reality, not necessarily with the current social consensus. Specifically in terms of reason, given a set of facts, being able to reason to a real place, not just to a socially given answer.

plorg• 2 years ago

Then surely he wouldn't be training on tweets.

GuB-423• 2 years ago

Which means essentially nothing. Most questions where alignment matter do not have a "true" answer, just a social consensus.

You don't need an "aligned" AI to tell you the distance between the Earth and the Moon. You need an "aligned" AI to tell you not to rape people even if you can get away with it, that's because the idea that rape is bad is not an objective truth based on the laws of nature, it is a "social consensus".

motoxpro• 2 years ago

I wonder what "Truth" is. If I say I want you to make a picture of a lion eating at a 5 star restaurant, is that Truth? Is it truth because it can't refuse an ask? That feels like it is uninhibited, but not Truth, or truth.

sangnoir4• 2 years ago

OpenAI trained on tweets, but it also trained on tweets

Not only that - Grok is/was trained on ChatGPT output, which I suppose Musk felt was turnabout. When asked about its identity, the first Grok would respond like ChatGPT (https://news.ycombinator.com/item?id=38584922)

bionhoward• 2 years ago

Unfortunately, Grok is not even Open Output, nor is Mistral’s platform or DeepSeek’s. None of them can be used for work (mistral with a fancy commercial license, but you gotta jump through hoops)

Only Meta’s Llama can be used for work. The rest are just toys for personal use noobs who don’t read the fine print.

infotainment• 2 years ago

I’m hoping we’ll see an open release of this in 6 months or so, as we saw with Grok-1.

I’m not hugely optimistic, though.

stainablesteel3• 2 years ago

i think it might happen because its just the code and imo its not that valuable

the more valuable part is the dataset which probably requires a lot of people to hand-filter

and even more impossible is acquiring the training rig, which your average person can't even afford

machdiamonds2• 2 years ago

Pretty funny to read the comments from xAI's initial announcement now.

https://news.ycombinator.com/item?id=36696473

srid• 2 years ago

Well, it should be clear to anyone who reads HN that the comments here are not to be taken as the most sensible opinion in all cases. Cynical outlook is not uncommon in some contexts, especially when it comes to Elon Musk & X ... and the thread you linked to is a stellar example of that.

PS: I have X Premium and I'm quite liking Grok 2.

Alifatisk• 2 years ago

You know what’s also impressive besides this beta release? How Claude 3.5 Sonnet is still able to keep up so well. Grok-2 beat every other LLM except Claude. How did Anthropic achieve this?

lossolo2• 2 years ago

A lot better quality of training data and instruction tuning (data again). There is no other secret sauce.

miohtama3• 2 years ago

Also the sauce cannot stay secret very long. There is no moat in AI.

aoeusnth1• 2 years ago

It’s possible Claude is using the same model tuning that they used to create Golden Gate Claude to dynamically tune the 3.5 model to be better at whatever task it’s doing.

sagivo• 2 years ago

Putting a new tool for developers behind an "enterprise API" gate is a sure way to kill it

lucasRW3• 2 years ago

Glad to see an uncensored AI able to compete with the other models.

miohtama3• 2 years ago

They likely compete exactly because of this, because censorship eats the performance of a model.

mupuff1234• 2 years ago

So all models seem to converge to a similar level of performance - is this the end of the line for LLMs?

mmastrac• 2 years ago

"Our AI Tutors engage with our models across a variety of tasks that reflect real-world interactions with Grok. During each interaction, the AI Tutors are presented with two responses generated by Grok"

My guess is that they're using one of the third party AI training outfits for this and that they are paying through the nose.

This looks exactly like a training task I got to see on one of those platforms.

cma• 2 years ago

What are the big platforms for that?

clarionbell2• 2 years ago

I don't really care. The model may be competitive, but my use cases require speed, local (semi-local) execution and reliability. Neither of these seem to be baked into whatever X produced now.

When they make the mini model available for download and quantizable. That's when I may be interested. But given the minimal improvement in the past several months, I'm inclined to believe that we have reached the plateau.

ComputerGuru• 2 years ago

Interesting that they’re rolling this out to Twitter/X Premium users, it was previously the biggest differentiator between Premium+ haves and Premium have-nots.

pantsforbirds2• 2 years ago

If the X.AI team is able to build out a good enough model with access to real-time tweets, they could have an incredible product. I'd love to be ask about current events and get really strong results back based on tweets + community notes.

borg16• 2 years ago

the results with grok-1 were unimpressive summaries based on the tweets, with a 10%-20% hallucinations (when enquiring about paris olympics specific events).

yet to see if this new model is able to do any better on that regard

pantsforbirds2• 2 years ago

I was also not very impressed, but I still think they are positioned to have a great product if they can get past the accuracy issues.

Havoc• 2 years ago

Seems like a solid result & more competition is always better.

That said I’m still cheering for mistral and meta with their more open stance

Grok-2 Beta Release

Related Stories