Low-code, no-model recommendation system with GPT3

https://medium.com/swlh/gpt3-empowered-recommendation-system-ba4bad29276b

by satorii • 6 years ago

95 60 6 years ago

dannyw• 6 years ago

GPT3 really is remarkable.

If you told me in 2015 that I can give just a few examples of movie recommendations, on a model trained for general text, and get perfectly coherent recommendations, I wouldn’t have believed you.

lowdose• 6 years ago

Yes right, but just hope the signal to noise ratio in the trainings remains stable. Garbage in ...

rvz2• 6 years ago

Indeed. But now OpenAI has become more like another DeepMind, but starting to rent out their APIs to its users, indicating that they probably won't ever release models for their research.

They might as well rename to Standard AI.

calebkaiser2• 6 years ago

I don't fully disagree, but I also think there is a layer of complication here that goes beyond OpenAI's hopes of monetization (which are clearly a factor).

I maintain an open source ML deployment platform, and I've interacted with a bunch of teams that have used it to deploy GPT-2. It was actually the platform AI Dungeon built their app on. GPT-2 is a beast to deploy—it's huge (almost 6 GB fully trained), requires GPUs, and scales fairly poorly. You need to autoscale GPU instances aggressively to handle any kind of real time inference situation with it, and even with spot instances, that gets expensive quick.

GPT-2 is 1.5 billion parameters, and at the time, was scandalously large. GPT-3 is 175 billion. For a model that large, there's real questions around whether it's even feasible for the average team to use it if it is not hosted somewhere else as a third party API.

From that perspective, I think the value OpenAI captures with the API is less about the exclusivity of the model itself, but the exclusivity of their infrastructure. Because of that, I wouldn't be surprised to see them open source the model for research.

However, I 100% agree that the fact that the model still isn't open is concerning, and it casts some doubts on whether or not it will ultimately happen in the future.

bqcty• 6 years ago

I'm not so surprised given that the paper generator has been around for much longer:

https://pdos.csail.mit.edu/archive/scigen/

Example:

https://pdos.csail.mit.edu/archive/scigen/rooter.pdf

In a way, I find the output better than GPT-3's. GPT-3 output is always a bit creepy to read, as if you are talking to a deranged person.

empiko• 6 years ago

I am curious how much it actually thinks and how much it simply uses text from recommendation engines that got crawled.

fancyfredbot2• 6 years ago

That is a very deep question.

The_rationalist3• 6 years ago

A neural network can't think, please stop cargo culting neural networks

plutonorm• 6 years ago

Panpsychism is a serious philosophical position.

mattkrause2• 6 years ago

Still seems like an interesting question if you replace “think” with “inferred”, “learnt” (in the statistical sense), or any other synonym that’s in opposition to “memorized and parroted”.

bausano_michael• 6 years ago

Slightly related excerpt:

"Supporters of the Strong AI Hypothesis insisted that consciousness was a property of certain algorithms – a result of information being processed in certain ways, regardless of what machine, or organ, was used to perform the task. A computer model which manipulated data about itself and its ‘surroundings’ in essentially the same way as an organic brain would have to possess essentially the same mental states. ‘Simulated consciousness’ was as oxymoronic as ‘simulated addition’.

Opponents replied that when you modelled a hurricane, nobody got wet. When you modelled a fusion power plant, no energy was produced. When you modelled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modelled the human brain, why should you expect real thought to occur?"

— Permutation City by Greg Egan

anaganisk• 6 years ago

Given that "Open" AI API is very pricey, Im waiting to see how economical is it to use this as it is heavily marketed by all these blog posts.

rococode• 6 years ago

For anyone else curious, the pricing according to gwern [1]:

    Explore: Free tier: 100K [BPE] tokens, Or, 3-month trial, Whichever comes first
    Create: $100/mo, 2M tokens/mo, 8 cents per additional 1k tokens
    Build: $400/mo, 10M tokens/mo, 6 cents per additional, 1k tokens
    Scale: Contact Us

Expensive but (unfortunately) pretty standard market price for cloud NLP stuff. The price is about $40 per million tokens.

For comparison, cloud translation costs per million characters range from $6 (Azure bulk pricing) to $20 (Google/AWS without bulk discounts). If we assume the average BPE token length is 5 characters, GPT-3 will cost about $8 per million characters.

[1] https://old.reddit.com/r/GPT3/comments/ikorgs/oa_api_prelimi...

rvz2• 6 years ago

In one of the Reddit comments about a service (PhilosopherAI.com) using the API outlining the possible total costs:

That makes for 400 million tokens in 2 or 3 weeks, which puts me at like $4000/mo minimum

Oh dear, that means...

Scale: Contact Us

So while they are using the API for free for now, by these metrics, they are least going to be soon having a running cost of at least $4K+/mo. That is an expensive toy. The same goes for AI Dungeon.

App Store earnings will be used to cover OpenAI costs & develop new features. Thank you for your support.

Well I will be seeing a tsunami of complaints from OpenAI projects that they can't pay up for their "game changers" since they will be priced out quickly. OpenAI Wins, Everyone else loses.

If this is for simple projects, then I wouldn't dare build an entire startup based on someone else's API. OpenAI will win by default here.

strombofulous• 6 years ago

What is a token? One character? Or is it a currency?

tgv• 6 years ago

Intersteller (with -er, indeed)? How did it pick that one up as an alternative for Inception? There is something with that title, but no information in IMDb.

Anyway, for the rest it looks similar to Netflix' recommendation: similarity in actors, directors, keywords. A toy.

janekm• 6 years ago

A2:B10 (including the results for Intersteller{sic}) are the prompts, it's very confusing in the video. The results for "The Irishman" are from GPT-3.

PeterStuer• 6 years ago

Nice as a stunt, but feels like taking a 40 ton truck to get your lunch sandwich from the bakers.

aabhay• 6 years ago

GPT-3 is the Rube Goldberg of AI solutions

lukasLansky• 6 years ago

I guess it depends how you think about it. From computational perspective, it's heavyweight. On the other hand, it really optimizes for minimal human effort, on several levels:

* GPT-3 is conceptually simple: the effort to invent it and engineer it amortizes well.

* Once the model is trained, it can be reused to very different tasks.

* Querying doesn't require programming knowledge.

feral2• 6 years ago

If this turns out to be easy and convenient, it won't matter whether the tech is overpowered.

A family car is really overpowered for most trips to the bakers.

Yet people take the car all the time, instead of walking or cycling, because it's just more convenient and easy, and people are lazy.

You can't write just this off as a stunt.

If you described how YouTube renders cat videos to Tim Berners Lee in the mid 90s, it would also seem like a 40-ton truck.

And yet here we are.

gitgud• 6 years ago

Well it's a no-model AI system if the interface is a HTTP endpoint that's out of your control...

fancyfredbot2• 6 years ago

Presumably when a new movie (not included in GPT3 training data) comes out the recommendations are not going to be too good, unless someone spends a lot of money retraining the underlying model?

feral2• 6 years ago

This will be the case for everything built on GPT-3, but you've got to imagine it's possible for them to keep refreshing it, eventually training or fine tuning it on a daily basis as their web crawl updates.

superasn2• 6 years ago

I wish there was a light version of gpt3 with maybe like 100 api query limit per day so just amateurs like me got to play with this API too on weekends.

I have no agenda of changing the world and I have nothing substantial to fill in the gpt3 invite form but sometimes the best ideas come from tinkering and I just wish they gave some limited access to everyone.

ersiees• 6 years ago

You might like: https://transformer.huggingface.co The GPT-2 model is very similar to GPT-3 just smaller.

superasn2• 6 years ago

Thanks for the link. So as someone who is completely new to this, if this guy used this model instead of gpt 3 would he get similar results?

dharma12• 6 years ago

I thought they have a free tier from October? https://twitter.com/hardmaru/status/1301362995356774401

Not sure what the 100k tokens translates to in terms of query limit

dtech• 6 years ago

That's 3 month max though

satorii3• 6 years ago

assume each prediction use 1000 tokens (including both input prompt and output text), then 100k tokens -> 100 predictions

sanxiyn2• 6 years ago

Serendipity is another GPT-3 based recommendation engine. Check its about page. http://serendipityrecs.com/

m12k3• 6 years ago

"Recommendation system is so successful in many products and services we interact everyday" ->

"Recommendation system[S] [ARE] so successful in many products and services we interact [WITH] every[ ]day"

I'm sorry, but four errors within the first sentence is just not good enough - did the author even read this themselves just once before hitting publish? Show just a little bit more respect toward the people who give their attention to what you've written.

mparramon• 6 years ago

Maybe the author's first language is not English?

m12k3• 6 years ago

Sure, neither is mine. But if you don't have the skills to proofread what you've written yourself, then get someone to help that has. I don't expect to be able to write something in Chinese deserving of a wide audience all on my own either.

jonathanstrange2• 6 years ago

Personally, I don't care about form if I find the content interesting.

ClearAndPresent• 6 years ago

Form is content, content form. How you do anything is how you do everything. Errors in grammar and syntax indicate errors in thought and care.

satorii3• 6 years ago

Thanks for pointing out the errors. I got them fixed in the blog. This is also a good use case of GPT3, checking grammar and spelling. I might build an application for this.

m12k3• 6 years ago

Thanks for fixing them - and sorry if I was overly harsh. Sharing your writing with the world is nerve-wracking enough already without having the spelling and grammar shot down on HN. Truth be told, my first thought was "maybe this is some example of text created by GPT-3?" But I think you're right that it would actually serve well as a combined spelling and error checker to catch things like that - I hope you do build an application for it, that would be neat.

jwiley• 6 years ago

Very creative use of the technology. I wonder when GPT3 will evolve to handle more languages, and what if any steps they take to reduce cultural bias? If I was in Japan these recommendations would certainly be less useful. :)

tastroder• 6 years ago

Given this blog post I have no idea if it even works well for English. Sure, the demo video contains sensible recommendations but what gives? Most of those I've seen seem to be included in the training data shown in the image below. If you have the term "AI" in your blog's blurb readers might expect the author to actually perform an evaluation of stuff they write ads about.

nerdponx• 6 years ago

No model? What? Of course this is a model, it's just a pre-trained one.

serendipityrecs• 6 years ago

Hey, cool, you built the same thing as me but with less code: http://serendipityrecs.com/

One thing that's cool about GPT-3 is how versatile it can be, so I'm exploring a natural language search interface, which is something that traditional recommendation algorithms can't do.

Are you planning to incorporate GPT-3 into your startup? I'd love to hear about what use cases you envision it solving. If you're interested in connecting, my email is in my profile and my website.

satorii3• 6 years ago

Cool! I will drop you a message or catch up with you on openai-api slack.

There are a couple of features we see GPT3 stands out and potentially benefits the product features.

Currently we are evaluating the end to end quality and cost versus our traditional ML based recommendation and semantic search pipeline.

Low-code, no-model recommendation system with GPT3

Related Stories