Ask HN: How should I burn $8k for Google Gemini 1.5 Pro?

https://news.ycombinator.com/item?id=40639986

by hoerzu • 2 years ago

12 10 2 years ago

news.ycombinator.com

Was looking into applying LLMs onto entity extraction in earnings calls. (Around 400mb of text data). 1 Million tokes is around 1-3$.

Open for ideas and experiments.

reneberlin3• 2 years ago

That's not how you do it. You use a Vectordatabase and a Retriever. This way, not all tokens of a document are used with the prompt, just the relevant parts will end up in the conversation / prompts. This way you save a lot of money and you are NOT limited to use GoogleAI - you can use whatever AI you want.

infecto2• 2 years ago

This is probably how you would do entity extraction. Why would you rag on entity extraction?

reneberlin3• 2 years ago

Use Flowise and a VectorDB like upstash or pinecone.

Flowise Tutorial https://www.youtube.com/watch?v=V7uBy3VQJAc

reneberlin3• 2 years ago

Just be careful with the "temperature" when dealing with numbers and financial data as well. You should check samples of the retrieved data by hand.

Setting the temperature lower than 0.9 means reducing the "creativity" and making it less prone to hallucinate.

infecto2• 2 years ago

What’s the goal?

dissahc• 2 years ago

are you sure you need a powerful model for this? llama3-8b is at least 10 times cheaper and might suffice for something like this

fenx3032• 2 years ago

I am a survivor please contact me

fenx3032• 2 years ago

if your not a survivor with the intention of finding each other and communicating, you should take dissahc out of your name and stop making these posts.

langcss• 2 years ago

Can’t you pre-filter it with a vector lookup or something cheap first (gpt3.5?). Or even simply filter first by other things you are considering for the trading decision (price to earnings for example). That might save you a lot of cost.

BOOSTERHIDROGEN• 2 years ago

Could someone from hedge funds kindly share their current technology stack if they utilize LLMs?

Ask HN: How should I burn $8k for Google Gemini 1.5 Pro?

Related Stories