zacksiri

Gemini 3.5 flash-lite is more expensive than Gemini 3.1 flash-lite. Every upgrade is getting more expensive.

They want you to buy their hosted service, that's where the convenience is sold. If they give you a one liner script you can paste in or a docker compose that does everything from scratch they cannot sell their hosted services.

I watched this movie recently, and the same thought crossed my mind. I was going to write a blog post about it, but was too afraid to read the responses. I'm glad you did. If I meet you I'd buy you a cup of coffee, we'd talk and we'd become friends.

They cherry pick and choose the parts of her work that fits their agenda, while forgetting the other parts.

This reminds me of the following quote

"When you see that in order to produce, you need to obtain permission from men who produce nothing - When you see that money is flowing to those who deal, not in goods, but in favors - When you see that men get richer by graft and by pull than by work, and your laws don’t protect you against them, but protect them against you - When you see corruption being rewarded and honesty becoming a self-sacrifice - You may know that your society is doomed." ― Ayn Rand, Atlas Shrugged

Thank you for publishing this. I've been following Paul Graham and his works for a long time. It's refreshing to see everything written down in a document like this. This is the bible for startups. Honestly, it's beautifully simple but not easy.

Do you have similar math for the flash-lite variant of the models? I'd be curious. Based on my testing / benchmark i think it's around the 100-120B mark.

With the Pro variant being around 600B - 800B

My testing is comparing it's performance / output to other models in the same size range, so not as scientific as yours.

It's a lot more than an LLM, it's backed by database + LLM. Gemini / ChatGPT on it's own believe it or not has hallucinated movies and imdb links.

While google can show you results it cannot personalize things for you. With this app you can mark movies as seen and you can ask for "things i haven't seen"

or ask it to "pick a movie for me" from a search result, you can tell it which region / streaming service you have and search within things you can actually watch.

You can't do any of that with google.

There are tons of queries i've tried where google will simply not be able to give me good results.

The goal is to have a TvOS app where you can just search across all your streaming options and with a single click open / stream any movies you find. Instead of having to search 10 different streaming apps to something you want to watch.

In the example you provided I could not see any streaming options for any of the movies or figure out if it's available in my region. I would have to do multiple other search to figure out where I can watch them. I also cannot see ratings etc...

Well with JustWatch and other service aggregators you cannot search for movies by themes / use natural language. For example:

- "Zombie or post apocalyptic movies released in the last 5 years"

- "Find me top 10 movies on [insert streaming provider]"

- "Can you find me top 10 movies released in the last 5 years I haven't seen"

- "Movies that will make me feel like I'm trapped in space"

- "Movies that put smart people under pressure"

You simply cannot do this with the streaming service aggregators or JustWatch.

I hope that one day humanity learns that in war there are no winners. We're all just brothers and sisters born on different corners of the planet. We share the same home.

I hope that we stop attacking one another and find peace and work together as a race to overcome our challenges.

It's ok, it's not the best. There are models that do better, I'd use it for some basic tasks but not actual complex tasks like query generation and retrieval.

I tested the model in an agentic workflow. Here is the report:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

I tested this model in an agentic workflow, it failed at some very basic tasks:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

Yes, my workflows use caching intensively. It's the only way to keep things fast / economical.

This is going to be a fun one to play with. I've been conducting tests on various models for my agentic workflow.

I was just wishing they would make a new flash-lite model, these things are so fast. Unfortunately 2.5-flash and therefore 2.5-flash-lite failed some of my agentic workflows.

If 3.1-flash-lite can do the job, this solves basically all latency issues for agentic workflows.

I publish my benchmarks here in case anyone is interested:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

P.S: The pricing bump is quiet significant, but still stomachable if it performs well. It is significant though.

Just released API Access for my agentic movie search product. So companies can build smart search into their streaming app / tracker apps:

Here is a demo link:

https://memovee.com/platform/demo?guest_account_id=019c481b-...

Try queries like:

- "Top 10 movies of 2024, sort by highest rating first"

- "Top 10 zombie apocalypse movies"

- "Find me some good movies that take place in space, no horrors please"

- "Some good movies that will make me appreciate life"

- "Find me movies like Bladerunner"

Or whatever else you can think of. You can also tell it to "filter out movies with less than 300 votes sort by highest rating first" etc...

I never thought of Elasticsearch as a database and always designed systems around what elasticsearch is supposed to be an index based document store for used with search.

I think their API is great and have had amazing results with it. Their recent innovations around quantization (bbq) has been amazing for my use case building an agentic movie database for discovering movies and personalized movie recommendations.

There are benefits to not using your database for everything, even if it adds a bit of complexity by introducing another dependency. If the benefits out weigh the cost of complexity reaching for elastic has almost always been worth it for me.

https://memovee.com - Agentic movie database

https://zacksiri.dev - My Blog

You obviously didn't read my comment about someone helping him get out of Iran.

IF he's out of Iran, works in a company with a founder or investor he's not violating any laws.

No one is failing to acknowledge that he's in a shitty situation. That's precisely the point of my post. It also doesn't mean he cannot be helped.

I feel for this guy, I really do.

I see comments like "is this a request to bypass sanctions" OR "he's iranian"

Let's remind ourselves of the following:

- First understand that he didn't choose to be born and raised in Iran.

- Second people grow up have families become attached to where they're born it's not easy to just 'pick up and leave' moving to a new country is expensive and extremely difficult especially from countries like Iran.

- Third he's building something he believes in which is probably better than most people who live in privileged countries who sit around and do nothing.

To me this reads like a plea for help.

He's built something and showing it to the world, if someone likes it and wants to fund him / get him out of Iran, so he can pursue his dreams AND have the people who help him benefit along with him. I'm sure he'll be all for that.

Been building an agentic movie database. https://memovee.com already in private testing.

The code for the agent is here https://github.com/upmaru/memovee-tama

This work is extremely consequential. When building agentic systems determinism will significantly improve the reliability.

I hope all the model providers adopt this.

The problem described in this post has nothing to do with LLMs. It has everything to do with work culture and bureaucracy. Rules and laws that don't make sense remain because changing it requires time, energy and effort that most people in companies have either tried and failed or don't care enough to make a change.

This is one example of the "horseless carriage" AI solutions. I've begun questioning further that actually we're going into a generation where a lot of the things we are doing now are not even necessary.

I'll give you one more example. The whole "Office" stack of ["Word", "Excel", "Powerpoint"] can also go away. But we still use it because change is hard.

Answer me this question. In the near future if we could have LLMs that can traverse to massive amount of data why do we need to make excel sheets anymore? Will we as a society continue to make excel spreadsheets because we want the insights the sheet provides or do we make excel sheets to make excel sheets.

The current generation of LLM products I find are horseless carriages. Why would you need agents to make spreadsheets when you should just be able to ask the agent to give you answers you are looking for from the spreadsheet.

Based on my testing the larger the model the better it is at handling larger context.

I tested with 8B model, 14B model and 32B model.

I wanted it to create structured json, and the context was quite large like 60k tokens.

the 8B model failed miserably despite supporting 128k context, the 14b did better the 32B one almost got everything correct. However when jumping to a really large model like grok-3-mini it got it all perfect.

The 8B, 14B, 32B models I tried were Qwen 3. All the models I tested I disabled thinking.

Now for my agent workflows I use small models for most workflow (it works quite nicely) and only use larger models when the problem is harder.

LLMs are relatively new technology. I think it's important to recognize the tool for what it is and how it works for you. Everyone is going to get different usage from these tools.

What I personally find is. It's great for helping me solve mundane things. For example I'm recently working on an agentic system and I'm using LLMs to help me generate elasticsearch mappings.

There is no part of me that enjoy making json mappings, it's not fun nor does it engage my curiosity as a programmer, I'm also not going to learn much from generating elasticsearch mappings over and over again. For problems like this, I'm happy to just let the LLM do the job. I throw some json at it and I've got a prompt that's good enough that it will spit out results deterministically and reliably.

However if I'm exploring / coding something new, I may try letting the LLM generate something. Most of the time though in these cases I end up hitting 'Reject All' after I've seen what the LLM produces, then I go about it in my own way, because I can do better.

It all really depends on what the problem you are trying to solve. I think for mundane tasks LLMs are just wonderful and helps get out of the way.

If I put myself into the shoes of a beginner programmer LLMs are amazing. There is so much I could learn from them. Ultimately what I find is LLMs will help lower the barrier of entry to programming but does not mitigate the need to learn to read / understand / reason about the code. Beginners will be able to go much further on their own before seeking out help.

If you are more experienced you will probably also get some benefits but ultimately you'd probably want to do it your own way since there is no way LLMs will replace experienced programmer (not yet anyway).

I don't think it's wise to completely dismiss LLMs in your workflow, at the same time I would not rely on it 100% either, any code generated needs to be reviewed and understood like the post mentioned.

You could say that! hahaha! I'm happy to see someone understand what it is.

Have a look here, it's an early preview

https://x.com/zacksiri/status/1922500206127349958

You can see it's going from introduction, asking me for my name, and then able to answer question about some topic. There is also another example in the thread you can see.

Behind the scenes, the system prompt is being modified dynamically based on the user's request.

All the information about movies is also being loaded into context dynamically. I'm also working on some technique to unload stuff from context when the subject matter of a given thread has changed dramatically. Imagine having a long thread of conversation with your friend, and along the way you 'context switch' multiple times as time progresses, you probably don't even remember what you said to your friend 4 years ago.

There is a concept of 'main thread' and 'sub threads' involved as well that I'm exploring.

I will be releasing the code base in the coming months. I need to take this demo further than just a few prompt replies.

I publish my findings on my youtube channel and my blog, you're welcome to have a look. Both links are in my profile.

I've been working on solving this with quite a bit of success, I'll be sharing more on this soon. It involves having 2 systems 1st system is the LLM itself and another system which acts like a 'curator' of thoughts you could say.

It dynamically swaps in / out portions of the context. This system is also not based on explicit definitions it relies on LLMs 'filling the gaps'. The system helps the llm break down problems into small tasks which then eventually aggregate into the full task.