hawtads

For model routers, do they have to retrain the routing model every time a new LLM is released?

Not sure about OCR specifically, but the newer (past quarter) vision language models all have a lot of post training on detecting garbled text specifically. You can feed some of the old stable diffusion outputs into a modern model and they can figure out pretty reliably if the text gen is mangled. I think the feature is probably used as part of the RL for the image gen to correct for bad text rendering.

I think OP is referring to the initial sign up based on an invite link, not the database storage itself.

It depends whether you want to check the invite code against the invitee's email. It very much depends on the application. For most use cases, just verifying the code is fine and accept any email addresses that posses it.

Essentially ask yourself this, if you change the UI a bit and allow any user to sign up with any email, and the workspace joining is done via a code they enter after login, is that acceptable for your product design and security posture.

The word champ comes from the world of horses actually

https://www.npr.org/sections/memmos/2016/06/09/605796769/che...

*champing at the bit

If you are on react/next.js, defer the client side initialization until after your app has painted. PostHog (especially with their session recording feature) likes to initialize a little before the rest of your app loads through either their context provider or instrumentation.ts (if you are on next.js). It's generally insignificant except if you are building a B2C web app where the extra 100ms makes a difference in retention.

On the server side, queue all analytics call and run them after the main request completes (on next.js it would be within an after() function on the server side).

You can paste this comment into Claude and it should handle the refactor just fine.

Note with the changes in analytics scheduling, you will lose out on real time analytics in favor of better time to first load. So keep the trade-off in mind.

What you're describing is a behavior tree: predefined logic, predefined responses, no learning, no inference, no model.

Stop calling everything AI, guys.

Depends on how pedantic you want to get, one could argue that regular expressions are AI too.

https://www.rand.org/content/dam/rand/pubs/research_memorand...

Regexes were invented for much higher order tasks (modeling neural networks) than just making find-and-replace easier.

He is losing a lot of information in that normalization pipeline (whole shell reduced/feature engineered into nothing but an outline). A CNN or something similar would be better and he can maybe get a better depth map of the mouth shape.

There is OpenCode and Pi, they both work pretty well

Why not use higher thinking effort?

Elenco is still around. I just bought a XP-720K linear power supply kit last week.

Have you tried Jetbrains Fleet? Their new editor isn't too bad.

We are working on making agentic ads and regulatory compliance scalable.

https://hawtads.com

Just launched the blog too

https://blog.hawtads.com/

The original Kleene Star Regex was invented to model neural networks. Have you tried throwing a transformer at the problem /s? Also O(n²) but at least you get hardware acceleration ¯\(ツ)/¯

Here's Kleene's Representation of Events in Nerve Nets and Finite Automata:

https://www.rand.org/content/dam/rand/pubs/research_memorand...

Ooh good find, thanks for the link. This will be my bedtime reading for this week :)

I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats?

Though if you think about it, a diffusion model is somewhat (partially) frequentist.

I think it would be interesting if frequentist stats can come up with more generative models. Current high level generative machine learning all rely on Bayesian modeling.

Well, hope they reinforced the wings, that's a massive weak point for dusters.

50 knots rotation is perfectly fine for a plane that size. A Cessna Skyhawk is certified to rotate at 55 knots fully loaded (and since the stall speed is around 40knots, for specialty take-offs like soft fields it's much lower, 50knots is more than enough).

No, Claude on GitHub Copilot is billed at 3X the usage rate of the other models e.g. GPT-5.4 and you get an extremely truncated context window.

See https://models.dev for a comparison against the normal "vanilla" API.

We are building an agentic ad tech system optimized for real time and scale. The process of making an ad, from ideation to distribution, is traditionally exceptionally labor intensive. We are making it possible to target, design, and distribute ads at scale and in real time.

https://hawtads.com

I have been working on the next generation of Canva and Photoshop for highly regulated verticals where there are specific demands placed on the generation and edit flow.

https://hawtads.com

If you are a brand who needs to deploy advertisements at scale, don't hesitate to reach out.

It could have exceeded either its real context window size (or the artificially truncated one) and the dynamic summarization step failed to capture the important bits of information you wanted. Alternatively, the information might be stored in certain places in the context window where it failed to perform well in needle in haystack retrieval.

This is part of the reason why people use external data stores (e.g. vector databases, graph tools like Bead etc. in the hope of supplementing the agent's native context window and task management tools).

https://github.com/steveyegge/beads

The whole field is still in its infancy. Who knows, maybe in another update or two the problem might just be solved. It's not like needle in the haystack problems aren't differentiable (mathematically speaking).

Okay, here's the tl;dr:

Attention based neural network architectures (on which the majority of LLMs are built) has a unit economic cost that scales (roughly) n^2 i.e. quadratic (for both memory and compute). In other words, the longer the context window, the more expensive it is for the upstream provider. That's one cost.

The second cost is that you have to resend the entire context every time you send a new message. So the context is basically (where a, b, and c are messages): first context: a, second context window: a->b, third context window: a->b->c. It's a mostly stateless (there are some short term caching mechanisms, YMMV based on provider, it's why "cached" messages, especially system prompts are cheaper) process from the point of view of the developer, the state i.e. context window string is managed by the end user application (in other words, the coding agent, the IDE, the ChatGPT UI client etc.)

The per token cost is an amortized (averaged) cost of memory+compute, the actual cost is mostly quadratic with respect to each marginal token. The longer the context window the more expensive things are. Because of the above, AI agent providers (especially those that charge flat fee subscription plans) are incentivized to keep costs low by limiting the maximum context window size.

(And if you think about it carefully, your AI API costs are a quadratic cost curve projected into a linear line (flat fee per token, so the model hosting provider in some cases may make more profit if users send in shorter contexts, versus if they constantly saturate the window. YMMV of course, but it's a race to the bottom right now for LLM unit economics)

They do this by interrupting a task halfway through and generating a "summary" of the task progress, then they prompt the LLM again with a fresh prompt and the "summary" so far and the LLM will restart the task from where it left of. Of course text is a poor representation of the LLM's internal state but it's the best option so far for AI application to keep costs low.

Another thing to keep in mind is that LLMs have poorer performance the larger the input size. This is due to a variety of factors (mostly because you don't have enough training data to saturate the massive context window sizes I think).

The general graph for LLM context performance looks something like this: https://cobusgreyling.medium.com/llm-context-rot-28a6d039965... https://research.trychroma.com/context-rot

There are a bunch of tests and benchmarks (commonly referred to as "needle in a haystack") to improve the LLM performance at large context window sizes, but it's still an open area of research.

https://cloud.google.com/blog/products/ai-machine-learning/t...

The thing is, generally speaking, you will get a slightly better performance if you can squeeze all your code and problem into the context window, because the LLM can get a "whole picture" view of your codebase/problem, instead of a bunch of broken telephone summaries every dozen of thousands of tokens. Take this with a grain of salt as the field is changing rapidly so it might not be valid in a month or two.

Keep in mind that if the problem you are solving requires you to saturate the entire context window of the LLM, a single request can cost you dollars. And if you are using 1M+ context window model like gemini, you can rack up costs fairly rapidly.

Copilot and many coding agents truncates the context window and uses dynamic summarization to keep costs low for them. That's how they are able to provide flat fee plans.

You can see some of the context limits here:

https://models.dev/

If you want the full capability, use the API and use something like opencode. You will find that a single PR can easily rack up 3 digits of consumption costs.

Don't forget volume. Just having well structured content isn't enough, you need large volumes of content such that it makes a statistical difference during the training process.

Google ads are the cheapest yes, but depending on your audience they may not be looking on Google now.

For ChatGPT (and similar) you need to have a strong FAQ page and lots of content marketing to increase the likelihood of being the suggested answer when a user asks ChatGPT a relevant question (it's a highly probabilistic system, look up AEO/GEO).

CloudFlare for example offers an option to block AI scraping bots by default. If you are in the services business, this is the opposite of what you want because having AI crawlers scrape your site would drive traffic down the road when users ask a related question.

I would also suggest having accounts with major chatbot companies and enabling the "allow training on my conversations" option and then talk to it about your services. Ultimately you just want to get your brand into the training data corpus, and the rest is just basic machine learning statistics.

It's not just Facebook, the entire ads industry is heading in this direction. There's a seismic change going on right now.