HN user

sc077y

89 karma

Software Engineer, currently working in genAI.

I'm just a student of the game.

Posts1
Comments31
View on HN

I believe the ARC-AGI benchmark fits that description, it's sort of an IQ test for LLMs, though I would caution against using the word "Intelligence" for LLMs.

Every time you use these tools irresponsibly, for instance for what I like to call headless programming (vibe coding), understand that you are incurring tech debt. Not just in terms of your project but personal debt regarding what you SHOULD have learned in order to implement the solution.

It’s like using ChatGPT in high school: it can be a phenomenal tutor, or it can do everything for you and leave you worse off.

The general lesson from this is that Results ARE NOT everything.

Critical Skills is spotting the potential bugs before they happen but in order to do that you need to have an extremely acute understanding or a have a lot of experience in the stack, libs and programming language of choice. Something that, ironically, you will not get by "vibe coding".

In that case, you use copy OR what you can can also do, if you really care about disk usage, is use symbolic links between the drives. have a .venv sym link on drive A (raid 1) point to the uv_cache_dir's venv on drive B (raid 0). I have not tested though what happens when you unmount and sync.

Claude Code Router 12 months ago

I tried installing and setting up the project today, it was miserable. I finally got it to work only to find out that the mistral models' tool calling does not work at all for claude code. Also, there is no mention anywhere of what models actually support anthropic level tool calling. If anyone knows if there are some open weight models (deepseek or others) I can host on my infra to get this to work out of the box that would be amazing.

Steve Jobs' cabinet 12 months ago

I recently saw an interview without someone on the mac team and what's interesting is that the original Mac team had a lot of friction because of this philosophy. Jobs constantly asked unreasonable design constraints of his engineering team, the team prepared two laptops one with the "esthetic" laptop and one with the pragmatic design. One vastly outperformed the other and Jobs conceded, this was a constant pattern for the engineering teams at Apple. What you see before you is not the esthetic version but the pragmatic version. Sometimes Jobs was right but most of the time he was delusional.

How to Firefox 1 year ago

I tried Multi-Account Containers to try isolate my work from my personal but it just wasn't good enough. History, bookmarks, passwords and plugins were all shared. Do I really need twitch emotes and every personal password on my work environment? No. What I found as a solution was `about:profiles` and creating a separate profile with a distinct theme to tell the envs apart, and to sync I had to add a different Firefox account but it all works quite well.

I'm wondering if one of the big reasons that OpenAI is making gpt-4.5 deprecated is not only because it's not cost-effective to host but because they don't want their parent model being used to train competitors' models (like deepseek).

The way that the embedding is done is using Matryoshka Representation Learning, truncating it allows to compress while losing as little meaning as possible. In some sense it's like dimensionality reduction.

Actually the rate of hallucination is not constant across the board. For one you're doing a sort of synthesis, not intense reasoning or retrieval with the llm. Second, the problem is segmented into sub problems much like how gpt-o1 or o3 does using CoT. Thus, the risk of hallucinations is significantly lower compared to a zero-shot raw LLM or even a naive RAG approach.

Damn I built a RAG agent during the past 3 months and a half for my internship. And literally everyone in my company was asking me why I wasn't using llangchain or llamaindex like I was a lunatic. Everyone else that built a rag in my company used llangchain, one even went into prod.

I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have to go through 5 layers of abstraction just to change a minute detail. Furthermore, you won't really understand every step in the process, so if any issue arises or you need to be improve the process you will start back at square 1.

This is honestly such a boost of confidence.

I'm skeptical of the on-device AI. They crave edge compute but I'm doubtful their chips can handle a 7B param model. Maybe ironically with Microsoft's phi 3 mini 4k you can run this stuff on a cpu but today it's no where near good enough.

Impressive not technically because nothing here is new but because it's the first real implementation for the average end consumers of "ai". You have semantic indexing which allows series to basically retrieve context for any query. You have image gen which gives you emojigen or messaging using genAI images. TextGen within emails. UX is world class as usual.

However, The GPT integration feels forced and even dare I say unnecessary. My guess is that they really are interested in the 4o voice model, and they're expecting openAI to remain the front runner in the ai race.

Very interesting. I'm building a RAG chatbot and I haven't done the inline citations yet, I honestly thought it was a lot more complicated then just telling the llms to cite with a number and then have numbers next the sources. I did something to that extent as kind of a joke and it worked but the llm didn't always listen. I thought either post processing (checking cosine distance between sentences and retrieved chunks) or function calling would be the way to go.

Thinking back, if LLMs are able to have Memory store and access then RAG becomes useless. RAG is like a system that shoves bits down the RAM (Context Window) and ask the cpu(LLM) to compute something. But If you expand the RAM to a ridiculous amount or you use the HDD, it's no longer necessary to do that. RAG is a suboptimal way of having long term memory. That being said, today it is useful. And when or if this problem gets solved is not easy to say. In the meantime, RAG is the way to go.

RAG is a fantastic solution and I think it's here to stay one way or another. Yes the libs surrounding it are lacking because the field is moving so fast and yes I'm mainly talking about LangChain. RAG is just one way of grounding, that being said I think it's Agent Workflows that will really be the killer here. The idea that you can assist or even perhaps replace an entire task fulfilling unit aka worker with an LLM assisted by RAG is going to be revolutionary.

The only issue right now is the cost. You can make a bet that GPU performance will double every year or even 6 months according to Elon. RAG addresses cost issues today aswell by only retrieving relevant context, once LLMs get cheaper and context windows widen which they will, RAG will be easier, dare I say trivial.

I would argue RAG is important today on its own and as a grounding, no pun intended, for agent workflows.