sneilan1

I've dumped python for anything that doesn't involve scipy / pytorch in favor of go. So yes, I'm having my llm's output go now which is generally more memory efficient than python.

I am so happy that Anthropic has signaled the possibility that their UI moat for agentic AI is copyable by competitors. At least that's the way I read this. When companies try to lock something down it can be a signal of weakness.

If so, it's possible to built great user interfaces in Chatbots and more companies/people can have amazing agentic development workflows! We don't have to live in a world where only the market leader has the most enjoyable model.

I wish the author had not used the words "Abandoned Mine in the Atacama Desert" Abandoned mine would've been fine. But now you've communicated the value of your find and given a basic hint of what mines to look up. The bright side is so far, author is probably fine because nobody's buying quantum crystals yet in a futures market.

At least the author has time to secure property rights and buy out old mines.

Awesome!! Super excited to try this out. Amazing work.

I've been using obsidian for years as a paying customer. Will continue to pay as price point is good and it just works. However, unless plugin security massively improves I will never install any plugins.

This is why I NEVER install any obsidian plugins - EVER. Obsidian itself is good. Obsidian plugins = security nightmare.

I did not realize Boris made the tutorial that got me into programming. Wow. What a great find! And is the Ilya S Ilya Sutskever?

This is insanely cool. Thanks for sharing. I'll follow you on https://muppetlabs.com/.

I'm learning how to train transformer models locally to do useful work instead of having to pay for claude. I regularly update my blog here https://seanneilan.com/posts

If it can be used for inference, who cares right? Just have claude vibe code some objective-C or run the Enclave app.

That's very funny. Bravo.

This is exactly what I needed. I've been thinking about making this tool. For running and experimenting with local models this is invaluable.

Does anyone have Jeff Dean's sources for how he computed these numbers? What's the margin of error? How accurate are they now? Is there a set of numbers that also talks about memory bandwidth in GPUs? Are these numbers intel/amd only? How do they differ between an m1 architecture?

I'm a lot happier now being an anthropic customer.

Yes, this is probably the piece I am not realizing. However, there is no better approach to getting more capital than by scaring people?

Yes, the CEO's of these AI companies are clearly not the people who should be selling AI products. They need to be hidden away and kept behind closed doors where they can do their best work. And they need advertising companies, PR firms and better marketing tactics to try and soothe the customers.

I don't understand why some of these AI companies check their egos at the door and hire public relations companies. Yes, I understand they are changing the world but customers do not open their wallets when they are scared. Very few people I know are as avant-guarde as I am with AI, but, most people look at these new technologies and simply feel fear. Why pay for something that will replace you?

Learned a lot about local models using them to scrape target.com.

I love it! I'm at level 6 and brave enough to try. I'm in. Giving this a shot!

This wins the internet today. Amazing work!!!

Published an edit today (post dated in Nov. but I've rewritten it 5x now) on my tutorial to use llama3.2:3b to generate fine tuning data to train tinyllama1.1b https://seanneilan.com/posts/fine-tuning-local-llm/ It took a while to figure out that when I made llama3.2 generate json, it didn't have enough horsepower to generate training data that was varied enough to successfully fine tune llama1.1b! Figured that out :) Something you never learn with the bigger models. Every token costs something even if it's a little bit.

https://seanneilan.com

I'm so confused. Why would the United States care about people's health? It feels out of character for this administration given the times.

Have you found any models that work better for your use case?

Does anyone have any information on how much this will cost? Or is it one of those products where if you have to ask you can't afford it.

You know what, you have a very good point. I'll give this another shot. Maybe it can be fast enough and I can just isolate the orm queries to some kind of repository pattern so I'm not testing sql queries over and over.

This is the way.

The other unspoken aspect of this is with agentic coding, the ability to have the ai also test queries quickly is very valuable. In a non-agentic coding setup, mongomock would not be as useful.

Not familiar with OrioleDB. I’ll look it up. May I ask how this helps? Just curious.

True but if you wind up using parts of postgres that aren't supported by sqlite then it's harder to use sqlite. I agree however, if I was able to just use sqlite, I would do that instead. But I'm using a lot of postgres extensions & fields that don't have direct mappings to sqlite.

Otherwise SQLITE :)