sillysaurusx

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353...

Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve

It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)

It's an unfortunate outcome. Now to be a big player in AI, you have to have enough capital to buy your own library worth of books and digitize them. (Fun fact: a pallet of books is called a "gaylord," and they buy hundreds of gaylords.)

I created books3 to help settle the question of whether AI companies should be allowed to train on books. The outcome of "it's okay to pirate books as long as you're only training on them" was a long shot, but it would've let individual hackers train their own AI models (assuming access to sufficient compute, which you can get e.g. via https://sites.research.google/trc/about/).

Now we're in a world where you have to have dozens of millions in capital to do substantial work.

I heard at one point Eleuther was gathering public domain training data. I wonder if they ever built a corpus large enough so that training on books doesn't really matter...

Are your comments no longer pinned at the top? I was surprised to see this so far down. It was what I needed.

Whoops. I've been using a fork of HN to manage my job interviews, and this time I accidentally submitted it to the real HN. I was very confused when I couldn't edit the story text.

Apparently you can no longer delete submissions, so I've deleted the Notion page.

Best wishes, whoever reads this.

(FWIW my exchange with you was enjoyable and I certainly didn't take any of it as an attack.)

I'm actually not sure how to do that. Do you mind posting some steps? That'd be useful to know in case I ever do lose access to Claude.

Exactly. I don’t fault them for it, but it’s a scary experience having Claude shut off.

Thanks for telling me about Inference Endpoints. That’s awesome.

So glad local models are getting good enough to be deployed. The uncensored model’s output was far better than expected in a domain that triggers guardrails with ChatGPT and Claude.

That is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.

It’s a PITA to offer a language model as a service. You’d need a beefy server, at minimum.

This particular use case might work, since no one can write fast enough to consume too many tokens — the whole session should fit in the context window. But you’ll need to handle all the people connecting to your service indefinitely, which will become expensive for a hobby project.

But sure, theoretically you could deploy it if you have resources. I’m not sure what you’d use to create instances of chat sessions, or if llama.cpp offers an API you can build the app on top of (probably) or whether that’s a workable solution.

As far as I can tell, Claude flagged me as high risk of suicide and then Anthropic issued a ban later on.

It wasn’t one prompt, it was a detailed conversation where I was trying to find out the exact dosage of barbiturates that assisted suicide programs use.

It would be ironic if you have to actually sign out from Twitter, not just use incognito mode, to bypass the signup nag. Or if they mark IPs as “signed up” or something. Thanks.

Thank you. As someone who tweets, this is helpful to know.

Every AI would refuse the prompt. I was banned for researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Claude refused, alerted Anthropic, and I was banned a couple days later. Thankfully the appeal form worked, but by then I was using a different Claude account praying they didn’t ban me again.

There’s an uncensored model floating around that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service).

For better or worse, fun is no longer allowed. It coincided with “AI psychosis” being coined as a term.

Because Twitter posts consisting of YouTube links die quickly. It’s very obvious once you have a few thousand followers.

Your option is basically either upload to twitter, or put the YouTube link at the end just before a screenshot. Or both a video and a YouTube link, I suppose.

If you trigger their YouTube embed, it seems like it gets penalized quite harshly. I’ve seen other people agree with the sentiment.

Can anyone verify that you can/can’t see a video if you’re not logged in?

Clickable https://twittervideodownloader.com

Works well. Bookmarking this.

They’re saying if the average task you actually use the model for is far less difficult than the benchmarks, you might incorrectly conclude that the model is costly when in fact it’s the best performing model for your actual use case.

I want a model that generates commit messages fast. Currently I have to wait up to a minute or two. That model doesn’t need to score very highly on SWE benchmarks, just highly enough that it can write out a good enough message in a few seconds. If you tested it on ${current top tier benchmark} you’d think it’s way too costly when in fact it’s the best tradeoff.

I was confused too. What they’re saying is, the average task you’re likely to do if you buy the model is the main predictor of costs. So if the average task benchmark is far higher than what you’re normally doing with it, you get a skewed perspective.

I’ve wanted a fast model to generate commit messages. No idea what that would be, but it doesn’t have to pass the SWE benchmarks very well. Just well enough that it understands the codebase.

Concrete example: I’ve been trying to use Claude to generate all my commit messages, but it takes 5-10x longer than if I just write them myself. Mine are less detailed, but one line changes are sometimes inconsequential (especially white space reformatting). I wish there was a model that understood the codebase well enough to generate commit messages in half the time.

A lot of people come to HN for the comments. It’s often useful to gauge a story by public sentiment first.

That said, you’re ultimately correct that it’s in the article, but I appreciated it. :)

Greenboots is so iconic. Other people use him as a marker. Glad he got some attention. It’s always seemed a shame that it’s impossible to give him a proper burial.

Fair, to be honest.

Location: Lake Saint Louis MO

Remote: Yes

Willing to relocate: yes

Technologies: Python, PyTorch, JAX, C++, JS/TypeScript, Objective C, Swift, Rails, Linux, Lisp, and anything else you give me a week to learn.

Résumé/CV: https://shawwn.net/docs/shawn-presser-resume.pdf

Email: shawnpresser@gmail.com

I was employee #2 at Carmack’s AI lab, Keen. I worked at Groq helping to design and implement their LPU chips. I made Books3, a dataset trained on by LLaMA and Claude. I’ve been programming for 25 years.

Please, email me with literally anything involving computers. I’ll do well, and I can learn and adapt to any working style you prefer.

I find it strange you’re arguing for something beyond the licensing terms. Imagine if companies were run that way. "Hey, we did an especially good job mowing your lawn, so please pay us double. You have a rich house, you can afford it. Oh and if you don’t pay us, you’re a bad person and should feel bad."

This is exactly the same argument: you’re saying the open source maintainer who knowingly released their code as open source (and got famous for it being open source) should be paid way more than they asked for ($0) based on vibes. Society doesn’t work that way. Companies don’t work that way. And it’s baffling people are saying open source should work that way. He already got the fame and free publicity from being the maker of Box2D, which he wouldn’t have gotten unless he released it for free. You can’t get the major benefits of that and then ask for a million dollars because "kindness".

If you want to be professional, keep it professional. Otherwise everyone here saying that the company should feel bad are fooling themselves. You’re owed what you ask for.

Tipping culture has obscured this somewhat. You’re supposed to give more money if they do a really good job. But it’s that way so that the business can pay employees less money. Tipping, like identity theft, is one of the most successful marketing campaigns of all time: you’re considered a bad person if you don’t tip, and that it’s your fault if your bank fails to verify your identity. Both of these are bogus.

I tip, because 15% is the normalized rate. But it shouldn’t be our problem. It’s the company’s problem to pay their employees. And it shouldn’t be the business’s problem that they made a lot of money using something that was knowingly given away for $0.

I find it baffling in the other direction. The whole point of licensing is to ensure you’re a decent person.

If you’re creating something new and you dare try to copy anything without a license, the pushback is unanimous and universal: everyone agrees you’re a bad person for even thinking of it.

This is the same argument in the opposite direction. By fulfilling the terms of a license, you’re ensuring you play by the rules. In fact, licenses are the only real protection that open source maintainers have.

So why diss on someone for following the license terms? This whole moralizing and tut-tutting is a weird branch of the convo.

It lists the original source as http://quotes.cat-v.org/programming/ but I found these to be presented in a more engaging way.

Jinx personal jinx no jinxback.

For western reference, ¥100,000+ is $615+, or €540+.

Not cheap. I managed to find a room in SF for like $250 when I was looking, though you’d have to reserve it a week in advance. Is Tokyo really that expensive now?

Thanks for doing this. I had no idea the system prompt was embedding things like "avoid abstractions; three similar lines of code are better than one helper." Stuff I disagree with.

Is there a way to modify these prompts e.g. by putting instructions in CLAUDE.md to override it? I know it won’t directly modify the system prompt, but it seems like CLAUDE.md should have the final say, shouldn’t it?