htsh

curious, how does one configure something like this for AWS Lambda? Appreciate it.

I am assuming it is a bunch of manual work.

Is it safe to assume they will come down in 2027?

i much prefer svelte but the rest of the world does not

yes! especially b/c i want to process a lot of email and directories full of old, personal documents

are we sure the RAM market will stop being insane in a year or two or could this be the new norm?

thanks! came in here to ask this.

we can do much better with a cheap model on openrouter (glm 4.7, kimi, etc.) than anything that I can run on my lowly 3090 :)

I have been doing this with claude code and openai codex and/or cline. One of the three takes the first pass (usually claude code, sometimes codex), then I will have cline / gemini 2.5 do a "code review" and offer suggestions for fixes before it applies them.

curious, why the 30b MoE over the 32b dense for local coding?

I do not know much about the benchmarks but the two coding ones look similar.

A lot of us have ryzen / nvidia combos... hopefully, soon, though.

assuming you want to run entirely in GPU, with 12gb vram, your sweet spot is likely the distill 14b qwen at a 4bit quant. so just run:

ollama run deepseek-r1:14b

generally, if the model file size < your vram, it is gonna run well. this file is 9gb.

if you don't mind slower generation, you can run models that fit within your vram + ram, and ollama will handle that offloading of layers for you.

so the 32b should run on your system, but it is gonna be much slower as it will be using GPU + CPU.

prob of interest: https://simonwillison.net/2025/Jan/20/deepseek-r1/

-h

As a longtime user of nodemailer, thank you.

I am gonna check out emailengine for future work.

Dreambooth was kinda great?

That said, I agree that I wish there were more done post-research towards products with some of this stuff.

Yes, offloading some layers to the GPU and VRAM should still help. And 11gb isn't bad.

If you're on linux or wsl2, I would run oobabooga with --verbose. Load a GGUF, start with a small number of GPU layers and creep up, keeping an eye on VRAM usage.

If you're on windows, you can try out LM Studio and fiddle with layers while you monitor VRAM usage, though windows may be doing some weird stuff sharing ram.

Would be curious to see the diffs. Specifically if there's a complexity tax in offloading that makes the CPU-alone faster but in my experience with a 3060 and a mobile 3080, offloading what I can makes a big diff.

openrouter, fireworks, together.

we use openrouter but have had some inconsistency with speed. i hear fireworks is faster, swapping it out soon.

Can one enter their own opeanai URL and api-key? (so we can use openai-compatible things like openrouter or lm-studio)?

That is what the RAG system does. The PDF is chunked and thrown into a vector store. And then when prompted, only the relevant bits are retrieved and stuffed into the context and sent to the LLM.

So yeah it's kinda smoke and mirrors. In some cases, for some long PDFs, it works really well. If it's a 500 page PDF with many disparate topics, it may do fine.

Cool! Any plans for Svelte?

supabase just added their $599 tier for their soc2/hippa compliant product. really appreciated that.

This subreddit remained open. Unfortunately, however, the oobabooga one went closed for a while and lost a lot of momentum. It is also back, however.

Are there good lemmy spaces for LLMs?

I know how that works. And my point was not that they should or will be replaced, but rather that they are no less expendable than developers (not very much).

But the decisions they make are one of the things that can be automated. I do not know if you have been inside one of these places but the executives are not doing a great job deciding (at mine they decided opensearch was a better bet than elastic and switched existing installations).

A new regime came in and then bad decision after bad decision drove our best talent away. Consultants, everywhere.

Also, that number is much lower. Full time devs are down, contractors and consultants are up. As a full time dev at one of these places, it felt like the number of executives was growing as everything else shrank.

Perhaps you are right about the highest levels, but think about all of the middlemen executives and what they do.

And even that -- I think an AI could choose to not spend millions on Deloitte or Accenture on software that inevitably failed.

Having just left a large enterprise, it certainly feels like executive jobs are replaceable soonest with the AI tech available to us now.

Not sure why those of us that live in code editors or even Excel should worry about our jobs more than those that live in Powerpoint.

I moved back to NYC after a long time away, and to a different part of town than where many of my old friends live, and getting a dog considerably improved my connection with the folks around me.

And of course, as others have said, volunteering.

1. go to a place where folks you want to hang out with live

2. get a dog

Oh cool, I do not use the site and that was what I was wondering. I was curious if a host could click on a user and see all of their reviews.

But could a future host see that they posted a negative review and not rent to them, fearing a honest review themselves?

1. the market being so hot for mid-level engineers means that we are considering and hiring good junior engineers where we cannot find adequate mid-level talent. we would rather hire and develop a junior than compromise.

2. show code on github, sure, but if you can demo the work deployed somewhere it goes much further. everyone has a crypto tracking or twitter sentiment project somewhere, but i need more. if you can deploy the work somewhere in a smart way, great. if you can communicate well, great. a project with a 2-5 minute video tells me a lot more than code alone (which anyone can do with a tutorial).

3. demonstrate that you are easy to work with and that you are willing to learn. if a candidate has enough code or work on their resume, that is great, but a good cultural fit goes a long way.

They ask you to install an extension but the web lookup is still available here:

https://www.fakespot.com/analyzer

Oh no, apologies for the lack of clarity. I would not get the whole thing and build it in the application layer before sending it to the client.

If I want a particular array item nested a few levels in a JSON, the query as JSON object instead of query as an SQL is just a bit easier to put together (which is what i mean by pure JS). Sometimes, putting complex queries together takes fewer lines of code. Neither is really better or worse.

But I want the database doing the work and want to transform as little as possible on the application layer. Sometimes it is inevitable, but sub-optimal, for sure.

In terms of anything where you need to do a lot of joins when querying -- NoSQL is not good and I would not advise Mongo at all. It depends on what you are building. A lot of applications (and even games) now just pass JSONs around. If I show a user profile, I get that json. If, on the profile, I need to show where she lies in a leaderboard, that react component hits a separate API call to get the data needed for it, etc.

But no difference really. And yes, if the JSON is just a column in postgres, even the early stuff is nearly as easy as it is in postgresql.

If I were looking at a DB for a simple key/value store, I would not look at Mongo. Others here likely know better, but I pick up Redis (out of habit, but also it is awesome even if overkill in many scenarios). Mongo would do it but I would not do so except temporarily. I tend to throw Redis in my stacks early in development as I tend to need it later.

BTW by no means do I love Mongo. For example, Mongo is awful when it comes to things simiilar to joining in SQL land. You end up in these hellscapes where there is no choice but to store a ton of redundant data. Is it the end of the world? No, storage is cheap. But is it good? No.

And I am old so I do not find the value in arguing about tools. It is like me hating a hammer. I am interested in what is better and what folks here think about things coming down the pipeline, or whether Arango is worth looking at, etc. But I do appreciate your sort of inquiry as I have learned from it -- for example I did not know you can index inner fields of JSONs. I really need to revisit is as recent versions of PG seem to get better and better.

Context matters a ton. If you are going right to production with something, that is not really where I operate. But in general, as many have said, if more than one thing will do it, the best tool is the one you know best. You are likely to implement it efficiently, etc.

I agree that it is very easy to make bad decisions with this product, but I do not think it is unique to Mongo. And it is the role of a good architect or a good team lead to make sure that this does not happen or that developers do not go down these roads.

I would even say the strengths that I have pointed out are precisely what makes it easier to take shortcuts. It gets out of your way, makes collections if they do not exist, allows for different document types in the same collections. It has no guard rails whatsoever.

I think we are in agreement -- one needs to be very careful using this. You can use it and put yourself in a spot where you have to throw out your entire codebase.

Well, me saying "can happen" does not mean it does not happen or that people have not seen it. It is saying the opposite. My point is it does not have to or that it does not always happen.

Developers take shortcuts, often because they do not know better, and you are right that some products make it easier to go down these roads.

I still do not think you want to use medical diseases as metaphors. You would not say "mongo aids" or "mongo alzheimers" or "mongo diabetes." Even if it is common, it is not particularly useful. Metastasis is better.

Querying is in pure JS, easy to do stuff programmatically. But if starting new, with a team that did not know Mongo well already, not a real advantage. I still prefer querying something nested in mongo to querying something nested in JSONB, but I can work with either.

And back when I started with it, JSONb did not exist. I would say JSONB was also a godsend though as our roadmaps now have a clear path to something we may prefer later. People up the chain are often terrified when they hear the word "mongo" so the existence of JSONB makes it easier to swap out your layer that handles database interactions.

But the speed advantage really lies in the early-stage prototype development. If I have to do it in postgres, the amount of code necessary up front, when inevitable schema changes happen, etc. makes mongo preferable until things settle down.

But again, it all depends on how well you know the thing up front. If you can anticipate a lot of iteration on the data model as you learn how people use your product, mongo is handy early. If I am given a spec for something where all of the things are known and foreseen, I am going with postgresql.

I assume that the hardest thing to do is making something people want to use. If I can achieve that with whatever I use, I have some good (and fun) problems on my hands. And it is all about structuring the work so that I am not tied to any database product. I am no longer working on startups with little-to-no funding, so right now I assume AWS or Azure will manage a database better than I can. And I assume new things are always coming.