LangChain Is a Black Box

https://news.ycombinator.com/item?id=41192069

by valianter • 2 years ago

42 27 2 years ago

news.ycombinator.com

I've been working with Langchain to develop a POC and when I try to understand the bugs in my code I feel like I am going down this huge rabbit hole of code. Every time I click into a piece of code there are 10 more places that I have to look that MIGHT be the source of the bug.

I get the idea that things should be abstracted away and that's the point of a library but this feels like a little much.

phren0logy3• 2 years ago

Oh my god, I feel exactly the same way. I am not a programmer, but I'm excited by the potential of LLMs in my work. So, I've dusted off my python skills that I haven't used since I was in an fMRI lab, and experimenting to see what works.

First of all, every library/framework I've found is moving so fast that all the tutorials and printed material (O'Reilly books etc) are already out of date. Many of the changes are out of necessity, as it's a rapidly developing space, but sometimes it just feels like someone got high and decided to add 3 more layers of abstraction. Although for many tasks, AI coding assistants would be a benefit for noobs like me, the code base and documentation are too loose for me to get the expected benefits I would find in a more established code base.

LangChain seems to be where a lot of the action is with regard to modularity, and using different components in each part of the pipeline. That's important for me, because I need either local or HIPAA-compliant tools (Azure OpenAI works, Anthropic won't return my requests for a BAA, and I need a bigger GPU).

But using LangChain is a pretty horrible experience because, at least for my uses and as a noob, it's much too buried in abstractions to make quick iterations. The GUI-based stuff like flowise and langflow are too limited with regard to available components, and mostly they hide the problems so that errors are tough to address.

I'm thrilled that there has been so much work on adding JSON output and agent stuff at the LLM level, as hopefully it can bring some of these astronauts back to earth (or at least in a low orbit).

phren0logy3• 2 years ago

Just in case anyone comes back to this, I found the following resource helpful:

Brandon Hancock’s (3 hour!!) YouTube video and accompanying GitHub repo: https://github.com/bhancockio/langchain-crash-course https://youtu.be/yF9kGESAi3M

I’m not in a position to say, but perhaps many people will ultimately go with directly using the python libraries and APIs of their components. But if you are messing around with different components until you settle on something, it’s useful.

jasondclinton• 2 years ago

Hi, CISO at Anthropic here. Sorry that we didn't respond to your BAA request. I am accountable for our response to BAA requests and I'd like to dig into what happened here. If you are comfortable, would you please reach out to me at j@anthropic.com to let me know how you sent your request in?

muzani• 2 years ago

Yeah, it's a meme by now. Langchain did not grow well with the new changes in AI. Generally avoid it where possible. It still has uses, but treat it as legacy code.

Ldorigo2• 2 years ago

We're moving our entire codebase away from Langchain and related products (langgraph, langsmith). I originally thought the library had potential and just needed some polishing which would come with time, but it's just been getting worse and worse. LCEL is the weirdest half-baked idea; it's a nice gimmick that makes your code impossible to debug in a step through debugger. Random stuff breaks with every other release. I deeply regret using it by now.

maeil• 2 years ago

Are you replacing LangGraph with something else? I've luckily avoided LangChain exactly because it's a black box, but been on the lookout for some kind of DAG visualization of LLM flows. LangGraph pops up first on any related searches.

jackmpcollins2• 2 years ago

I haven't used LangGraph myself, but the latest magentic release is compatible with it if you'd like to check out the examples here https://github.com/jackmpcollins/magentic/issues/287

Ldorigo2• 2 years ago

We're not sure what to do with langgraph yet, interested if anyone here has tried viable alternatives. We could implement a simplified execution graph in-house, but it feels like reinventing the wheel. Also thinking to keep the "graph" part of langgraph but replace all everything else (i.e. tool calls, retrievers, etc.) by non-langchain alternatives.

e1gen-v• 2 years ago

I tried using it for a simple POC and was very frustrated with it. Now we just use raw api calls, and a state machine to create our dag. Much simpler and very extensible.

specproc• 2 years ago

Langchain has the odd nice component I use, I prefer their recursive text splitter to anything else I've seen or cobbled together, but I don't find the code readable or the abstractions particularly intuitive.

ujjwalk304• 2 years ago

The reason I use pinecone without langchain and people call me crazy

danielbln• 2 years ago

Langchain is a spaghetti rabbit hole. In our consultancy, we've basically switched entirely to Haystack, and keep the abstractions fairly light, really only use it to give the pipelines some structure that applies across projects. Very easy to add custom components with pretty straightforward interfaces. The way you assemble the components isn't everyone's cup of tea (string references), but it does make it very easy to understand at a glance what each pipeline is doing and when. Worth checking out if Langchain isn't doing it for you.

toomuchtodo• 2 years ago

https://github.com/deepset-ai/haystack

vortegne• 2 years ago

Last I checked Langchain was just basically a pile of helpers? There isn't really a coherent model of anything that's presented to the user, just some light wrappers around stuff. I played around for a bit, it was fine, but I opted to write my own helpers in the end.

Did I miss out on some major developments there? Because I don't see why it's a thing that's being talked about everywhere, when it's barely anything.

meiraleal• 2 years ago

Langchain is not just basically a pile of helpers. It is *just basically a pile of helpers with 35M in investment*.

https://venturebeat.com/ai/langchain-lands-25m-round-launche...

_andrei_• 2 years ago

Get rid of it, Langchain really doesn't do anything you can't easily do yourself.

samier-trellis• 2 years ago

It is indeed a terrible library on multiple levels. Get rid of it. Just use the "native" Python SDK for the LLMs you are using for your POC.

jackmpcollins2• 2 years ago

I've built a lightweight package that provides a standard interface to the LLM providers, as well as taking care of boilerplate around structured outputs, function calling, and opentelemetry/tracing. It's hopefully a good compromise between ease-of-use and complexity.

https://github.com/jackmpcollins/magentic

efriis• 2 years ago

Howdy! Erick from LangChain here. Would love to learn a bit more about your bug hunting process and see how we can improve the debugging process for everyone else! Could you email me at erick@langchain.dev?

siamese_puff• 2 years ago

Erick, please just read this entire thread.

choilive• 2 years ago

Langchain is frequently crapped on here on HN. Do some searching and you'll see it's pretty much a universal sentiment that langchain is terrible and you shouldn't use it.

MH15• 2 years ago

Remember than LangChain raised $10m a few months ago https://blog.langchain.dev/announcing-our-10m-seed-round-led... (wait, over a year ago!)

glorpsicle• 2 years ago

Ugh, I feel exactly the same way. In fact, I've spent the last 2 weeks refactoring our entire LLM pipeline to completely root out LangChain. I'm just implementing basic RAG from scratch, which 1) has me learning a lot more quickly and 2) feeling much more in control of the entire pipeline.

sleepytree• 2 years ago

How are you running the pipeline exactly? I'm new to AI dev and when I hear pipeline I think either CI/CD or something like Apache Airflow or Kestra.

imtringued• 2 years ago

I don't know if this will help you, but have you tried langroid?

Even if you don't want to use agents, it is still useful as a convenient library for calling an Open AI compatible endpoint.

https://langroid.github.io/langroid/quick-start/llm-interact...

d4rkp4ttern• 2 years ago

Thanks for mentioning Langroid[1], I am the lead architect/dev. We are starting to see companies use it in production after evaluating alternative frameworks, whether agent-oriented or not.

We started building Langroid in early 2023 after finding existing frameworks lacking in terms of good dev-ex, extensibility and clarity of code. We prioritize code transparency, flexibility, stability, good test coverage. We designed it to be agent-oriented from the start, with an elegant agent orchestration mechanism loosely inspired by various paradigms such as Blackboard Architectures, Actor Model, and Production systems, and process calculi. But as you said, Langroid is useful even if you want to just use a single LLM conversation-state.

[1] https://github.com/langroid/langroid

phren0logy3• 2 years ago

Just in case anyone comes back to see this: I found the following (3 hour!!) YouTube video to be really helpful. It starts slow, but skip ahead and watch in 2x until you get to what you need.

https://youtu.be/yF9kGESAi3M?feature=shared

LangChain Is a Black Box

Related Stories