I think this is the reasonable answer. Langchain gets a lot of derision and rightly so but it does have uses for prototyping. It’s also a good resource for learning the landscape specifically because of the integrations. I haven’t used it in a while so I’m not familiar with the most recent updates.

Thanks for explaining.

If you are using Python, check out the package I've been building, magentic https://github.com/jackmpcollins/magentic It supports structured outputs and streaming, and aims to avoid making unnecessary abstractions (but might require some more understanding of LLM patterns as a result).

Also recently released is pydantic-ai, which is also based around pydantic / structured outputs, though works at level of "agents". https://github.com/pydantic/pydantic-ai

Frankly, just Python. LLM generation is just a function call, fetching from a vector db is just a function call.

LLMs are hard to tame at scale, the focus is on tightly controlling the LLM inputs, making sure it has the information it needs to be accurate, and having detailed observability over outputs and costs. For that last part this new wave of AI observability tools can help (Helicone, Langsmith, W&B Weave...).

Frameworks like LangChain obscure the exact inputs and outputs and when the LLM is called. Fancy agentic patterns and one-size-fits-all RAG are expensive and their effectiveness in general is dubious. It's important to tightly engineer the prompt for every individual use-case and to think of it as a low-level input-output call, just like coding a good function, rather than a magical abstract intelligent being. In practice, I prefer to keep the control and simplicity of vanilla Python so I can focus on the actually difficult part of prompting the LLM well.

To be honest, I think a lot of people realize that what Langchain is doing is providing a small amount of value in the form of a huge amount of abstraction, which means it can be convenient for very simple off the shelf solutions, and a huge headache for anything else. Most people realize the value add if Langchain can be recreated with a few lines of code, and end up just building their own.

> No one ITT is saying what the thing that replaced LangChain is.

LangChain never solved a real problem to begin with, so there's nothing that needs to be replaced.

Just write your own Python code that does the same thing that LangChain needs 10 layers of abstraction to do.