propagate callbacks through load_summarize_chain

[alecf]

This lets you pass callbacks when you create the summarize chain:

summarize = load_summarize_chain(llm, chain_type="map_reduce", callbacks=[my_callbacks])
summary = summarize(documents)

See #5572 for a similar surgical fix.

tagging @hwchase17 for callbacks work

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)			Jul 24, 2023 9:11pm

[baskaryan]

this introduces the converse problem, of making it hard to create a map reduce docs chain with only callbacks on the MapReduceDocumentsChain.

maybe we call this arg something like shared_callbacks? and for the final map reduce chain we check if kwargs contains 'callbacks', and only used shared_callbacks if it doesn't

[alecf]

is that kind of flexibility thing that people want? I think this is an area where I'm a little confused on best practices in langchain.

What I feel like I'm seeing in most of the codebase is that callbacks are passed into agent/chain/llm constructors or utility functions, and that means "all children of this object get the same callbacks" are passed down to child chains / llms / etc.

[alecf]

It might be worth noting that the docs call out this general pattern here:

The callbacks argument is available on most objects throughout the API (Chains, Models, Tools, Agents, etc.) in two different places:

• Constructor callbacks: defined in the constructor, eg. LLMChain(callbacks=[handler], tags=['a-tag']), which will be used for all calls made on that object, and will be scoped to that object only, eg. if you pass a handler to the LLMChain constructor, it will not be used by the Model attached to that chain.
• Request callbacks: defined in the run()/apply() methods used for issuing a request, eg. chain.run(input, callbacks=[handler]), which will be used for that specific request only, and all sub-requests that it contains (eg. a call to an LLMChain triggers a call to a Model, which uses the same handler passed in the call() method).

The verbose argument is available on most objects throughout the API (Chains, Models, Tools, Agents, etc.) as a constructor argument, eg. LLMChain(verbose=True), and it is equivalent to passing a ConsoleCallbackHandler to the callbacks argument of that object and all child objects. This is useful for debugging, as it will log all events to the console.

i.e. the notion of "shared" callbacks is the default rather than the exception.

This particular root function (load_summarize_chain) is technically not a constructor but it is doing a tremendous amount of scaffolding and calling constructors that would otherwise take callbacks.

The way this case seems to be addressed elsewhere in the codebase is to add a xxx_kwargs parameter, e.g. agent_kwargs. So here you could have doc_chain_kwargs but personally that feels like an awkward way to change this interface to support what would seem to me to be a somewhat esoteric case (supporting only callbacks on MapReduceDocumentsChain)

[alecf]

@d1sounds - this is an example of some of the callbacks propagation I've been working on

[hwchase17]

thanks!