propagate callbacks to ConversationalRetrievalChain

[alecf]

Allow callbacks to monitor ConversationalRetrievalChain

I ran into an issue where load_qa_chain was not passing the callbacks down to the child LLM chains, and so made sure that callbacks are propagated. There are probably more improvements to do here but this seemed like a good place to stop.

Note that I saw a lot of references to callbacks_manager, which seems to be deprecated. I left that code alone for now.

Before submitting

Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
@agola11

[dev2049]

you can pass in a CallbackManager at runtime (see L90 for example) that propagates (inheritable) callbacks to child chains. would that work, or is there a need to pass them in when initializing child chains?

[alecf]

thanks @dev2049 !

you can pass in a CallbackManager at runtime (see L90 for example) that propagates (inheritable) callbacks to child chains. would that work, or is there a need to pass them in when initializing child chains?

Yes, I suppose that would work, though not what I would expect to be the accepted pattern.

I'm not 100% familiar with the langchain conventions, and my intention is to use callbacks for monitoring/etc. To me I would expect that when you "set up" a chain (i.e. create one with ConversationalRetrievalChain.from_llm, etc) that callbacks are a sort of background thing that just works whenever you run the chain... and that once the chain exists, you don't have to think about monitoring any more. And from that perspective, it seems like it would be the responsibility of langchain to "wire up" the callbacks that are passed in.

WDYT?

[dev2049]

WDYT?

i think i agree!

you technically can pass in CallbackManagers (with inheritable callbacks) when initializing an object, but i'm pretty sure we ignore inheritable callbacks unless they're passed in at runtime. @agola11 thoughts on changing this behavior?

[agola11]

Either passing through the callbacks to constructors or initializing an object with a callback manager (specifying inheritable callbacks) will work. This seems fine to me

[samthedataman]

Can you some one post just bare bones python code that exemplifies calling the openai api and using this method and print the chat history + answers.

I'm confused why this is so confusing. This is my current code

condense_prompt = PromptTemplate.from_template()
combine_docs_custom_prompt_og = PromptTemplate.from_template()

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

chain = ConversationalRetrievalChain.from_llm(
ChatOpenAI(temperature=0.3),
vectordb.as_retriever(), # see below for vectorstore definition
memory=memory,
condense_question_prompt=condense_prompt,
combine_docs_chain_kwargs=dict(prompt=combine_docs_custom_prompt),

how do you return the chat history?