feat: ctransformers support async chain #6859
Conversation
semoal
commented
Jun 28, 2023
semoal
commented
- Description: Adding async method for CTransformers
- Issue: I've found impossible without this code to run Websockets inside a FastAPI micro service and a CTransformers model.
- Tag maintainer: Not necessary yet, I don't like to mention directly
- Twitter handle: @_semoal
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
semoal
force-pushed
the
semoal/ctransformers-async
branch
from
fcd4255 to
ff13a40
Compare
QodoAI-Agent
left a comment
QodoAI-Agent
left a comment
There was a problem hiding this comment.
PR Analysis
- 🎯 Main theme: Adding async method for CTransformers
- 🔍 Description and title: Yes
- 📌 Type of PR: Enhancement
- 🧪 Relevant tests added: Yes
⚠️ Unrelated changes: No- ✨ Minimal and focused: Yes, this PR is small and focuses only on adding async method for CTransformers
PR Feedback
-
💡 Suggestions: The PR is well-structured and follows best practices. The new feature is well-documented and the added test ensures its functionality. However, the async method
_acallis currently private (starts with_). If it's intended to be used by the end-users, it should be a public method. -
🌱 Minor suggestions: The PR checklist in the description is incomplete. Please ensure to fill it out completely for better tracking and understanding of the PR.
-
🤖 Code suggestions:
- In file
langchain/llms/ctransformers.py: The async method_acallis currently private. If it's intended to be used by the end-users, consider renaming it toacall. [important]
- In file
|
Going to fix lint step asap 😅 |
semoal
force-pushed
the
semoal/ctransformers-async
branch
from
ff13a40 to
acf28e9
Compare
|
Fixed @hwchase17 :) thanks to you btw |
semoal
force-pushed
the
semoal/ctransformers-async
branch
from
acf28e9 to
ddd4e57
Compare
| """ | ||
| text_callback = None | ||
| if run_manager: | ||
| text_callback = partial(run_manager.on_llm_new_token, verbose=self.verbose) |
There was a problem hiding this comment.
@agola11 if i understand correctly, we don't need to pass self.verbose in to run manager anymore right? since relevant callback handlers are added during Chain init if verbose is set?
|
Can we merge this? |
|
done, thanks @semoal! |
5 participants
