update MosaicML inputs and outputs #7348
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
Collaborator
|
makes sense to me, cc would be great if you could double check @dakinggg |
margaretqian
approved these changes
Jul 7, 2023
Contributor
margaretqian
left a comment
margaretqian
left a comment
There was a problem hiding this comment.
Thank you for making this change! Apologies for the API changes, the version in the docs will be stable moving forward. We currently have backwards compatibility wrt the changes here, but were also in the process of making the update to the new API, so thanks for helping out here! We'll will propagate the same change to the embedding models shortly as well.
Collaborator
|
thanks @pmeier! |
baskaryan
added a commit
that referenced
this pull request
Aug 25, 2023
As noted in prior PRs (#6060, #7348), the input/output format has changed a few times as we've stabilized our inference API. This PR updates the API to the latest stable version as indicated in our docs: https://docs.mosaicml.com/en/latest/inference.html The input format looks like this: `{"inputs": [<prompt>]} ` The output format looks like this: ` {"outputs": [<output_text>]} ` --------- Co-authored-by: Bagatur <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As of today (July 7, 2023), the MosaicML API uses
"inputs"for the promptbut we have
https://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L125
Furthermore, the output should be inside the
"outputs"keybut we have
https://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L163-L170
This PR adds support for this new format.
<rant>It is beyond me, why they apparently have versioned their API (endpoints look like
https://models.hosted-on.mosaicml.hosting/mpt-7b-instruct/v1/predict; note thev1in there), but just keep changing the API without upping the version number. There is a good reason forhttps://github.com/hwchase17/langchain/blob/0ed2da70200ada39f5319f604b4c5248bc98e81b/langchain/llms/mosaicml.py#L160-L161
</rant>CC @hwchase17, @baskaryan