Added AzureML Endpoint Integration for Azure Foundation Models #6493
Conversation
Add AzureML Online Endpoint for LLM integration on foundational models
|
@matthewdeguzman is attempting to deploy a commit to the LangChain Team on Vercel. A member of the Team first needs to authorize it. |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
overall looks good, just a few quick q's! |
dev2049
left a comment
dev2049
left a comment
There was a problem hiding this comment.
few more small comments but basically there!
| class OSSContentFormatter(ContentFormatterBase): | ||
| """Content handler for LLMs from the OSS catalog.""" | ||
|
|
||
| content_type = "application/json" |
There was a problem hiding this comment.
nit: no need to specify since it's default (here or in other child classes)
There was a problem hiding this comment.
note if you're setting api key as env var you don't need to pass it in when initializing class (you can drop endpoint_api_key=os.get("ENDPOINT_API_KEY"))
| def validate_client(cls, field_value: Any, values: Dict) -> AzureMLEndpointClient: | ||
| """Validate that api key and python package exists in environment.""" | ||
| endpoint_key = get_from_dict_or_env( | ||
| values, "endpoint_api_key", "ENDPOINT_API_KEY" |
There was a problem hiding this comment.
can we give the env var a more class-specific name, like AZURE_ENDPOINT_API_KEY? if it's generic may collide with other env vars folks have
| endpoint_key = get_from_dict_or_env( | ||
| values, "endpoint_api_key", "ENDPOINT_API_KEY" | ||
| ) | ||
| endpoint_url = values["endpoint_url"] |
There was a problem hiding this comment.
should we just make it possible to pass in endpoint_url and deployment_name as env vars as well?
| class AzureMLOnlineEndpoint(LLM, BaseModel): | ||
| """Wrapper around Azure ML Hosted models using Managed Online Endpoints. | ||
| Example: | ||
| .. code-block:: python |
There was a problem hiding this comment.
need extra new line after .. code-block... or it won't render correctly
3 participants
Description
We have added a new LLM integration
azureml_endpointthat allows users to leverage models from the AzureML platform. Microsoft recently announced the release of Azure Foundation Models which users can find in the AzureML Model Catalog. The Model Catalog contains a variety of open source and Hugging Face models that users can deploy on AzureML. Theazureml_endpointallows LangChain users to use the deployed Azure Foundation Models.Dependencies
No added dependencies were required for the change.
Tests
Integration tests were added in
tests/integration_tests/llms/test_azureml_endpoint.py.Notebook
A Jupyter notebook demonstrating how to use
azureml_endpointwas added todocs/modules/llms/integrations/azureml_endpoint_example.ipynb.Twitters
Prakhar Gupta
Matthew DeGuzman
Who can review?
Tag maintainers/contributors who might be interested: