Skip to content

Conversation

dmchoiboi
Copy link
Collaborator

Reverts #518

This was not correctly done. Misses various other recursive imports and worst of all, the import of openai's model makes this approach not feasible.

/usr/local/lib/python3.10/dist-packages/pydantic/_internal/_config.py:334: UserWarning: Valid config keys have changed in V2:                                                          │
* 'orm_mode' has been renamed to 'from_attributes'                                                                                                                                     │
  warnings.warn(message, UserWarning)                                                                                                                                                  │
Traceback (most recent call last):                                                                                                                                                     │
  File "/workspace/vllm_batch.py", line 16, in <module>                                                                                                                                │
    from model_engine_server.common.dtos.llms import (                                                                                                                                 │
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load                                                                                                                   │
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked                                                                                                          │
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked                                                                                                                    │
  File "/usr/local/lib/python3.10/dist-packages/ddtrace/internal/module.py", line 225, in _exec_module                                                                                 │
    self.loader.exec_module(module)                                                                                                                                                    │
  File "/workspace/model-engine/model_engine_server/common/dtos/llms.py", line 8, in <module>                                                                                          │
    from model_engine_server.common.dtos.model_endpoints import (                                                                                                                      │
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load                                                                                                                   │
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked                                                                                                          │
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked                                                                                                                    │
  File "/usr/local/lib/python3.10/dist-packages/ddtrace/internal/module.py", line 225, in _exec_module                                                                                 │
    self.loader.exec_module(module)                                                                                                                                                    │
  File "/workspace/model-engine/model_engine_server/common/dtos/model_endpoints.py", line 150, in <module>                                                                             │
    class GetModelEndpointsSchemaV1Response(BaseModel):                                                                                                                                │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py", line 197, in __new__                                                                                             │
    fields[ann_name] = ModelField.infer(                                                                                                                                               │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/fields.py", line 504, in infer                                                                                             │
    return cls(                                                                                                                                                                        │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/fields.py", line 434, in __init__                                                                                          │
    self.prepare()                                                                                                                                                                     │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/fields.py", line 555, in prepare                                                                                           │
    self.populate_validators()                                                                                                                                                         │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/fields.py", line 829, in populate_validators                                                                               │
    *(get_validators() if get_validators else list(find_validators(self.type_, self.model_config))),                                                                                   │
  File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/validators.py", line 765, in find_validators                                                                               │
    raise RuntimeError(f'no validator found for {type_}, see `arbitrary_types_allowed` in Config')                                                                                     │
RuntimeError: no validator found for <class 'fastapi.openapi.models.OpenAPI'>, see `arbitrary_types_allowed` in Config

We'll need to a do a proper upgrade to pydantic2 to unblock the vllm upgrade.

@dmchoiboi dmchoiboi requested a review from yunfeng-scale May 16, 2024 02:28
@dmchoiboi dmchoiboi merged commit a36f7a2 into main May 16, 2024
@dmchoiboi dmchoiboi deleted the revert-518-michaelchoi/mli-2295-upgrade-vllm-version-for-batch-completion branch May 16, 2024 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants