[Enhancement]: Complete RESTful v2 coverage and semantic parity for public PyMilvus 3.0 APIs

[zhuwenxing]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Create a tracking task to close the confirmed coverage and semantic gaps between RESTful v2 and the public PyMilvus 3.0 MilvusClient API.

This is not a request to expose every Milvus gRPC method. Internal, diagnostic, and component-management RPCs that are not available through the public high-level PyMilvus client are explicitly out of scope.

Audit baseline

• Milvus upstream/3.0: 07df976ff67561906ffb8bcbbc4964fbb861d7ca
• PyMilvus upstream/3.0: cb16a04cb0f0f7c07260c56c4bfd5e8442692ed7
• Method scope: public high-level pymilvus.MilvusClient
• Verification method: read-only source audit
• Runtime and E2E confirmation: still required as implementation acceptance work

The audit found 17 usable or intended public high-level PyMilvus capabilities without RESTful v2 equivalents.

PyMilvus also declares get_flush_all_state(), but its current high-level implementation does not pass the required flush_all_ts to the underlying handler. It is therefore not counted as an independent working capability in this audit. RESTful v2 should nevertheless provide usable completion semantics together with flush_all().

Current RESTful v2 coverage: missing capabilities

Capability	Public PyMilvus API	Backend gRPC API	Current RESTful v2 state
Server version	get_server_version()	GetVersion	No equivalent RESTful v2 endpoint
Detailed server/build information	get_server_version(detail=True)	Connect	No equivalent RESTful v2 endpoint
Replica inspection	describe_replica()	GetReplicas	No equivalent RESTful v2 endpoint
Global flush	flush_all()	FlushAll, GetFlushAllState	No global flush endpoint or usable completion workflow
Loaded segment inspection	list_loaded_segments()	GetQuerySegmentInfo	No equivalent RESTful v2 endpoint
Persistent segment inspection	list_persistent_segments()	GetPersistentSegmentInfo	/segments/describe invokes GetSegmentsInfo; its request and response semantics are different
Compaction plans	get_compaction_plans()	GetCompactionStateWithPlans	Compaction operations exist, but RESTful v2 cannot return compaction plan details
Replication configuration update	update_replicate_configuration()	UpdateReplicateConfiguration	No equivalent RESTful v2 endpoint
Replication configuration inspection	get_replicate_configuration()	GetReplicateConfiguration	No equivalent RESTful v2 endpoint
Replication state inspection	get_replicate_info()	GetReplicateInfo	No equivalent RESTful v2 endpoint
Replication message dump	dump_messages()	DumpMessages	No equivalent RESTful v2 endpoint; the server-streaming transport requires an explicit REST design
Native snapshot lifecycle	create_snapshot(), drop_snapshot(), list_snapshots(), describe_snapshot(), restore_snapshot(), pin_snapshot_data(), unpin_snapshot_data()	Native snapshot RPCs	RESTful v2 exposes restore-state/job inspection and external export/restore workflows, but cannot perform the complete native snapshot lifecycle

Current RESTful v2 coverage: request and behavior gaps

Area	Current RESTful v2 behavior	Public PyMilvus 3.0 behavior
Collection and partition load	Primarily forwards collection or partition names	Supports replica count, resource groups, load fields, skipping the dynamic field, priority, and load parameters
Refresh load	Only forwards collection identification	PyMilvus load behavior supports additional load configuration
Compaction	Only exposes clustering-compaction selection	Also supports is_l0 and target_size
Delete by primary key	REST request binding requires filter; the handler contains an ID fallback, but that path is unreachable through the public request schema	MilvusClient.delete() supports ids as well as filters
Search highlighting	No public highlighter input or hit-highlight output	Search accepts highlighter and exposes highlighting results
Namespace schema configuration	Collection schema creation cannot set enable_namespace	PyMilvus schemas can enable namespaces
Namespace data operations	Insert, query, search, and hybrid search cannot pass a namespace	The corresponding PyMilvus operations accept namespace selection
Hybrid-search metric	The request declares top-level metricType, but the handler does not forward it	The metric affects search behavior
Role removal	No force_drop input	drop_role() exposes force_drop
Field description	Full collection creation does not preserve field descriptions	PyMilvus schema fields expose descriptions
Field type parameters	typeParams is declared in the REST request schema but is not forwarded when building the field schema	PyMilvus forwards field type parameters
Array-of-vector fields	Full collection creation does not forward ArrayOfVector.elementDataType	PyMilvus preserves the element data type
Collection properties	Only selected hard-coded properties are forwarded	PyMilvus forwards arbitrary supported collection properties
Replica transfer routing	The call site invokes TransferReplica, but route metadata and fullMethod use the nonexistent TransferMaster name	The public operation maps to TransferReplica
Replica transfer database	dbName is parsed by the REST wrapper but is not written to TransferReplicaRequest	The request can be scoped to a database

Current RESTful v2 coverage: response and error gaps

Area	Current RESTful v2 response	Public PyMilvus 3.0 result
Collection description	Omits namespace enablement, schema version, creation timestamp, and update timestamp	Exposes these public collection metadata fields
Query result metadata	Omits the returned session timestamp	Preserves session_ts in result metadata
Search result metadata	Omits the returned session timestamp	Exposes SearchResult.get_session_ts()
Error classification	Returns only code and message	MilvusException also exposes retriable and is_input_error
Privilege groups	Joins privileges into a comma-separated string	Preserves privileges as a structured collection
Alias listing	Returns only alias names	The PyMilvus handler can also preserve collection and database context returned by the server

Confirmed non-gaps and scope exclusions

The following should not be treated as missing RESTful v2 capabilities in this tracking task:

• Names-only results from list_collections(), list_partitions(), and list_databases() are aligned with the public PyMilvus API.
• Mutation timestamps are not exposed by the high-level MilvusClient result and therefore are not a REST parity requirement.
• Full protobuf Status details are not automatically part of the public parity target. The relevant public PyMilvus error properties are retriable and is_input_error.
• Hybrid function chains and search aggregation are rejected by PyMilvus 3.0 as well.
• failedPlanNo is not exposed by the high-level PyMilvus compaction result.
• Internal or diagnostic RPCs such as AllocTimestamp, RegisterLink, Dummy, GetMetrics, GetComponentStates, CalcDistance, BackupRBAC, RestoreRBAC, and ComputePhraseMatchSlop are not public high-level MilvusClient capabilities.
• Internal replication-stream and feder/index-management RPCs are not included merely because they exist in the gRPC service.
• LoadBalance and TransferNode only exist through deprecated ORM utility APIs and should be evaluated separately as legacy compatibility requirements.

Requested implementation work

Add missing public capabilities

• Add server version and detailed build information.
• Add replica inspection.
• Add global flush and usable completion-state semantics.
• Add loaded and persistent segment inspection.
• Add compaction plan inspection.
• Add replication configuration, replication state, and message-dump capabilities.
• Add the complete native snapshot lifecycle.

Align existing request behavior

• Preserve public load and refresh-load options.
• Support is_l0 and target_size for compaction.
• Support deletion by primary-key IDs.
• Support search highlighter input and output.
• Support namespace schema configuration and namespace-scoped data operations.
• Forward hybrid-search metricType.
• Support forced role removal.
• Preserve all supported field-schema and collection properties.
• Correct replica-transfer routing metadata and database propagation.

Align responses and errors

• Return the public collection metadata exposed by PyMilvus.
• Preserve query and search session timestamps.
• Preserve public error retry and input-error classification.
• Return privilege-group privileges as structured data.
• Preserve alias collection and database context when returned by Milvus.

Why is this needed?

RESTful v2 and PyMilvus are both public Milvus interfaces, but users currently cannot perform several public PyMilvus operations through REST.

For operations that exist in both interfaces, silently dropped request fields can change server behavior. Missing response metadata and error classification can also prevent REST clients from making the same consistency, retry, and diagnostic decisions as SDK clients.

A parity boundary based on the public PyMilvus 3.0 client provides a concrete and testable target while preventing internal-only gRPC methods from being exposed accidentally.

Anything else?

Acceptance criteria

• Every requested capability is implemented or explicitly excluded through an approved and documented API or transport decision.
• Request fields preserve the behavior of the corresponding public PyMilvus operation while following existing REST naming conventions.
• Responses preserve user-visible data types, metadata, and error classification.
• REST handler tests and public API E2E parity tests are added for each implemented capability.
• Route metadata and fullMethod values match the gRPC methods actually invoked.
• Streaming-specific behavior such as dump_messages() has an approved REST transport, cancellation, and lifecycle design before implementation.
• REST API documentation is updated together with each new or changed public contract.

Related historical issues include #31180, #31181, and #31387. They cover narrower behavior or previous API decisions and do not replace this 3.0-wide tracking task. In particular, previously rejected behavior should be explicitly re-evaluated rather than silently changed.

[superezzdev]

Request to Assign

Hi maintainers,

I'd like to work on this issue.

I plan to start with the Server Version parity work, which includes:

• Add a RESTful v2 endpoint equivalent to MilvusClient.get_server_version().

• Support both:

  • Basic server version (GetVersion)
  • Detailed server/build information (Connect) similar to get_server_version(detail=True)

• Follow existing RESTful v2 conventions for routing, request/response formatting, and error handling.

• Add handler tests and REST API tests.

• Update the REST API documentation accordingly.

I'll keep the implementation focused on this capability only so it can be reviewed independently before moving on to the remaining parity items.

Please assign this issue to me if that approach sounds good.

Thank you!