Support per-request API keys and update staging/autopush config by keyurva · Pull Request #140 · datacommonsorg/agent-toolkit

keyurva · 2026-01-16T21:29:29Z

This became a much deeper PR than what I'd initially thought. This is because the hosted MCP servers in all environments are connected to the prod APIs instead of to the ones in the respective environments. This PR changes that.

Updating those environments then needed updating secrets, other env vars and also the MCP startup validation routines.

As far as the code changes go, they are as follows:

Implements middleware to propagate X-API-Key headers to datacommons_client
Updates CLI to register middleware and support environment-specific key validation via DC_API_KEY_VALIDATION_ROOT
Updates staging.yaml and autopush.yaml with necessary environment variables and secrets
The changes in the scripts folder was just a result of running ruff format. It has no real code changes,

Testing

In addition to unit tests, did the following manual tests:

To test against an apigee fronted instance, deployed manually to staging (would love a dev apigee instance but that does not exist currently) and verified that the middleware was called as expected. Note that I've updated the log level to debug after this verification.
Since no invalid keys can make it via the apigee layer, tested invalid keys locally to verify that the downstream calls fail when that happens.
While staging deployments have validated the staging config, I've not tested the autopush config changes.
- Let me know if you know a good way of testing this else I can monitor it on PR merge.
- Also, there's some funkiness in the autopush config. See comments in autopush.yaml for details.

gemini-code-assist · 2026-01-16T21:29:53Z

Summary of Changes

Hello @keyurva, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical configuration issue where hosted MCP servers in non-production environments were incorrectly communicating with production APIs. It introduces a robust solution by implementing per-request API key support via a new middleware, ensuring that X-API-Key headers are correctly propagated. Furthermore, it updates the deployment configurations for staging and autopush environments to utilize their respective API endpoints and secrets, enhancing environment isolation and security.

Highlights

Per-Request API Key Support: Introduced APIKeyMiddleware to automatically propagate X-API-Key headers to the datacommons_client for individual requests.
Environment-Specific API Key Validation: Updated the CLI to register the new middleware and enable environment-specific API key validation using the DC_API_KEY_VALIDATION_ROOT environment variable.
Deployment Configuration Updates: Configured staging.yaml and autopush.yaml to use environment-specific API endpoints and secrets, resolving the issue of hosted MCP servers incorrectly connecting to production APIs.
Dependency and Testing Enhancements: Upgraded the datacommons-client dependency to version 2.1.5 and added dedicated unit tests for the new API key middleware to ensure its correct functionality.
Code Formatting: Applied automatic code formatting (ruff) to various Python scripts in the scripts folder.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces support for per-request API keys via a Starlette middleware and updates deployment configurations for staging and autopush environments. However, a High-severity Server-Side Request Forgery (SSRF) vulnerability has been identified in the API key validation logic, potentially allowing arbitrary URL requests. There's also a Medium-severity issue in the new API key middleware due to improper exception handling, which could leak sensitive API keys into logs. Additionally, a suggestion has been made to improve the readability of the middleware code.

packages/datacommons-mcp/datacommons_mcp/utils.py

packages/datacommons-mcp/datacommons_mcp/middleware.py

clincoln8

Thanks Keyur!

re:testing on autopush -- I think we can just submit the PR and monitor/test directly on autopush. We can rollback or forward if modifications are required.

clincoln8 · 2026-01-20T17:33:53Z

packages/datacommons-mcp/.env.sample

 # When using a local instance, you may also need to use the 
 # --skip-api-key-validation command-line flag if running without a DC_API_KEY.


Should these two comments be moved to be closer to DC_API_KEY_VALIDATION_ROOT?

clincoln8 · 2026-01-20T17:35:27Z

packages/datacommons-mcp/.env.sample

Should we move away from using the word "local" in these comments?

clincoln8 · 2026-01-20T17:36:48Z

packages/datacommons-mcp/.env.sample

+# =============================================================================
+# HOSTED MCP SERVER OPTIONAL CONFIGURATION
+# =============================================================================


Should we remove this section break and keep DC_API_KEY_VALIDATION_ROOT in the same section as DC_API_ROOT and DC_SEARCH_ROOT? It seems like that might be the same logical grouping.

clincoln8 · 2026-01-20T17:56:01Z

packages/datacommons-mcp/.env.sample

+
+# Root URL for Data Commons API key validation
+# Configure for non-prod environments
+# DC_API_KEY_VALIDATION_ROOT=https://api.datacommons.org


How are we thinking about api key validation for CustomDCs? Since this setting does affect custom instances (since api key is enabled by default and we fallback to validating against api.datacommons.org), we should make sure this is communicated effectively -- ie even for custom instances, the api key is required and validated against base DC?

(there's not really an action item here, just something to think about!)

We mention these as base DC only configs so Custom DC users should not be using them. We could be more explicit about it but leaving it as-is for now.

clincoln8 · 2026-01-20T18:05:53Z

packages/datacommons-mcp/datacommons_mcp/middleware.py

+from collections.abc import Awaitable, Callable
+
+from datacommons_client import use_api_key
+from starlette.middleware.base import BaseHTTPMiddleware


ooc, why use this class instead of fastmcp's https://gofastmcp.com/servers/middleware#modifying-requests?

(no need to change it since it's working, just curious!)

This is what gemini recommended :)

I also liked that it was explicitly registered for http mode only.

keyurva

Thanks for the review!

re:testing on autopush -- I think we can just submit the PR and monitor/test directly on autopush. We can rollback or forward if modifications are required.

SG.

keyurva · 2026-01-20T22:00:57Z

packages/datacommons-mcp/.env.sample

+# =============================================================================
+# HOSTED MCP SERVER OPTIONAL CONFIGURATION
+# =============================================================================


keyurva · 2026-01-20T22:02:49Z

packages/datacommons-mcp/.env.sample

+
+# Root URL for Data Commons API key validation
+# Configure for non-prod environments
+# DC_API_KEY_VALIDATION_ROOT=https://api.datacommons.org


We mention these as base DC only configs so Custom DC users should not be using them. We could be more explicit about it but leaving it as-is for now.

keyurva · 2026-01-20T22:03:54Z

packages/datacommons-mcp/datacommons_mcp/middleware.py

+from collections.abc import Awaitable, Callable
+
+from datacommons_client import use_api_key
+from starlette.middleware.base import BaseHTTPMiddleware


This is what gemini recommended :)

I also liked that it was explicitly registered for http mode only.

keyurva added 11 commits January 15, 2026 14:51

Require datacommons-client >= 2.1.5

cbcb9bf

Add APIKeyMiddleware

3d8bc52

Register APIKeyMiddleware in HTTP mode

21383f2

Add tests for APIKeyMiddleware

0dda490

Log at info level temporarily to verify on staging

d7257a1

Fix ruff checks

2fd95a0

Don't log keys

0501c3a

Add staging env vars to cloud build config

195cd94

Update API key validation to support staging environments

a3f9f77

Add env vars for autopush

bbf82de

Update autopush to use prod search root, log middleware at debug level

d7afdcf

keyurva had a problem deploying to evals-and-secrets January 16, 2026 21:29 — with GitHub Actions Error

gemini-code-assist bot reviewed Jan 16, 2026

View reviewed changes

packages/datacommons-mcp/datacommons_mcp/utils.py Show resolved Hide resolved

packages/datacommons-mcp/datacommons_mcp/middleware.py Show resolved Hide resolved

Fix tests

7863230

keyurva had a problem deploying to evals-and-secrets January 16, 2026 21:52 — with GitHub Actions Failure

keyurva requested a review from clincoln8 January 16, 2026 21:58

clincoln8 approved these changes Jan 20, 2026

View reviewed changes

Address review comments

0c5c1b4

keyurva had a problem deploying to evals-and-secrets January 20, 2026 22:26 — with GitHub Actions Error

keyurva commented Jan 20, 2026

View reviewed changes

keyurva merged commit dc1fab2 into datacommonsorg:main Jan 20, 2026
8 of 9 checks passed

keyurva deleted the per-request-keys branch January 20, 2026 22:28

		# When using a local instance, you may also need to use the
		# --skip-api-key-validation command-line flag if running without a DC_API_KEY.

Conversation

keyurva commented Jan 16, 2026

Testing

Uh oh!

gemini-code-assist bot commented Jan 16, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

clincoln8 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

keyurva left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants