Commit c7a6fec
authored
Add timeout enforcement and comprehensive tests for gojq middleware (#961)
Implements Priority 2 recommendations from the Go Fan module review for
the gojq middleware, adding timeout protection and comprehensive test
coverage.
## Changes
**Timeout Protection**
- Added `DefaultJqTimeout` constant (5 seconds) to prevent query hangs
- Enhanced `applyJqSchema()` to automatically enforce timeout when
context lacks a deadline
- Preserves existing context deadlines when present
- Improved error handling for timeout, cancellation, and compilation
failures
**Documentation**
- Updated code comments to reference gojq v0.12.18 features (536M
element array limit, improved concurrency, enhanced type error messages)
- Enhanced `init()` function documentation explaining fail-fast behavior
- Added detailed timeout behavior documentation in `applyJqSchema()`
**Test Coverage**
- Added 9 comprehensive timeout-related tests covering:
- Default timeout application for contexts without deadlines
- Context deadline preservation
- Large array processing (10,000 elements)
- Deeply nested structures (10 levels)
- Compilation error scenarios
- Context cancellation behavior
**Code Quality**
- Fixed 4 unused `require` declarations in `internal/sys/sys_test.go`
## Example
```go
// Context without deadline automatically gets 5-second timeout
result, err := applyJqSchema(context.Background(), jsonData)
// Existing deadlines are preserved
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result, err := applyJqSchema(ctx, jsonData)
```
All middleware tests pass (30+ test cases). The implementation leverages
pre-existing query compilation caching (10-100x speedup) while adding
robust timeout protection.
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `example.com`
> - Triggering command: `/tmp/go-build1592018226/b275/launcher.test
/tmp/go-build1592018226/b275/launcher.test
-test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a
64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get
go-sdk/internal/-v ache/Python/3.12-buildid
ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis��
64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias` (dns
block)
> - Triggering command: `/tmp/go-build3921665002/b275/launcher.test
/tmp/go-build3921665002/b275/launcher.test
-test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
/opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test
--gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose`
(dns block)
> - Triggering command: `/tmp/go-build1222286590/b275/launcher.test
/tmp/go-build1222286590/b275/launcher.test
-test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests
ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d
aw-mcpg/internal/testutil/mcptest/config.go
aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go
x_amd64/vet bash` (dns block)
> - `invalid-host-that-does-not-exist-12345.com`
> - Triggering command: `/tmp/go-build1592018226/b260/config.test
/tmp/go-build1592018226/b260/config.test
-test.testlogfile=/tmp/go-build1592018226/b260/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true sonrpc2/conn.go
sonrpc2/frame.go .12/x64/bin/as ;
s#git --global bin/git 64/pkg/include ortc�� 64/src/runtime/cgo
64/src/crypto/internal/fips140de--gdwarf2 inux.go esnew.go ocknew.go
nix_cgo.go nix_cgo_res.go` (dns block)
> - Triggering command: `/tmp/go-build1856060892/b260/config.test
/tmp/go-build1856060892/b260/config.test
-test.testlogfile=/tmp/go-build1856060892/b260/testlog.txt
-test.paniconexit0 -test.timeout=10m0s rev-�� --abbrev-ref HEAD
cal/bin/git tion_pool.go er.go 64/pkg/tool/linux_amd64/vet base64 -d`
(dns block)
> - `nonexistent.local`
> - Triggering command: `/tmp/go-build1592018226/b275/launcher.test
/tmp/go-build1592018226/b275/launcher.test
-test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a
64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get
go-sdk/internal/-v ache/Python/3.12-buildid
ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis��
64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias` (dns
block)
> - Triggering command: `/tmp/go-build3921665002/b275/launcher.test
/tmp/go-build3921665002/b275/launcher.test
-test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
/opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test
--gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose`
(dns block)
> - Triggering command: `/tmp/go-build1222286590/b275/launcher.test
/tmp/go-build1222286590/b275/launcher.test
-test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests
ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d
aw-mcpg/internal/testutil/mcptest/config.go
aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go
x_amd64/vet bash` (dns block)
> - `slow.example.com`
> - Triggering command: `/tmp/go-build1592018226/b275/launcher.test
/tmp/go-build1592018226/b275/launcher.test
-test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a
64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get
go-sdk/internal/-v ache/Python/3.12-buildid
ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis��
64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias` (dns
block)
> - Triggering command: `/tmp/go-build3921665002/b275/launcher.test
/tmp/go-build3921665002/b275/launcher.test
-test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
/opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test
--gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose`
(dns block)
> - Triggering command: `/tmp/go-build1222286590/b275/launcher.test
/tmp/go-build1222286590/b275/launcher.test
-test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests
ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d
aw-mcpg/internal/testutil/mcptest/config.go
aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go
x_amd64/vet bash` (dns block)
> - `this-host-does-not-exist-12345.com`
> - Triggering command: `/tmp/go-build1592018226/b284/mcp.test
/tmp/go-build1592018226/b284/mcp.test
-test.testlogfile=/tmp/go-build1592018226/b284/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/runtime/cgo
Vl8znsXtV x_amd64/cgo rt-size '1280,
7/opt/hostedtoolcache/go/1.25.7/x64/pkg/tool/linux_amd64/compile
fips140/check cal/bin/npx x_amd64/cgo ortc�� 0938582/b193/_pk-p
mon/httpcommon.ggithub.com/github/gh-aw-mcpg/internal/logger/sanitize
x_amd64/vet unset --global ache/Python/3.12--version x_amd64/vet` (dns
block)
> - Triggering command: `/tmp/go-build1856060892/b284/mcp.test
/tmp/go-build1856060892/b284/mcp.test
-test.testlogfile=/tmp/go-build1856060892/b284/testlog.txt
-test.paniconexit0 -test.timeout=10m0s rev-�� --abbrev-ref HEAD
ache/go/1.25.7/x64/pkg/tool/linux_amd64/compile
/opt/hostedtoolc/opt/hostedtoolcache/go/1.25.7/x64/pkg/tool/linux_amd64/link
0938582/b194/ ache/go/1.25.7/x/tmp/go-build1856060892/b260/config.test
ache/go/1.25.7/x-importcfg de 1665002/b293/_pk-s HEAD 1665002/b293=>
/guard/context.ggrep /guard/guard.go 64/pkg/tool/linu(create|run) git`
(dns block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/github/gh-aw-mcpg/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
----
*This section details on the original issue you should resolve*
<issue_title>[go-fan] Go Module Review: gojq - JSON Query
Processing</issue_title>
<issue_description># 🐹 Go Fan Report: github.com/itchyny/gojq
## Module Overview
**gojq** is a pure Go implementation of jq - the powerful JSON query
language and processor. It provides both a CLI tool and a Go library for
programmatically processing JSON data with jq queries. This module is
critical for the MCP Gateway's middleware layer, enabling sophisticated
JSON transformations and schema generation on MCP tool responses.
- **Version**: v0.12.18 (latest)
- **Repository**: https://github.com/itchyny/gojq
- **Stars**: 3,692 ⭐
- **License**: MIT
- **Last Update**: Jan 31, 2026 (13 days ago - very active!)
## Current Usage in gh-aw-mcpg
Based on GitHub code search, gojq is used in **2 files**:
### Files
- `internal/middleware/jqschema.go` - Main implementation for jq-based
schema generation
- `internal/middleware/jqschema_bench_test.go` - Performance benchmarks
### Key APIs Used
The middleware likely leverages:
- `gojq.Parse()` - Parse jq query strings
- `gojq.Compile()` - Compile queries into executable code
- `Code.Run()` - Execute compiled queries on JSON data
- Iterator pattern for efficient result processing
### Context
The middleware uses gojq for:
- **JSON Schema Generation**: Transform MCP tool response payloads into
JSON schemas
- **Payload Processing**: Handle large JSON responses from backend MCP
servers
- **Performance**: Active benchmarking indicates optimization focus
## Research Findings
### Recent Updates (v0.12.18 - December 2025)
🎉 **Major improvements in latest release:**
1. **New Functions**
- ✨ `trimstr/1` - Efficient prefix/suffix removal (better than string
slicing)
- ✨ `toboolean/0` - Clean type conversion
2. **Performance & Scale**
- 🚀 **Array index limit increased to 536,870,912 (2^29 elements)** -
huge improvement!
- 🚀 Stopped numeric normalization for concurrent execution - better
parallel performance
- ✨ Support for binding expressions with binary operators (`1 + 2 as $x
| -$x`)
3. **Bug Fixes**
- 🐛 Fixed `last/1` to be included in `builtins/0`
- 🐛 Fixed `--indent 0` to preserve newlines
- 🐛 Fixed string repetition to emit error when result is too large
### Very Recent Activity (January 2026)
- **Jan 31, 2026**: Fixed type error messages for split() and match()
functions
- **Jan 7, 2026**: Updated copyright year and GitHub Actions
- **Ongoing**: Active maintenance with regular updates
### Best Practices from gojq Documentation
1. **Compile Once, Run Many**: Compile queries once and reuse for
massive performance gains
2. **Iterator Pattern**: Use `Run()` which returns an iterator for
memory-efficient processing
3. **Error Handling**: Check both compilation errors (syntax) and
runtime errors (types, null access)
4. **Custom Functions**: Extend jq with Go functions using
`gojq.WithFunction()`
5. **Variables**: Pass variables to queries for dynamic behavior
6. **Memory Management**: Be mindful of large arrays (now supports up to
536M elements!)
## Improvement Opportunities
### 🏃 Quick Wins (High Impact, Low Effort)
#### 1. Leverage New v0.12.18 Functions
**Impact**: Medium | **Effort**: Low
- Use `trimstr/1` instead of manual string slicing for prefix/suffix
removal
- Use `toboolean/0` instead of custom type conversion logic
- **Benefit**: Simpler, more readable jq queries with better performance
**Example**:
``````jq
# Before
.[1:] | if . == "true" then true else false end
# After (with v0.12.18)
trimstr("x") | toboolean
``````
#### 2. Utilize Increased Array Index Limit
**Impact**: High | **Effort**: Low
v0.12.18 dramatically increased the array index limit to **536,870,912
elements (2^29)**:
- Review any artificial limits or pagination in payload processing
- Large MCP tool responses can now be handled directly without chunking
- **Benefit**: Simpler code, better performance for large datasets
#### 3. Improve Error Messages
**Impact**: Medium | **Effort**: Low
Recent fixes improved type error messages for `split()` and `match()`:
- Ensure error handling captures and logs these enhanced messages
- Add context about which MCP server/tool caused the error
- **Benefit**: Faster debugging and troubleshooting
### ✨ Feature Opportunities (High Impact, Medium/High Effort)
#### 1. Query Compilation Caching 🔥
**Impact**: High | **Effort**: Medium
**Problem**: If jq queries are recompiled on every request, it wastes
significant CPU.
**Solution**: Implement a compilation cache using `sync.Map`:
``````go
var compiledQueries sync.Map // Thread-safe cache
func getOrCompileQuery(queryStr string) (*gojq.Code, error) {
// Check cache first
if cached, ok := compiledQueries.Load(queryStr); ok {
return cached.(*gojq.Code), nil
}
// Parse and compile
query, err := gojq.Parse(queryStr)
if err != nil {
return nil, fmt.Errorf("failed to parse j...
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
- Fixes #9213 files changed
Lines changed: 858 additions & 618 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
20 | 25 | | |
21 | 26 | | |
22 | 27 | | |
| |||
33 | 38 | | |
34 | 39 | | |
35 | 40 | | |
36 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
37 | 53 | | |
38 | 54 | | |
39 | 55 | | |
| |||
54 | 70 | | |
55 | 71 | | |
56 | 72 | | |
57 | | - | |
| 73 | + | |
58 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
59 | 81 | | |
60 | 82 | | |
61 | 83 | | |
62 | 84 | | |
63 | | - | |
| 85 | + | |
| 86 | + | |
64 | 87 | | |
65 | 88 | | |
66 | 89 | | |
67 | 90 | | |
68 | 91 | | |
69 | | - | |
| 92 | + | |
| 93 | + | |
70 | 94 | | |
71 | 95 | | |
72 | 96 | | |
73 | | - | |
| 97 | + | |
| 98 | + | |
74 | 99 | | |
75 | 100 | | |
76 | 101 | | |
| |||
85 | 110 | | |
86 | 111 | | |
87 | 112 | | |
88 | | - | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
89 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
90 | 127 | | |
91 | 128 | | |
92 | 129 | | |
93 | | - | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
94 | 139 | | |
95 | 140 | | |
96 | 141 | | |
| |||
102 | 147 | | |
103 | 148 | | |
104 | 149 | | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
105 | 155 | | |
106 | 156 | | |
107 | 157 | | |
| |||
111 | 161 | | |
112 | 162 | | |
113 | 163 | | |
114 | | - | |
| 164 | + | |
| 165 | + | |
115 | 166 | | |
116 | 167 | | |
117 | 168 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
592 | 593 | | |
593 | 594 | | |
594 | 595 | | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
| 780 | + | |
| 781 | + | |
| 782 | + | |
| 783 | + | |
| 784 | + | |
| 785 | + | |
| 786 | + | |
| 787 | + | |
595 | 788 | | |
596 | 789 | | |
597 | 790 | | |
| |||
0 commit comments