Skip to content

feat(matrixify): implement matrix of any charts as core Superset feature - #34526

Merged
mistercrunch merged 42 commits into
masterfrom
matrixify
Aug 19, 2025
Merged

feat(matrixify): implement matrix of any charts as core Superset feature#34526
mistercrunch merged 42 commits into
masterfrom
matrixify

Conversation

@mistercrunch

@mistercrunch mistercrunch commented Aug 3, 2025

Copy link
Copy Markdown
Member

Summary

Screenshot 2025-08-03 at 3 23 33 AM

This PR introduces Matrixify - a powerful feature that enables any Superset chart to be displayed as a matrix/grid of charts (similar to Tableau's Trellis charts). This allows users to create small multiples visualizations, comparing the same metric across different dimensions or multiple metrics side-by-side.

What it enables

  • Matrix visualization of any chart type: Transform any Superset chart into a grid of charts
  • Flexible matrix configuration:
    • Rows and columns can be configured independently
    • Use metrics or dimension values for either axis
  • Universal availability: Works in both Explore view and Dashboards
  • Recursive composition: Since Matrixify is implemented at the SuperChart level, it theoretically supports matrices of matrices
  • Smart layout: Automatic grid sizing with configurable row heights and column arrangements
  • TopN dimension selection: Select top N dimension values with custom sorting metrics and order
  • Column wrapping: Configure charts per row for better layout control

Gallery

Screenshot 2025-08-03 at 12 03 50 PM

Example use cases

  • A simple array of say 6 "Big Number with Trendline" showing 6 different metrics at the top of your dashboard (instead of creating/saving 6 charts)
  • Compare sales metrics across different regions (rows) and time periods (columns)
  • Show the same visualization for top 10 products by revenue
  • Deck .gl, plot detailed maps of different markets side by side (autozoom/autocenter)
  • stuff your face with as many pie charts as you could dream of
  • possibilities are endless

Latest Updates (Aug 14, 2025)

Features Added

  • TopN dimension selection: Can now select top N dimension values (e.g., "top 12 countries by population") with configurable sorting metrics and order
  • Flexible grid layout: Charts per row setting for dimension overflow (e.g., top 50 countries with 12 per row = 4 full rows + 2 charts on the last row)
  • Improved controls: Better UI for matrixify controls with proper label and column headers display

Fixes & Improvements

  • Fixed all failing tests: Updated MatrixifyDimensionControl tests to work with new data fetching approach
  • Disabled drill-by when matrixify is enabled: Prevents confusing behavior since drill-by doesn't work correctly with matrixified charts
  • Fixed React hooks violations: Resolved hooks ordering issues in MatrixifyGridRenderer
  • Fixed theme and TypeScript issues: Proper theme color usage and type safety
  • Improved test coverage: All 25 tests now passing with proper mocking

Implementation approach

Instead of implementing Matrixify as a separate visualization or explore-only feature, we integrated it directly into SuperChart as a core capability. This architectural decision provides several benefits:

  1. Universal availability: Any place that uses SuperChart (Explore, Dashboard, embedded views) automatically supports Matrixify
  2. Clean architecture: Matrixify is just another way SuperChart can render charts
  3. Recursive composition: SuperChart can render grids of SuperCharts, enabling powerful nested visualizations
  4. Minimal code changes: By working at the SuperChart level, we avoid duplicating logic across different parts of the application

Key components

  1. Type definitions (@superset-ui/core):

    • Moved all Matrixify types to the core package for better reusability
    • Defines the configuration structure for matrix axes
  2. SuperChart integration:

    • Added Matrixify detection in SuperChart.renderChart()
    • When enabled, renders MatrixifyGridRenderer instead of the regular chart
    • Passes rawFormData (snake_case) to maintain compatibility with existing APIs
  3. Grid generation:

    • MatrixifyGridGenerator creates the grid structure and generates formData for each cell
    • Preserves all necessary fields while removing Matrixify-specific configuration
    • Handles both metric-based and dimension-based axes
  4. StatefulChart component:

    • New component that handles data fetching for individual grid cells
    • Supports dynamic sizing and loading states
    • Reuses existing chart data fetching logic
  5. Control panel integration:

    • Added Matrixify sections to chart control panels
    • Custom MatrixifyDimensionControl for dimension selection with TopN support
    • Integrated into existing control panel system

Technical notes

  • FormData handling: Discovered and resolved snake_case vs camelCase conversion issues between different parts of the system
  • CSS/Layout: Implemented proper flexbox and grid layouts to handle overflow and scrolling
  • Performance: Grid cells are memoized to prevent unnecessary re-renders
  • Backwards compatibility: All changes are additive; existing charts continue to work unchanged
  • Drill-by disabled: When matrixify is enabled, drill-by feature is disabled to prevent confusing behavior

Future improvements

  • Add tests for SuperChart Matrixify detection and rendering
  • Add tests for MatrixifyGridGenerator
  • Document StatefulChart component for plugin developers
  • Add user-facing documentation for the Matrixify feature
  • Fix remaining overflow issues in certain view contexts
  • Add support for synchronized axes across grid cells
  • Add export functionality for the entire grid
  • Performance optimizations for large grids
  • Add grid cell highlighting/selection features
  • Implement aggregate chart states for better loading UX
  • Add matrix-level actions (bulk export, etc.)

Testing

  • Matrixify works in Explore view
  • Matrixify works in Dashboard view
  • Both metric and dimension modes work correctly
  • TopN dimension selection works
  • Grid layout adjusts properly with different configurations
  • Column wrapping for dimension overflow works
  • All chart types work correctly when matrixified
  • Drill-by is properly disabled when matrixify is enabled
  • Recursive matrices (matrix of matrices) - theoretical support exists

Next Steps

Core Features

  • QA validation - Verify that existing features work correctly within matrix cells
  • Aggregate chart states - Show overall progress and handle failures at matrix level

Polish & UX

Axis Synchronization

  • Lock X/Y axis controls - Allow forcing consistent axis scales across all charts in the matrix
    • Calculate global min/max values across all chart data
    • Pass unified bounds to underlying charts for consistent comparison
    • Enable per-axis locking (X-axis only, Y-axis only, or both)

Smart Layout Optimization

  • Intelligent axis display - Show axes only where needed to reduce visual clutter
    • Y-axis labels only on leftmost charts
    • X-axis labels only on bottom row charts
    • Note: May not be worth the implementation complexity

Enhanced Functionality

  • Matrix-level actions - Integrate chart actions (CSV export, etc.) at the matrix level
    • Refactor existing dashboard chart actions into SuperChart
    • Enable bulk operations across all matrix cells
    • Clean up and consolidate action handling code

@korbit-ai

korbit-ai Bot commented Aug 3, 2025

Copy link
Copy Markdown

Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment /korbit-review.

Your admin can change your review schedule in the Korbit Console

color: theme.colorText,
borderColor: theme.colorBorder,
},
textStyle: {

@mistercrunch mistercrunch Aug 3, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bycatch ... fixing Bubble Chart's legend stylin'

const resourceTypeReports = reportsState[resourceType] || {};
const reportData = resourceTypeReports[resourceId];

// Debug logging to understand what's happening

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bycatch, was polluting the logs

@villebro

villebro commented Aug 4, 2025

Copy link
Copy Markdown
Member

I love this, great if we can make this work on all charts! I'll take a look at the code tomorrow if it's still open for review then!

@mistercrunch

mistercrunch commented Aug 4, 2025

Copy link
Copy Markdown
Member Author

The feature is super neat, very dynamic, charts show up instantly as you pick metrics and/or dimension members (most new controls are renderTrigger: true).

At first I though it was going to be super messy architecturally (that there would be lots of special cases between Explore, Dashboard and embedded), but by building the feature in <SuperChart /> (clean foundation for all charts in Superset), and creating a new, simple <StatelessChart /> wrapper around it (lands in @superset-ui, receives either formData or simply a chartId), the code is pretty squeaky clean, and Matrixify becomes a core, foundational feature deeply baked into Superset.

@rusackas
rusackas requested review from justinpark and rusackas August 4, 2025 18:10
@mistercrunch
mistercrunch marked this pull request as ready for review August 5, 2025 14:54
@korbit-ai

korbit-ai Bot commented Aug 5, 2025

Copy link
Copy Markdown

Korbit doesn't automatically review large (3000+ lines changed) pull requests such as this one. If you want me to review anyway, use /korbit-review.

@github-actions

github-actions Bot commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

@mistercrunch Processing your ephemeral environment request here. Action: up. More information on how to use or configure ephemeral environments

@dosubot dosubot Bot added change:frontend Requires changing the frontend viz:charts Namespace | Anything related to viz types labels Aug 5, 2025
@github-actions

github-actions Bot commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

@mistercrunch Ephemeral environment spinning up at http://18.236.155.237:8080. Credentials are 'admin'/'admin'. Please allow several minutes for bootstrapping and startup.

@rusackas
rusackas requested review from kasiazjc and villebro August 7, 2025 17:58
mistercrunch and others added 9 commits August 18, 2025 22:29
…ata fetches

- Add shouldRefetchData helper to check if formData changes require new data
- Import ChartControlPanelRegistry to access control metadata
- Only refetch data when non-renderTrigger controls change
- Re-render without fetching for renderTrigger-only changes
- Improves performance by avoiding unnecessary API calls for UI-only updates

Falls back to refetching if control panel config unavailable for safety.
## Summary

Added comprehensive unit test coverage for the Matrixify feature including:

1. **StatefulChart renderTrigger optimization** - Tests for `shouldRefetchData` function that prevents unnecessary data fetches when only UI-only controls change
2. **MatrixifyGridRenderer column wrapping** - Tests for the complex column grouping and wrapping logic in the grid layout
3. **shouldMapStateToProps logic** - Tests for the dynamic state mapping in Matrixify dimension controls
4. **ChartRenderer matrixify change detection** - Tests for the component update logic when matrixify properties change

## Test Coverage Added

### StatefulChart.test.tsx
- Tests renderTrigger-only changes (should NOT refetch data)
- Tests non-renderTrigger changes (should refetch data)
- Tests mixed control changes (should refetch if any non-renderTrigger)
- Tests viz_type changes (always refetch)
- Tests fallback behavior when no control panel config available
- Tests error handling for registry access failures
- Tests force prop and chartId changes

### MatrixifyGridRenderer.test.tsx
- Tests column grouping logic for dynamic vs fixed column layouts
- Tests header visibility with wrapping vs non-wrapping scenarios
- Tests grid cell placement in wrapped layouts
- Tests edge cases (null grid, empty grid, missing config)
- Tests exact division and overflow scenarios for column wrapping

### shouldMapStateToProps.test.tsx
- Tests shouldMapStateToProps change detection for both X and Y dimensions
- Tests mapStateToProps value mapping from form_data and controls
- Tests performance optimization (only checks relevant fields)
- Tests fallback behavior and default values
- Tests priority of form_data over control values

### ChartRenderer.test.jsx
- Tests matrixify property change detection
- Tests component update logic for matrixify-enabled charts
- Tests nested property changes in matrixify configurations
- Tests addition and removal of matrixify properties

## Technical Improvements

- **Intelligent data fetching**: StatefulChart now avoids unnecessary API calls when only UI controls change
- **Robust column wrapping**: MatrixifyGridRenderer handles complex grid layouts with proper header management
- **Dynamic state mapping**: Matrixify controls efficiently detect when recalculation is needed
- **Performance optimization**: Change detection focuses only on relevant properties

All tests use proper mocking strategies and follow React Testing Library best practices.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add tests for StatefulChart renderTrigger optimization
- Add tests for MatrixifyGridRenderer column wrapping logic
- Add tests for shouldMapStateToProps in dimension controls
- Add tests for ChartRenderer matrixify change detection
- Refactor ChartRenderer tests to avoid require() statements

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Flatten nested describe blocks to individual test functions
- Improve test organization and readability
- Maintain all test functionality while using consistent test() pattern
- All 77 tests continue to pass

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add guideline to CLAUDE.md about using test() instead of describe():
- References Kent C. Dodds' 'avoid nesting when testing' principles
- Documents the hard rule of no describe() blocks in new test code
- Provides clear guidance for future test file development

This captures the requirement and ongoing migration to flat test structure
that we've implemented across all new Matrixify test files.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Improve test file organization by co-locating tests with source files:

Moved files:
- StatefulChart.test.tsx → src/chart/components/ (alongside StatefulChart.tsx)
- MatrixifyGridRenderer.test.tsx → src/chart/components/Matrixify/ (alongside MatrixifyGridRenderer.tsx)
- matrixify.test.ts → src/chart/types/ (alongside matrixify.ts)
- __mocks__/matrixify.ts → src/chart/types/matrixify.mocks.ts (renamed and co-located)

Updated import paths in moved test files to reflect new locations.
Removed empty __mocks__ directory.

Benefits:
- Better test discoverability - tests next to source files
- Easier maintenance - related files grouped together
- Cleaner test directory structure
- Follows common testing patterns

All tests continue to pass with updated paths.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@mistercrunch

mistercrunch commented Aug 19, 2025

Copy link
Copy Markdown
Member Author

@michael-s-molina all comments addressed, checked "download as image" in master and got the same results as on my branch (in my case looked the same, and was imperfect (big number shown but now the canvas/viz)).

top-10-countries-2025-08-19T05-31-36 218Z
top-10-countries-2025-08-19T05-29-18 524Z

So very likely unrelated.

@michael-s-molina michael-s-molina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the great feature and for addressing my comments @mistercrunch!

@mistercrunch mistercrunch removed the hold:testing! On hold for testing label Aug 19, 2025
@mistercrunch
mistercrunch merged commit e6c8343 into master Aug 19, 2025
56 checks passed
@mistercrunch
mistercrunch deleted the matrixify branch August 19, 2025 15:36
@ghost

ghost commented Sep 2, 2025

Copy link
Copy Markdown

Do we know what version this is slated to be included in?

@mistercrunch

Copy link
Copy Markdown
Member Author

I don't think it made 6.0, so should be 7.0. Hoping we can back-to-back these two!

@mistercrunch mistercrunch added the v6.0 Label added by the release manager to track PRs to be included in the 6.0 branch label Sep 23, 2025
@sadpandajoe sadpandajoe removed the v6.0 Label added by the release manager to track PRs to be included in the 6.0 branch label Sep 29, 2025
sadpandajoe pushed a commit that referenced this pull request Dec 4, 2025
…ure (#34526)

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit e6c8343)
sadpandajoe pushed a commit that referenced this pull request Dec 4, 2025
…ure (#34526)

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit e6c8343)
sadpandajoe pushed a commit that referenced this pull request Dec 16, 2025
…ure (#34526)

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit e6c8343)
aminghadersohi pushed a commit to aminghadersohi/superset that referenced this pull request Jan 17, 2026
…ure (apache#34526)

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit e6c8343)
aminghadersohi pushed a commit to aminghadersohi/superset that referenced this pull request Jan 24, 2026
…ure (apache#34526)

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit e6c8343)
qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026
…ure (apache#34526)

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:frontend Requires changing the frontend dependencies:npm packages plugins preset-io size/XXL viz:charts Namespace | Anything related to viz types

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants