Track Module reporting errors within Email Reporting

[benbowler]

Feature Description

How do we map Analytics and Search Console API errors within the batch so that they can be surfaced in the Admin Settings and Email Reporting panel.

#11860 shows error notices in the frontend, we didn't correctly

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• Errors from Analytics, Search Console and AdSense APIs that occur during Email Report generation are logged with a category identifier and displayed as per the PUE Errors spreadsheet.
  • These include "permission errors" which are caused due to the user not having access to the linked properties and other "report errors" that occur when requests to the Google APIs fail for reasons other than permissions access.
  • The errors should be displayed only when all three attempts have failed for all emails within a single batch.
  • The errors should only be displayed for admin users within the User Settings Side Panel and within the Admin Settings for Email Reports.

Implementation Brief

Backend

• Update file includes/Core/Email_Reporting/Email_Reporting_Data_Requests.php

  • Add error category constants:

    • PERMISSIONS_ERROR = 'permissions_error'
    • REPORT_ERROR = 'report_error'

  • Add categorize_report_errors( WP_Error $error ) method:

    • Extract HTTP status from $error->get_error_data()['status']
    • Extract reason from $error->get_error_data()['reason']
    • Return PERMISSIONS_ERROR when ANY of the following conditions are met:
      • HTTP status is 401 or 403
      • Reason matches any of these Google API error reasons:
        • unauthorized - User is not authorized to perform the request
        • authError - Invalid authorization credentials
        • expired - Session has expired
        • required - User must be logged in
        • forbidden - Operation is forbidden
        • insufficientPermissions - User does not have sufficient permissions
        • accountDeleted - User account has been deleted
        • accountDisabled - User account has been disabled
        • accessNotConfigured - Project not configured to access API
    • Return REPORT_ERROR for all other cases (quota limits, invalid parameters, not found, server errors, etc.)

  • Update collect_analytics_payloads():

    • When is_wp_error( $payload ), call categorize_report_errors( $payload )
    • Add category_id to the existing WP_Error data by creating a new WP_Error object similar to this:

    site-kit-wp/includes/Core/Email_Reporting/Email_Report_Sender.php

    Lines 97 to 101 in 9e0b7c1

    	$send_result = new WP_Error(
    	$send_result->get_error_code(),
    	$send_result->get_error_message(),
    	array_merge( $error_data, array( 'category_id' => 'sending_error' ) )
    	);

  • Update collect_search_console_payloads() and collect_adsense_payloads():

    • When is_wp_error( $response ), call categorize_report_errors( $response )
    • Add category_id to the existing WP_Error data as above.

Frontend

• Create file assets/js/components/settings/email-reporting/notices/error/ReportErrorNotice.js

  • Display when category_id is report_error
  • Title and description copy per PUE Errors spreadsheet specifications and should depend on the module_slug passed when the category is report_error

• Create file assets/js/components/settings/email-reporting/notices/error/PermissionsErrorNotice.js

  • Display when category_id is permissions_error
  • Title and description copy per PUE Errors spreadsheet specifications and should depend on the module_slug passed when the category is permissions_error

• Update file assets/js/components/settings/email-reporting/EmailReportingErrorNotice.js

  • Check category_id from error data
  • Render PermissionsErrorNotice when category_id matches permissions_error
  • Render ReportErrorNotice when category_id matches report_error
  • Fall back to existing error handling for other error types (e.g., sending_error, server_error)

Test Coverage

• Update file tests/phpunit/integration/Core/Email_Reporting/Email_Reporting_Data_RequestsTest.php

  • Test categorize_report_errors returns PERMISSIONS_ERROR for 401/403 status codes
  • Test categorize_report_errors returns PERMISSIONS_ERROR for permission reasons (unauthorized, authError, insufficientPermissions, etc.)
  • Test categorize_report_errors returns REPORT_ERROR for other errors (quota, rate limit, invalid params)
  • Test that category_id is added to WP_Error data while preserving existing data fields

• Add JS tests for ReportErrorNotice and PermissionsErrorNotice components

  • Verify correct title and description rendering
  • Test component selection logic in EmailReportingErrorNotice

QA Brief

• Set up email reporting with a user having a valid email address. Set the frequency to say weekly.

• To test permissions_error variant:

• Set up site kit with analytics. But now, remove permissions to the property/account for the user subscribed for the email report above. This should throw several Insufficient Permissions errors on the dashboard as usual.

• Use the tester plugin to trigger the weekly cron task in the Email Reporting tab. Wait for about 20-30 mins for all three attempts to fail. If you want to speed this process up, wait for the cron to run at least once (about 2 mins), then verify the last batch failed once in the "Email logs" page added by the tester plugin and then go to the wp_posts_meta table in your DB, sort by descending order of meta_id and change the value of _send_attempts from 1 to 3.

• Now go to the Email Reports side panel and verify the error notice as per the AC.

• To test report_error variant:

• Set up site kit normally with a user having all permissions for SC/Analytics. Now trigger the cron in the tester plugin. If testing locally, switch off your wifi for 20-30 mins or follow the same instructions above to modify the _send_attempts record.

• Verify the Email Reports side panel shows the report error notice as per the AC.

Changelog entry

• Add error tracking for Site Kit modules to Email Reporting.

[benbowler]

@jimmymadon, here's my shake on how to implement this. This model can be expanded/extended to log the event category in different ways, depending on the other ways we need to categorise errors in the system. Then, for Scheduler errors, we extend the Email Log to take a "batch" in the case where errors occur which prevent any schedulers from being scheduled.

[jimmymadon]

@benbowler I have updated the ACs here to actually display the error in the front end as well as per the spreadsheet.

AC ✅

However, after reviewing the IB properly, I realised that we will lose some data from WP Error such as status and other existing data fields if we end up extending the Exception class - unless we pass those too. So I think we might have to revert to the previous IB to simply add data to the existing wp_error and pass it forward.

We also need to add a point in the IB to create new ReportErrorNotice and PermissionsErrorNotice notice components with appropriate copies from the spreadsheet and return those components within EmailReportingErrorNotice.js if the email category within the error returned is as per the spreadsheet.

[benbowler]

Hey @jimmymadon sounds good. I've reverted and added additional detail on the notice changes required to close the loop here.