Skip to content

Add roofline_analyzer 0.1.1#182

Merged
ajassani merged 1 commit intomainfrom
feature/roofline_analyzer
May 26, 2025
Merged

Add roofline_analyzer 0.1.1#182
ajassani merged 1 commit intomainfrom
feature/roofline_analyzer

Conversation

@tykow
Copy link
Copy Markdown
Contributor

@tykow tykow commented May 26, 2025

I've created a python cli application that
as an input takes the XLSX that is, to my knowledge. generated by generate_perf_report.py from
#138

and when combined with a config-file that has e.g. MI300X bfloat16 values:
image

It will generate a naive roofline analysis for the GEMMs used in e.g. a client model:
image
image

and it will generate a new XLSX as an output, that has:

  1. bound_type: what is the first limiting roofline (memory or compute), based on the "AI ridge point"
  2. reference_roofline: what is used as the reference value for roofline distance calculation
  3. bound_distance: what is the distance from kernel to the maf_roofline in question
  4. bound_distance_pct: above in percentage

image
and AI ridge points for both the max achievable flops (maf) and max theoretical flops (mtf) i.e. the point that divides whether the kernel is memory or compute limited:
image

What is new in 0.1.1 compared to 0.1.0 is that the kernels will get a rolling numbering, which is also used to indicate which point on the plot belongs to which kernel:
image
image

For additional information, consult the extensive README.md

@ajassani ajassani requested a review from Copilot May 26, 2025 13:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds version 0.1.1 of the roofline_analyzer CLI tool, introducing kernel ID numbering for plot annotations and full end-to-end Excel-based roofline analysis.

  • Introduces an ID column for kernels and ties IDs into plot annotations.
  • Implements modular functions for reading, analyzing, plotting, and exporting Excel data.
  • Updates project metadata (pyproject.toml), sample config, and documentation for 0.1.1.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
examples/custom_workflows/roofline_analyzer/src/xlsx.py Excel data parsing, analysis column addition, and export functions
examples/custom_workflows/roofline_analyzer/src/roofline.py Roofline model calculations and plot generation
examples/custom_workflows/roofline_analyzer/src/cli.py Command-line interface for config handling and summary display
examples/custom_workflows/roofline_analyzer/pyproject.toml Project metadata, version bump to 0.1.1, dependencies, and entry point
examples/custom_workflows/roofline_analyzer/mi300x_bfloat16.toml Sample accelerator configuration for MI300X bfloat16
examples/custom_workflows/roofline_analyzer/main.py Main CLI command integrating all modules
examples/custom_workflows/roofline_analyzer/README.md Installation, usage instructions, and output schema documentation
examples/custom_workflows/roofline_analyzer/.gitignore Ignore patterns for build artifacts and generated files
Comments suppressed due to low confidence (2)

examples/custom_workflows/roofline_analyzer/src/xlsx.py:1

  • Key analysis and export functions lack accompanying unit tests; consider adding tests for read_xlsx, add_analysis_columns, and export_to_xlsx to verify behavior and guard against regressions.
import pandas as pd

examples/custom_workflows/roofline_analyzer/README.md:93

  • [nitpick] Markdown bullets for bound_distance and bound_distance_pct aren't indented under the nested list for the analyzed sheet section; adjust indentation for consistent formatting.
* **bound_distance**: Distance to the nearest roofline

def add_analysis_columns(
df: pd.DataFrame,
config: Dict[str, Any],
ai_ridge_mtf: float,
Copy link

Copilot AI May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter ai_ridge_mtf is never used in add_analysis_columns; consider removing it or using it as intended to keep the signature accurate.

Suggested change
ai_ridge_mtf: float,

Copilot uses AI. Check for mistakes.
Comment on lines +56 to +57
# Set config to use 'ID' as kernel_name_column for plotting
config['excel']['kernel_name_column'] = 'ID'
Copy link

Copilot AI May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutating the passed config dict inside this function may lead to unexpected side effects; use a local variable for kernel_name_column instead of modifying the original config.

Suggested change
# Set config to use 'ID' as kernel_name_column for plotting
config['excel']['kernel_name_column'] = 'ID'
# Use a local variable for 'ID' as kernel_name_column for plotting
kernel_name_column = 'ID'

Copilot uses AI. Check for mistakes.
@ajassani ajassani merged commit 1971b05 into main May 26, 2025
@ajassani ajassani deleted the feature/roofline_analyzer branch May 26, 2025 13:54
lauri9 pushed a commit that referenced this pull request Jun 11, 2025
I've created a python cli application that 
as an input takes the XLSX that is, to my knowledge. generated by
`generate_perf_report.py` from
#138

and when combined with a config-file that has e.g. MI300X bfloat16
values:

![image](https://github.com/user-attachments/assets/e8ab7f6e-fa19-4537-aa35-ffd3e1a43eb5)

It will generate a naive roofline analysis for the GEMMs used in e.g. a
client model:

![image](https://github.com/user-attachments/assets/06e3a4b1-fc6b-43af-86fe-70bfd2fde24f)

![image](https://github.com/user-attachments/assets/3fb5c4f2-bfd5-40c0-96a9-c8524d523c2e)

and it will generate a new XLSX as an output, that has:
1. **bound_type**: what is the first limiting roofline (memory or
compute), based on the "AI ridge point"
2. **reference_roofline**: what is used as the reference value for
roofline distance calculation
3. **bound_distance**: what is the distance from kernel to the
maf_roofline in question
4. **bound_distance_pct**: above in percentage


![image](https://github.com/user-attachments/assets/d6e4087b-b499-4fde-8a7d-a3465c75c68b)
and AI ridge points for both the max achievable flops (maf) and max
theoretical flops (mtf) i.e. the point that divides whether the kernel
is memory or compute limited:

![image](https://github.com/user-attachments/assets/a32c7b9d-1abe-4246-9b9c-417d1308383b)

What is new in 0.1.1 compared to 0.1.0 is that the kernels will get a
rolling numbering, which is also used to indicate which point on the
plot belongs to which kernel:

![image](https://github.com/user-attachments/assets/2ab46e4c-2363-43ac-8f8d-48f2a094f159)

![image](https://github.com/user-attachments/assets/835baeb7-5fc9-4836-85a1-3c012022f0f9)

For additional information, consult the extensive README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants