Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
![](https://img.shields.io/pypi/pyversions/skyline-profiler.svg)
[![](https://img.shields.io/pypi/v/skyline-profiler.svg)](https://pypi.org/project/skyline-profiler/)

Skyline is a tool to profile and debug the training performance of [PyTorch](https://pytorch.org) neural networks.
DeepView.Profile is a tool to profile and debug the training performance of [PyTorch](https://pytorch.org) neural networks.

- [Installation](#installation)
- [Usage example](#getting-started)
Expand All @@ -15,9 +15,9 @@ Skyline is a tool to profile and debug the training performance of [PyTorch](htt

<h2 id="installation">Installation</h2>

Skyline works with *GPU-based* neural networks that are implemented in [PyTorch](https://pytorch.org).
DeepView.Profile works with *GPU-based* neural networks that are implemented in [PyTorch](https://pytorch.org).

To run Skyline, you need:
To run DeepView.Profile, you need:
- A system equipped with an NVIDIA GPU
- Python 3.7+
- [Poetry](https://python-poetry.org/)
Expand All @@ -32,7 +32,7 @@ To run Skyline, you need:

### Installation from source
```bash
git clone https://github.com/CentML/skyline.git
git clone https://github.com/CentML/DeepView.Profile
cd skyline
poetry install
poetry run skyline --help
Expand Down Expand Up @@ -62,7 +62,7 @@ python3 skyline

<h2 id="getting-started">Usage example</h2>

To use Skyline in your project, you need to first write an entry point file, which is a regular Python file that describes how your model is created and trained. See the [Entry Point](docs/providers.md) for more information.
To use DeepView.Profile in your project, you need to first write an entry point file, which is a regular Python file that describes how your model is created and trained. See the [Entry Point](docs/providers.md) for more information.

Once your entry point file is ready, there are two ways to profile interactive profiling and standalone profiling.

Expand All @@ -72,11 +72,11 @@ poetry run skyline interactive
```

### Standalone Profiling
Standalone profiling is useful when you just want access to Skyline's profiling functionality. Skyline will save the profiling results (called a "report") into a [SQLite database file](https://www.sqlite.org/) that you can then query yourself. We describe the database schema for Skyline's run time and memory reports in the [Run Time Report Format](docs/run-time-report.md) and [Memory Report Format](docs/memory-report.md) pages respectively.
Standalone profiling is useful when you just want access to DeepView.Profile's profiling functionality. DeepView.Profile will save the profiling results (called a "report") into a [SQLite database file](https://www.sqlite.org/) that you can then query yourself. We describe the database schema for DeepView.Profile's run time and memory reports in the [Run Time Report Format](docs/run-time-report.md) and [Memory Report Format](docs/memory-report.md) pages respectively.

To have Skyline perform run time profiling, you use the `skyline time`
To have DeepView.Profile perform run time profiling, you use the `skyline time`
subcommand. In addition to the entry point file, you also need to specify the
file where you want Skyline to save the run time profiling report using the
file where you want DeepView.Profile to save the run time profiling report using the
`--output` or `-o` flag.

```zsh
Expand Down Expand Up @@ -115,11 +115,11 @@ See [Releases](https://github.com/UofT-EcoSystem/skyline/releases)

<h2 id="meta">Meta</h2>

Skyline began as a research project at the [University of Toronto](https://web.cs.toronto.edu) in collaboration with [Geofrey Yu](mailto:[email protected]), [Tovi Grossman](https://www.tovigrossman.com) and [Gennady Pekhimenko](https://www.cs.toronto.edu/~pekhimenko/).
DeepView.Profile began as a research project at the [University of Toronto](https://web.cs.toronto.edu) in collaboration with [Geofrey Yu](mailto:[email protected]), [Tovi Grossman](https://www.tovigrossman.com) and [Gennady Pekhimenko](https://www.cs.toronto.edu/~pekhimenko/).

The accompanying research paper appears in the proceedings of UIST'20. If you are interested, you can read a preprint of the paper [here](https://arxiv.org/pdf/2008.06798.pdf).

If you use Skyline in your research, please consider citing our paper:
If you use DeepView.Profile in your research, please consider citing our paper:

```bibtex
@inproceedings{skyline-yu20,
Expand All @@ -136,4 +136,4 @@ It is distributed under Apache 2.0 license. See [LICENSE](LICENSE) and [NOTICE](

<h2 id="contributing">Contributing</h2>

Check out [CONTRIBUTING.md](CONTRIBUTING.md) for more information on how to help with Skyline.
Check out [CONTRIBUTING.md](CONTRIBUTING.md) for more information on how to help with DeepView.Profile.
14 changes: 7 additions & 7 deletions docs/memory-report.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
This page describes the database schema of the memory report that is generated by Skyline's `memory` subcommand. Recall that Skyline's reports (memory and run time) are [SQLite database files](https://www.sqlite.org/).
This page describes the database schema of the memory report that is generated by DeepView.Profile's `memory` subcommand. Recall that DeepView.Profile's reports (memory and run time) are [SQLite database files](https://www.sqlite.org/).

**NOTE:** Skyline's memory profiling is for GPU memory only.
**NOTE:** DeepView.Profile's memory profiling is for GPU memory only.

## Overview

Skyline tracks the memory usage associated with a model's *weights* and *activations*. Skyline will also report the peak amount of memory allocated during a training iteration.
DeepView.Profile tracks the memory usage associated with a model's *weights* and *activations*. DeepView.Profile will also report the peak amount of memory allocated during a training iteration.

Just like the run time report, Skyline also includes the stack trace associated with each activation or weight in the report. Skyline only includes the stack frames associated with files inside your project (i.e. files under your project's root directory).
Just like the run time report, DeepView.Profile also includes the stack trace associated with each activation or weight in the report. DeepView.Profile only includes the stack frames associated with files inside your project (i.e. files under your project's root directory).

## Tables

Expand Down Expand Up @@ -42,7 +42,7 @@ CREATE TABLE entry_types (
name TEXT NOT NULL
);
```
This is a table that stores mappings of Skyline's memory entry types (activations, weights) to numeric identifiers. Skyline maps weights to an entry type of `1`, and activations to an entry type of `2`.
This is a table that stores mappings of DeepView.Profile's memory entry types (activations, weights) to numeric identifiers. DeepView.Profile maps weights to an entry type of `1`, and activations to an entry type of `2`.

### `stack_correlation`

Expand Down Expand Up @@ -73,7 +73,7 @@ CREATE TABLE stack_frames (
```
This table holds the stack frames associated with a memory usage entry (both weights and activations). The `correlation_id` column is a foreign key that references the `correlation_id` in the `stack_correlation` table. File paths stored in the `file_path` column will be relative to the project's root directory and line numbers are 1-based.

**NOTE:** Skyline does not add an explicit foreign key constraint to the `correlation_id` column.
**NOTE:** DeepView.Profile does not add an explicit foreign key constraint to the `correlation_id` column.

**Ordering.**
There may be multiple stack frames associated with any given memory entry (i.e. any given `correlation_id`). The `ordering` column is used to keep track of the ordering among stack frames that share the same `correlation_id`. When sorted in ascending order by the `ordering` column, the stack frames will be ordered from most-specific (i.e. *closest* to the weight or operation responsible for the activation) to least-specific (i.e. *farthest* from the weight or operation responsible for the activation).
Expand All @@ -90,4 +90,4 @@ CREATE TABLE misc_sizes (
);
```

This table holds any miscellaneous memory usage information that is reported by Skyline. Currently, Skyline only uses this table to report the peak memory usage during one training iteration. This memory usage is reported using the `peak_usage_bytes` key.
This table holds any miscellaneous memory usage information that is reported by DeepView.Profile. Currently, DeepView.Profile only uses this table to report the peak memory usage during one training iteration. This memory usage is reported using the `peak_usage_bytes` key.
2 changes: 1 addition & 1 deletion docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,6 @@ def skyline_iteration_provider(model):
optimizer.step()
return iteration
```
One important thing to highlight is our use of a wrapper `ModelWithLoss` module. Skyline only provides breakdowns for operations that run inside the module returned by the model provider. We included the loss function in this wrapper module to have Skyline include it in the breakdown. We could have also placed the loss function call in the `iteration` function.
One important thing to highlight is our use of a wrapper `ModelWithLoss` module. DeepView.Profile only provides breakdowns for operations that run inside the module returned by the model provider. We included the loss function in this wrapper module to have DeepView.Profile include it in the breakdown. We could have also placed the loss function call in the `iteration` function.

You can place these provider functions either in a new file or directly in `model.py`. Whichever file contains the providers will be your project's entry point file. In this example, we defined the providers in a separate file called `entry_point.py` inside `my_project`.
22 changes: 11 additions & 11 deletions docs/remote.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Remote Profiling

## Terminology
- **Client:** The local machine where you run the Skyline plugin.
- **Server:** The remote machine where you want to run the Skyline profiler.
- **Client:** The local machine where you run the DeepView.Explore plugin.
- **Server:** The remote machine where you want to run the DeepView.Profile.

## Prerequisites
**SSH Access.**
At a minimum, you need SSH access to a server that allows SSH tunnelling. If the server machine exposes ports then it does not need to support SSH tunnelling.

**Skyline and Habitat.**
Install Skyline and (optionally Habitat) on your server to allow remote profiling.
**DeepView.Profile and DeepView.Predict.**
Install DeepView.Profile and (optionally DeepView.Predict) on your server to allow remote profiling.

**[VSCode Remote - SSH extension.](https://code.visualstudio.com/docs/remote/ssh)**
This extension allows users to connect to a remote machine and run extensions remotely. The extension handles most of the heavy lifting so it makes it easy to use the Skyline plugin on remote machines.
This extension allows users to connect to a remote machine and run extensions remotely. The extension handles most of the heavy lifting so it makes it easy to use DeepView.Explore on remote machines.

**Installing the Skyline Plugin on the Server**
To install the Skyline plugin on the server, take the following steps.
**Installing the DeepView.Explore on the Server**
To install the DeepView.Explore plugin on the server, take the following steps.
1. Connect to your server via SSH.
2. Get the VSIX file following the installation instructions. Take note the path to the VSIX file.
2. Open VSCode on your client and connect to your server.
Expand All @@ -25,7 +25,7 @@ To install the Skyline plugin on the server, take the following steps.
## Starting a Remote Profiling Session

### Starting the Skyline Profiler
The Skyline Profiler needs to running on the server to enable the plugin. You can connect to the server via SSH and start the Skyline profiler by running the `skyline interactive` command as usual.
DeepView.Profile needs to running on the server to enable the plugin. You can connect to the server via SSH and start DeepView.Profile by running the `skyline interactive` command as usual.

```zsh
poetry run skyline interactive
Expand All @@ -34,8 +34,8 @@ poetry run skyline interactive
If you want to use a different port, you can use the `--port` flag to tell the profiler to listen on a different port.

```zsh
poetry run skyline interactive
poetry run skyline interactive --port portNumber
```

### Starting the Skyline Plugin
Launch VSCode and open Skyline by running the Skyline command in the command palette (Ctrl-Shift-P on Linux/Windows, ⌘-Shift-P on macOS). Select your project root and begin profiling.
### Starting DeepView.Explore
Launch VSCode and open DeepView.Explore by running the Skyline command in the command palette (Ctrl-Shift-P on Linux/Windows, ⌘-Shift-P on macOS). Select your project root and begin profiling.
12 changes: 6 additions & 6 deletions docs/run-time-report.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
This page describes the database schema of the run time report that is generated by Skyline's `time` subcommand. Recall
that Skyline's reports (memory and run time) are [SQLite database files](https://www.sqlite.org/).
This page describes the database schema of the run time report that is generated by DeepView.Profile's `time` subcommand. Recall
that DeepView.Profile's reports (memory and run time) are [SQLite database files](https://www.sqlite.org/).

## Overview

Skyline's run time report contains a breakdown of the run times of each operation that runs in one training iteration. Skyline only tracks the
DeepView.Profile's run time report contains a breakdown of the run times of each operation that runs in one training iteration. DeepView.Profile only tracks the
operations that execute as a part of either the forward and backward pass.

For each tracked operation, Skyline also includes the stack trace leading to that operation. Skyline only includes the stack frames associated with files inside your project (i.e. files under your project's root directory).
For each tracked operation, DeepView.Profile also includes the stack trace leading to that operation. DeepView.Profile only includes the stack frames associated with files inside your project (i.e. files under your project's root directory).

## Tables

Expand All @@ -23,7 +23,7 @@ CREATE TABLE run_time_entries (

This table holds the measured run time(s) of each tracked operation. Each entry in this table represents one operation *instance* (i.e. one invocation of an operation). The columns in this table are self-explanatory.

**NOTE:** Skyline reports run times in milliseconds.
**NOTE:** DeepView.Profile reports run times in milliseconds.

**Backward Pass.**
Note that not every operation is necessarily involved in the backward pass. When an operation is not in the backward pass, `backward_ms` will be `NULL`.
Expand All @@ -43,7 +43,7 @@ CREATE TABLE stack_frames (

This table holds the stack frames associated with each tracked operation. The `entry_id` column is a foreign key that references the `id` column in `run_time_entries`.

**NOTE** Skyline does not add an explicit foreign key constraint to the `entry_id` column.
**NOTE** DeepView.Profile does not add an explicit foreign key constraint to the `entry_id` column.

**Ordering.**
There may be multiple stack frames associated with any given tracked operation (i.e. any given `entry_id`). The `ordering` column is used to keep track of the ordering among stack frames that share the same `entry_id`. When sorted in ascending order by the `ordering` column, the stack frames will be ordered from most-specific (i.e. *closest* to the operation's call site) to least-specific (i.e. *farthest* from the operation's call site).
2 changes: 1 addition & 1 deletion examples/gnmt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This directory contains an implementation of GNMT that was adapted from the
code found in the [MLPerf training repository](https://github.com/mlperf/training/tree/master/rnn_translator).

To launch an interactive Skyline profiling session for GNMT, run
To launch an interactive DeepView.Profile session for GNMT, run
```
skyline interactive entry_point.py
```
Expand Down