-
Notifications
You must be signed in to change notification settings - Fork 243
Add initial MDS Metrics definitions #487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
schnuerle
merged 8 commits into
openmobilityfoundation:feature-metrics
from
lacuna-tech:add_metrics_definitions
Oct 16, 2020
Merged
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
a56dc25
Add MDS core metrics definitions.
whereissean d53a4be
Add initial MDS dockless metrics definitions.
whereissean 48bc4e7
update readme
joanathan 04e1f0d
update readme
joanathan 9d0d2c1
update link
joanathan 55dfab9
updated main mds readme, added toc and an example folder
joanathan 512c7d5
Merge remote-tracking branch 'omf/dev' into add_metrics_definitions
joanathan 1346c8f
update link to geography
joanathan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| # Mobility Data Specification: Metrics | ||
|
|
||
| An API for requesting **historical** calculated [metrics](core_metrics.md) and aggregations of MDS data. | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| - [General Information](#general-information) | ||
| - [Date and Time Format](#date-and-time-format) | ||
| - [Metrics Discovery API](#metrics-discovery-api) | ||
| - [Metrics Query API](#metrics-query-api) | ||
|
|
||
| ## General Information | ||
|
|
||
| Objectives: | ||
| - Cities need a number of clearly defined best practice metrics for operating, measuring, and managing emerging micro mobility programs using MDS data. | ||
| - There is currently no standard counting methodology that mobility providers and cities have agreed upon. This consequently causes friction when establishing a new mobility program and evaluating its impact | ||
| - Cities need to rely upon trusted data sources upon which to perform longer term studies on citizen impact. | ||
| - Mobility providers would like consistent rules between each city deployment in order to make their operations more scalable. | ||
|
|
||
| Initial Design Use Cases: | ||
| - For cities to republish data ingested from MDS (Agency or Provider data) for use in visualization, analysis, or other applications | ||
| - For cities to publish calculated metrics back to providers allowing shared understanding of how policies are measured and enforced | ||
|
|
||
| ## Date and Time Format | ||
|
|
||
| All dates and times (datetime) are [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) formatted strings (YYYY-MM-DDTHHMM), with minute granularity supported and time zone (default UTC) or included offset. Dates and times may also be specified using a numeric *Unix epoch/timestamp* | ||
|
|
||
| All interval durations (duration) are [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) duration format strings (e.g. PT15M, PT1H, P1D). | ||
|
|
||
| ## Metrics Discovery API | ||
|
|
||
| ### Method | ||
|
|
||
| `GET /metrics` | ||
|
|
||
| Returns a discovery response describing the supported metrics, times, intervals, dimensions and filters. | ||
|
|
||
| ### Parameters | ||
|
|
||
| None. | ||
|
|
||
| ### Response | ||
|
|
||
| | Name | Type | Required | Comments | | ||
| | ------------------ | ---------- | -------- | --------------------------------------------------------------------------------------------------------- | | ||
| | `metrics` | metric[] | Yes | List of supported metrics. | | ||
| | `metric.measures` | string[] | Yes | List of measure names. [See metric names](core_metrics.md) | | ||
| | `metric.since` | datetime | Yes | Earliest supported start date for fetching metrics. Minute (MM) must be divisible by minimum `interval`. | | ||
| | `metric.intervals` | duration[] | Yes | A list of interval durations in ascending order. Minimum (first) interval duration is the default. | | ||
| | `max_intervals` | integer | Yes | Maximum number intervals that can be returned. | | ||
| | `dimensions` | string[] | Yes | List of supported dimensions. [See dimensions.](core_metrics.md#dimensions) | | ||
| | `filters` | string[] | Yes | List of supported filters for metrics. [See filters.](core_metrics.md#filters) | | ||
|
|
||
| ### Response Schema | ||
| ```js | ||
| { | ||
| "metrics": [ | ||
| { | ||
| "measures": [string], | ||
| "since": datetime, | ||
| "intervals": [duration] | ||
| } | ||
| ], | ||
| "max_intervals": number, | ||
| "dimensions": [string], | ||
| "filters": [string] | ||
| } | ||
| ``` | ||
| See the [Metrics Examples](examples) for ways these can be implemented. | ||
|
|
||
| ## Metrics Query API | ||
|
|
||
| ### Method | ||
|
|
||
| `POST /metrics` | ||
|
|
||
| Supports querying one or more metrics with the following parameters. | ||
|
|
||
| ### Parameters | ||
|
|
||
| | Name | Type | Required | Comments | | ||
| | --------------- | ------------- | -------- | ----------------------------------------------------------------------- | | ||
| | `measures` | string[] | Yes | list of measures to return. [See metric names](core_metrics.md) | | ||
| | `interval` | duration | Yes | Duration for metrics intervals. | | ||
| | `start_date` | datetime | Yes | ISO 8601 formatted start date or numeric timestamp to fetch metrics. | | ||
| | `end_date` | datetime | No | ISI 8601 formatted end date or numberic timestamp to fetch metrics. | | ||
| | `timezone` | timezone | No | TZ Database time zone name (default: "UTC") | | ||
| | `dimensions` | string[] | No | List of dimension names. [See dimensions.](#dimensions) | | ||
| | `filters` | filter[] | No | Filters for metrics to return of format [See filters.](#filters) . | | ||
| | `filter.name` | string | No | Name of filter (e.g. 'vehicle_type') | | ||
| | `filter.values` | string[] | No | List of values to filter for (e.g ['car', 'moped']) | | ||
|
|
||
| Note: If `timezone` is specified then `start_date`, `end_date`, and all _datetime_ column values will be | ||
| converted to the specified time zone. If not, parameters will be converted to and the results will be | ||
| displayed in UTC. If `start_date` is specified as a numeric _Unix timestamp_ then all _datetime_ column values will be | ||
| displayed using numeric _Unix timestamp_ values as well. The `timezone` parameter is not allowed when using numeric | ||
| _Unix timestamp_ values. | ||
|
|
||
| Note: If `end_date` is specified, all intervals that *begin* between the specified `start_date` and the `end_date` *(inclusive)* are fetched. If `end_date` is not specified, only the interval that begins *on* the specified `start_date` is fetched. | ||
|
|
||
| ### Response | ||
|
|
||
| | Name | Type | Comments | | ||
| | -------------------- | ---------- | --------------------------------------------------------------- | | ||
| | `id` | uuid | Unique id for query | | ||
| | `query.measures` | string[] | From request. | | ||
| | `query.interval` | duration | From Request. | | ||
| | `query.start_date` | datetime | From Request. | | ||
| | `query.end_date` | datetime | From Request. | | ||
| | `query.dimensions` | string[] | From Request. | | ||
| | `query.filters` | filter[] | From Request. | | ||
| | `query.timezone` | timezone | From Request. | | ||
| | `columns` | column[] | Array of column information | | ||
| | `column.name` | string | Name of metric or dimension column. | | ||
| | `column.column_type` | string | Type of column: `measure` or `dimension`. | | ||
| | `column.data_type` | string | Data type of column: `datetime`, `string`, `integer`, `float`. | | ||
| | `rows` | values[][] | Array of row arrays containing the dimension and metric values. | | ||
|
|
||
| ### Response Schema | ||
| ```js | ||
| { | ||
| "id": string, | ||
| "query": { | ||
| "measures": [string], | ||
| "interval": duration, | ||
| "start_date": datetime, | ||
| "end_date": datetime, | ||
| "dimensions": [string], | ||
| "filters": [ | ||
| { | ||
| "name": string, | ||
| "values": [string] | ||
| } | ||
| ] | ||
| }, | ||
| "columns": [ | ||
| { | ||
| "name": string, | ||
| "column_type": string, | ||
| "data_type": string | ||
| } | ||
| ], | ||
| "rows": [ | ||
| [string | number] | ||
| ] | ||
| } | ||
| ``` | ||
| See the [Metrics Examples](examples) for ways these can be implemented. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| ## MDS Core Metrics | ||
|
|
||
| The core metrics are a set of defined, consistent MDS metrics that provide building blocks for other MDS metrics, regardless of vehicle type. | ||
|
|
||
| The table below represents supported MDS core metrics and definition. All metrics are aggregated by time interval and geographic areas. This [document](https://docs.google.com/document/d/1rOhnaKWPSZApfWhFd1lzurXMbWLuZTJAYCLoxT2PQ14/edit?usp=sharing) provides methodologies and sample calculations of MDS metrics. | ||
|
|
||
|
|
||
| |No| Metric | Description | | ||
| |--| --| -- | | ||
| |1.1 | vehicles.[status].avg | The average number of vehicles in [MDS status](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-events). | | ||
| |1.2 | vehicles.[status].min | The minimum number of vehicles in [MDS status](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-events). | | ||
| |1.3| vehicles.[status].max | The maximum number of vehicles in [MDS status](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-events). | | ||
| |1.4| vehicles.[status].duration.sum | The total duration (in seconds) vehicles spent in a specified [MDS status](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-events). | | ||
| |1.5| events.[event_type].count |The number of [MDS event type](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-events) received. | | ||
| |1.6| trips.[start_loc/end_loc].count|The number of trips aggregated by either the start or first enter, or end or final leave locations.| | ||
| |1.7| trips.[start_loc/end_loc]_duration.avg|The average trip duration (in seconds) aggregated by either the start or first enter, or end or final leave locations.| | ||
| |1.8| trips.[start_loc/end_loc]_duration.sum| The total trip duration (in seconds) aggregated by either the start or first enter, or end or final leave locations. | | ||
| |1.9| trips.[start_loc/end_loc]_distance.avg| The average trip distance (in meter) aggregated by either the start or first enter, or end or final leave locations. | | ||
| |1.10| trips.[start_loc/end_loc]_distance.sum|The total trip distance (in meter) aggregated by either the start or first enter, or end or final leave locations.| | ||
|
|
||
| ### Dimensions | ||
|
|
||
| The following represent the suggested MDS core metric dimensions: | ||
|
|
||
| | Dimension | Description | | ||
| | ------------ | -------------------------------------------------------------------------------------------------------------------------- | | ||
| | provider_id | Transportation provider id issued by OMF and [tracked here](https://github.com/openmobilityfoundation/mobility-data-specification/blob/dev/providers.csv) | | ||
| | [geography_type] | [MDS Geography](https://github.com/openmobilityfoundation/mobility-data-specification/blob/dev/policy/README.md#geographies) e.g. policy, jurisdictions, council_districts | | ||
| | vehicle_type | [Vehicle Type](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-type) defined by MDS | | ||
|
|
||
| ### Filters | ||
|
|
||
| The following represent the suggested MDS core metric filters: | ||
|
|
||
| | Filter | Description | | ||
| | ------------ | -------------------------------------------------------------------------------------------------------------------------- | | ||
| | provider_id | Transportation provider id issued by OMF and [tracked here](https://github.com/openmobilityfoundation/mobility-data-specification/blob/dev/providers.csv) | | ||
| | geography_id | [MDS Geography](https://github.com/openmobilityfoundation/mobility-data-specification/blob/dev/policy/README.md#geographies) | | ||
| | vehicle_type | [Vehicle Type](https://github.com/openmobilityfoundation/mobility-data-specification/tree/dev/agency#vehicle-type) defined by MDS | | ||
|
|
||
|
|
||
| ## Other MDS Metrics | ||
|
|
||
| - [MDS Dockless Metrics](dockless_metrics.md) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| ## MDS Dockless Metrics | ||
|
|
||
| MDS dockless metrics are a set of defined metrics and compound metrics useful for measuring dockless vehicle activity with MDS data. Dockless metrics measure vehicle activity when they are in the public right of way, which means vehicles were in either available, unavailable, reserved, or trip state. | ||
|
|
||
| The table below represents supported MDS dockless metrics extension and definition. All metrics are aggregated by time interval and geographic areas. This [document](https://docs.google.com/document/d/1rOhnaKWPSZApfWhFd1lzurXMbWLuZTJAYCLoxT2PQ14/edit?usp=sharing) provides methodologies and sample calculations of MDS metrics. | ||
|
|
||
| |No| Metric | Description | | ||
| |--|-- | -- | | ||
| |2.1|dockless.deployed.avg | The average number of vehicles in the public right of way.| | ||
| |2.2|dockless.deployed.avg.min|The minimum of the average number of vehicles in the public right of way. | | ||
| |2.3| dockless.deployed.min | The minimum number of vehicles in the public right of way.| | ||
| |2.4| dockless.deployed.avg.max| The maximum average number of vehicles in the public right of way. | | ||
| |2.5| dockless.deployed.max | The maximum number of vehicles in the public right of way. | | ||
| |2.6| dockless.active.count |The number of vehicles in the public right of way that have at least one trip during the interval, i.e. hourly. | | ||
| |2.7| dockless.inactive.count|The number of vehicles in the public right of way that have no trip during the interval. | | ||
| |2.8| dockless.utilization.idle.percent|The percentage of time vehicles spent not in trip to the total time vehicles spent in the public right of way.| | ||
| |2.9 | dockless.utilization.active.percent|The percentage of vehicles that had at least one trip during the interval.| | ||
| |2.10|dockless.utilization.trips_per_vehicle.avg| The average number of trips for all vehicles in the public right of way during the interval.| | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.