Skip to content

Adding Plugin query UI, filters and Variables integration #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0e5b558
Adding Datastreams related observations
MostafaMagdyy Jul 6, 2025
3d893b3
Adding Things Historical Locations
MostafaMagdyy Jul 8, 2025
9012f85
Adding Entites Visuallization in addition to grafana format and filte…
MostafaMagdyy Jul 9, 2025
fa49461
Update README.md
MostafaMagdyy Jul 9, 2025
97215dc
Adding Filters for Datastreams (Basic,Measurement, Temporal, Spatial,…
MostafaMagdyy Jul 10, 2025
6c0c92d
Fixing a bug in getfields function
MostafaMagdyy Jul 10, 2025
fe38282
Refactoring DataStream Transformation
MostafaMagdyy Jul 10, 2025
8d63711
Adding Observations filters
MostafaMagdyy Jul 10, 2025
8e4afc3
Adding Grafana Variables
MostafaMagdyy Jul 21, 2025
16225d9
Adding Multiple Variables support
MostafaMagdyy Jul 22, 2025
dff76ad
Variables handling as filters and displayed in Query preview
MostafaMagdyy Jul 22, 2025
38e390d
Adding Custom Query Editor for Variables Selection
MostafaMagdyy Jul 23, 2025
4af3927
Updating Readme.md
MostafaMagdyy Jul 23, 2025
35f15d8
Update README.md
MostafaMagdyy Jul 23, 2025
9271308
Adding POLYGON Geography Support for spatial filters
MostafaMagdyy Jul 23, 2025
83cc72a
Testing Connection to postgreSQL database as datasource
MostafaMagdyy Jul 23, 2025
ffda622
Adding Things trasformations
MostafaMagdyy Aug 4, 2025
133f33e
Refactoring transformation logic into modular functions
MostafaMagdyy Aug 5, 2025
eeec490
Adding Terramap with leaflet adapter to draw polygons
MostafaMagdyy Aug 5, 2025
cb4704b
Adding Entity filters
MostafaMagdyy Aug 6, 2025
0c15986
Installing orchestracities-map-panel to support Polygon Visualization
MostafaMagdyy Aug 6, 2025
f5fca9a
Adding Locations,Historical Locations Transformations
MostafaMagdyy Aug 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 166 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,166 @@
# istSOS4-GrafanaPlugin
A Grafana Plugin to Create Customized Dashboards
# istSOS4 Grafana Plugin

A powerful Grafana data source plugin for connecting to istSOS4 server to create customized dashboards and visualize sensor data in real-time.

## 🚀 Features

- **SensorThings API Integration**: Support for OGC SensorThings API
- **Advanced Filtering**: Comprehensive filter system supporting:
- Basic filters (field-based comparisons)
- Temporal filters (date ranges and temporal functions)
- Spatial filters (geometric queries)
- Measurement filters (sensor-specific data filtering)
- Complex OData expressions
- **Variable Support**: Dashboard template variables for dynamic queries
- **Real-time Data**: Live data visualization with automatic refresh
- **Dynamic Panels & Dashboards**: Ability to create panels and dashboards to visualize:
- Datastream Observations
- Locations and Historical Locations of Things

## 📋 Prerequisites

- **Node.js**: Version 18.x or higher
- **npm**: Version 8.x or higher
- **Grafana (Local Installation)**: Required for Method 1
- **Docker**: Required for Method 2 (Complete Development Environment)

## 🛠️ Installation & Setup

### Method 1: Plugin Development Only (npm run dev)

**Use this when**: You want to develop the plugin and use it with your **existing Grafana installation**.

> ⚠️ You must have Grafana installed and running locally.

1. **Clone the repository**:
```bash
git clone https://github.com/MostafaMagdyy/istSOS4-GrafanaPlugin.git
cd istSOS4-GrafanaPlugin/istsos4-istsos4grafana-datasource
```

2. **Install dependencies**:
```bash
npm install
```

3. **Build the plugin**:
```bash
npm run build
```

4. **Run in development mode** (watches for changes and rebuilds automatically):
```bash
npm run dev
```

5. **Install plugin in your local Grafana**:
- Copy the `dist/` folder to your Grafana plugins directory
- Restart your local Grafana instance
- The plugin will auto-reload when you make changes

**Note**: This method requires you to have Grafana already installed and running separately.

---

### Method 2: Complete Development Environment (npm run server) – **Recommended**

**Use this when**: You want a complete, isolated development environment with Grafana + Plugin.

1. **Clone and navigate to the project**:
```bash
git clone https://github.com/MostafaMagdyy/istSOS4-GrafanaPlugin.git
cd istSOS4-GrafanaPlugin/istsos4-istsos4grafana-datasource
```
2. **Install dependencies**:
```bash
npm install
```

3. **Start the complete development environment**:
```bash
npm run server
```

This command:
- Builds a Docker container with Grafana + your plugin
- Starts Grafana on `http://localhost:3000`
- Automatically loads the plugin (unsigned plugins enabled)
- Mounts your source code for live development

4. **Access Grafana**:
- Open your browser and go to `http://localhost:3000`
- Default credentials: `admin` / `admin`
- The istSOS4 plugin will be available in the data sources

5. **Development workflow**:
```bash
# In another terminal, watch for changes and rebuild
npm run dev

# The Docker container will pick up the new build automatically
# Just refresh your browser to see changes
```

---

## 🔧 Configuration

### 1. Add Data Source

1. In Grafana, go to **Configuration** → **Data Sources**
2. Click **Add data source**
3. Search for and select **istSOS4**
4. Configure the connection:
- **URL**: Your istSOS4 server URL
- **Token URL**: Your server Token URL
- **Basic Auth**: Configure your username and password

### 2. Test Connection

Click **Save & Test** to verify the connection to your istSOS4 server.

---

## 📊 Usage

### Creating Queries

1. **Create a new dashboard panel**
2. **Select istSOS4 as the data source**
3. **Configure your query**:
- **Entity**: Choose from Things, Datastreams, Sensors, etc.
- **Entity ID**: (Optional) Specify a particular entity
- **Expand**: Include related entities
- **Filters**: Add complex filtering logic
- **Variables**: Use dashboard variables for dynamic queries

### Filter Types

- **Basic**: Simple field comparisons (`name eq 'Temperature'`)
- **Temporal**: Date/time filtering with ranges or functions
- **Spatial**: Geographic queries using geometries
- **Measurement**: Sensor-specific data filtering
- **Complex**: Custom OData expressions

### Template Variables

Create dashboard variables to make your dashboards dynamic:

1. Go to **Dashboard Settings** → **Variables**
2. Add a new variable with istSOS4 data source
3. Use the variable in your queries with `$variableName` syntax

---

## 📄 License

This project is licensed under the Apache-2.0 License.

---

## 🏗️ Built With

- [Grafana Plugin SDK](https://grafana.com/developers/plugin-tools/)
- [React](https://reactjs.org/)
- [TypeScript](https://www.typescriptlang.org/)
- [OGC SensorThings API](https://www.ogc.org/standards/sensorthings)
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,4 @@ services:
GF_LOG_LEVEL: debug
GF_DATAPROXY_LOGGING: 1
GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: istsos4-istsos4grafana-datasource
GF_INSTALL_PLUGINS: orchestracities-map-panel
3 changes: 3 additions & 0 deletions istsos4-istsos4grafana-datasource/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,6 @@ ci/
.idea

.eslintcache

# docs
docs/
135 changes: 0 additions & 135 deletions istsos4-istsos4grafana-datasource/README.md
Original file line number Diff line number Diff line change
@@ -1,136 +1 @@
# istSOS4 Grafana data source plugin

This template is a starting point for building a Data Source Plugin for Grafana.

## What are Grafana data source plugins?

Grafana supports a wide range of data sources, including Prometheus, MySQL, and even Datadog. There’s a good chance you can already visualize metrics from the systems you have set up. In some cases, though, you already have an in-house metrics solution that you’d like to add to your Grafana dashboards. Grafana Data Source Plugins enables integrating such solutions with Grafana.

## Getting started

### Backend

1. Update [Grafana plugin SDK for Go](https://grafana.com/developers/plugin-tools/key-concepts/backend-plugins/grafana-plugin-sdk-for-go) dependency to the latest minor version:

```bash
go get -u github.com/grafana/grafana-plugin-sdk-go
go mod tidy
```

2. Build backend plugin binaries for Linux, Windows and Darwin:

```bash
mage -v
```

3. List all available Mage targets for additional commands:

```bash
mage -l
```

### Frontend

1. Install dependencies

```bash
npm install
```

2. Build plugin in development mode and run in watch mode

```bash
npm run dev
```

3. Build plugin in production mode

```bash
npm run build
```

4. Run the tests (using Jest)

```bash
# Runs the tests and watches for changes, requires git init first
npm run test

# Exits after running all the tests
npm run test:ci
```

5. Spin up a Grafana instance and run the plugin inside it (using Docker)

```bash
npm run server
```

6. Run the E2E tests (using Playwright)

```bash
# Spins up a Grafana instance first that we tests against
npm run server

# If you wish to start a certain Grafana version. If not specified will use latest by default
GRAFANA_VERSION=11.3.0 npm run server

# Starts the tests
npm run e2e
```

7. Run the linter

```bash
npm run lint

# or

npm run lint:fix
```

# Distributing your plugin

When distributing a Grafana plugin either within the community or privately the plugin must be signed so the Grafana application can verify its authenticity. This can be done with the `@grafana/sign-plugin` package.

_Note: It's not necessary to sign a plugin during development. The docker development environment that is scaffolded with `@grafana/create-plugin` caters for running the plugin without a signature._

## Initial steps

Before signing a plugin please read the Grafana [plugin publishing and signing criteria](https://grafana.com/legal/plugins/#plugin-publishing-and-signing-criteria) documentation carefully.

`@grafana/create-plugin` has added the necessary commands and workflows to make signing and distributing a plugin via the grafana plugins catalog as straightforward as possible.

Before signing a plugin for the first time please consult the Grafana [plugin signature levels](https://grafana.com/legal/plugins/#what-are-the-different-classifications-of-plugins) documentation to understand the differences between the types of signature level.

1. Create a [Grafana Cloud account](https://grafana.com/signup).
2. Make sure that the first part of the plugin ID matches the slug of your Grafana Cloud account.
- _You can find the plugin ID in the `plugin.json` file inside your plugin directory. For example, if your account slug is `acmecorp`, you need to prefix the plugin ID with `acmecorp-`._
3. Create a Grafana Cloud API key with the `PluginPublisher` role.
4. Keep a record of this API key as it will be required for signing a plugin

## Signing a plugin

### Using Github actions release workflow

If the plugin is using the github actions supplied with `@grafana/create-plugin` signing a plugin is included out of the box. The [release workflow](./.github/workflows/release.yml) can prepare everything to make submitting your plugin to Grafana as easy as possible. Before being able to sign the plugin however a secret needs adding to the Github repository.

1. Please navigate to "settings > secrets > actions" within your repo to create secrets.
2. Click "New repository secret"
3. Name the secret "GRAFANA_API_KEY"
4. Paste your Grafana Cloud API key in the Secret field
5. Click "Add secret"

#### Push a version tag

To trigger the workflow we need to push a version tag to github. This can be achieved with the following steps:

1. Run `npm version <major|minor|patch>`
2. Run `git push origin main --follow-tags`

## Learn more

Below you can find source code for existing app plugins and other related documentation.

- [Basic data source plugin example](https://github.com/grafana/grafana-plugin-examples/tree/master/examples/datasource-basic#readme)
- [`plugin.json` documentation](https://grafana.com/developers/plugin-tools/reference/plugin-json)
- [How to sign a plugin?](https://grafana.com/developers/plugin-tools/publish-a-plugin/sign-a-plugin)
6 changes: 6 additions & 0 deletions istsos4-istsos4grafana-datasource/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@ services:
extends:
file: .config/docker-compose-base.yaml
service: grafana
# networks:
# - istsos4_shared_net

# networks:
# istsos4_shared_net:
# external: true
Loading