Add Panel Frontend.

[MarcSkovMadsen]

🚀 Feature

Add PanelFrontend similarly to StreamlitFrontend.

Motivation

Panel and the HoloViz ecosystem provides unique and powerful features such as big data viz via DataShader, cross filtering via HoloViews, Streaming and much more.

It ties into the PyData and Jupyter Ecosystem as users can develop in notebooks and use ipywidgets. They can also develop in .py files.

This is not possible with Flask, Django, FastApi, Gradio, Dash or Streamlit.

Panel is one of the 4 most popular data app frameworks in Python with more than 400.000 downloads a month. It's especially popular in the scientific community.

Panel is used by for example Rapids to power CuxFilter, a CuDF based big data viz framework.

An added benefit of adding a Panel Frontend is that afterwards supporting a BokehFrontend is almost copy paste.

Pitch

I can do the implementation and documentation, but would need help reviewing and similar.

I can also promote the new opportunity to the Panel community.

Alternatives

None.

Additional context

Coming soon

Status

• Explore by creating example repo and get it deployed
  • Repository, Example App
• [] Contribute Panel App from Scratch example documentation similar to Dash App from Scratch
• [] Contribute PanelFrontend similar to StreamlitFrontend
• [] Contribute some examples to the Lightning.AI Gallery
• [] Write blog post about the panel+lightning.ai opportunity.
• [] Add section to the Panel documentation about lightning.ai as a deployment option.

---

cc @Borda

[williamFalcon]

@MarcSkovMadsen
love this idea! go for it. We’ll help you land the PR once you are ready. In terms of promoting, we can put together an app or two to share with both communities.

nice idea!

[lantiga]

@MarcSkovMadsen that’s great! Let us know if you need directions with designing this.

For awareness, at a minimum you can already run a Panel application in a LightningWork and expose it through the tab frontend. This is a good example to take inspiration from: https://lightning.ai/lightning-docs/workflows/add_web_ui/dash/basic.html

The more advanced integration is subclassing the Frontend ABC https://github.com/Lightning-AI/lightning-app/blob/f2e9628f9a509fcd4516ba7f2c9d6c68e2e880ca/lightning_app/frontend/frontend.py#L8
This gives users more flexibility to interact with the state of a flow + work tree. In comparison, the previous option will only be able to access the state of the Panel Work itself.

Just a bit of orientation here, just feel free to explore, looking forward!

[MarcSkovMadsen]

Thanks @lantiga

I will then start by making a PR for Panel similar to https://lightning.ai/lightning-docs/workflows/add_web_ui/dash/basic.html.

Run app Locally

lightning-ai.mp4

Create a file called requirements.txt containing

panel

Create and activate a virtual environment. Run

pip install lightning
pip install -r requirements.txt`

Create a file called app.py

# Inspiration: https://github.com/Lightning-AI/lightning/blob/master/docs/source-app/workflows/build_lightning_app/from_scratch.rst
from __future__ import annotations

from typing import Callable, Dict

import lightning_app as lapp
import panel as pn

ACCENT = "#792EE5"

def page1():
        pn.extension(sizing_mode="stretch_width")
    
        info = """
        **[Panel](https://panel.holoviz.org/)** and the [HoloViz](https://holoviz.org/) ecosystem provides unique and powerful features such as big data viz via [DataShader](https://datashader.org/), easy cross filtering via [HoloViews](https://holoviews.org/), streaming and much more.

        Panel works with the tools you know and love ❤️. Panel ties into the PyData and Jupyter ecosystems as you can develop in notebooks and use ipywidgets. You can also develop in .py files.

        Panel is one of the 4 most popular data app frameworks in Python with [more than 400.000 downloads a month](https://pyviz.org/tools.html#dashboarding). It's especially popular in the scientific community.

        Panel is used by for example Rapids to power [CuxFilter](https://github.com/rapidsai/cuxfilter), a CuDF based big data viz framework.
        
        Panel can be deployed on your favorite server or cloud including **[lightning.ai](https://lightning.ai)**.
        """
        
        def lightning_string(value):
            return "# " + "⚡"*value

        lightning = pn.widgets.IntSlider(value=5, start=0, end=10, name="⚡")
        ilightning_string=pn.bind(lightning_string, value=lightning)
        
        component = pn.Column(info, lightning, ilightning_string,)

        template = pn.template.FastListTemplate(
            title="Panel - Introduction", accent_base_color=ACCENT, header_background=ACCENT,
            main = [component])
        return template

def page2():
        template = pn.template.FastListTemplate(title="Hello", main=[pn.pane.Markdown("hello world 2")])
        template.param.update(accent_base_color=ACCENT, header_background=ACCENT)
        return template

class LitPanelPage(lapp.LightningWork):
    def __init__(self, page: Callable[[], pn.viewable.Viewable], parallel=True, **params):
        """_summary_

        Args:
            page (Callable[[], pn.viewable.Viewable]): A function returning a Panel Viewable
            parallel (bool, optional): Whether or not to run in parallel. Defaults to True.
        """
        self._page = page
        
        super().__init__(parallel=parallel, **params)

    def run(self):
        pn.serve({"/": self._page}, port=self.port, address=self.host, websocket_origin="*", show=False)

    def get_tab(self, name: str) -> Dict[str, str | 'LitPanelPage']:
        """Returns a *tab* definition to be included in the layout of a LightingFlow"""
        return {"name": name, "content": self}

class LitApp(lapp.LightningFlow):
    def __init__(self):
        super().__init__()
        self.lit_panel1 = LitPanelPage(page=page1)
        self.lit_panel2 = LitPanelPage(page=page2)

    def run(self):
        self.lit_panel1.run()
        self.lit_panel2.run()

    def configure_layout(self):
        return [
            self.lit_panel1.get_tab(name="Introduction"), 
            self.lit_panel2.get_tab(name="Big Data Viz"),
            self.lit_panel2.get_tab(name="Crossfiltering"),
            self.lit_panel2.get_tab(name="Streaming"),
            ]

if __name__=="__main__":
    app = lapp.LightningApp(LitApp())
    # lightning run app app.py
elif __name__.startswith("bokeh"):
    page1().servable()
    # panel serve app.py --autoreload --show

Run

lightning run app app.py

Run app in the cloud - lightning.ai is blacklisted

In theory it should be as simple as

lightning run app app.py --cloud

Unfortunately I am on a forced VPN where lightning.ai is blocked so I cannot login and run the app.

I will try to get the url whitelisted.

If any one can run the app on the cloud and share a link I would be able to view it on my mobile phone. That would be nice.

[MarcSkovMadsen]

Open Questions

• I set websocket_origin="*". This is to allow connections from for example 127.0.0.1 or lightning.ai. Is it possible to be more specific and determine the origin from some method or attribute on lapp.LightningWork?

[lantiga]

Hi @MarcSkovMadsen, this is great!

Open Questions

• I set websocket_origin="*". This is to allow connections from for example 127.0.0.1 or lightning.ai. Is it possible to be more specific and determine the origin from some method or attribute on lapp.LightningWork?

Great question. You can access the origin host using a constant the lightning_app package exposes:

from lightning_app.core import constants
constants.APP_SERVER_HOST

I'm going to run the app now and will share the link with you.

[lantiga]

@MarcSkovMadsen https://01g63kmn3cp6mba0hp0zaqwgvb.litng-ai-03.litng.ai/view/Introduction

[philippjfr]

What would be the best way for me to help out here? I really would love to see official support here and am happy to contribute to make that happen.

[MarcSkovMadsen]

Hi @philippjfr

Seen from the Panel perspective you could 1) Check out the status list I've added to the initial post 2) Explore the repository https://github.com/marcskovmadsen/awesome-panel-lightning to get a basic understanding and try to deploy it 3) Try to deploy your own app.

Think about whether we can/ should solve the "slow" first response on the Panel side. Right now I use pn.state.onload to speed up the first initial request. If its not fast the lightning.ai server sends a new request and continues like that for ever.

It would be very helpful for me to get suggestion and reviews/ feedback on the items in the status list above.

[williamFalcon]

@lantiga @tchaton can we help land this? very excited to enable this integration @philippjfr