##Introduction

###What is Shiny for Python?

Announced at rstudio::conf 2022, this is a Python port of Shiny, the dashboard creation framework using R language. Those who have used Shiny before must have been surprised 😲

As of August 2022, it's still an α version that was just released, so there's no guarantee of operation and I barely see any usage reports.

This time, I managed to write code and successfully deploy to shinyapps.io, so in this article I'll introduce that entire process. At the end, I'll also compare it with Streamlit and R Shiny 💡

Commands like shiny-python and rsconnect-python may change in the future, so make sure to check the latest information when trying them out ⚠️

###Differences from Other Dashboard Frameworks

Shiny is a web framework developed by none other than Rstudio, which enables creating interactive web pages (Apps) using R language. It has a fairly long history - first appearing in 2012, and by around 2015 add-ins like flexdashboard were already created, enabling almost the same development experience as today.

There are constraints like R needing to run on the backend, and the deployment destinations for created Apps are limited to places like shinyapps.io and ShinyServer, but at the time it was praised as a fairly advanced framework.

However, nowadays Python frameworks like Jupyter Notebook sharing through Google Colab, Plotly's Dash, and Streamlit are more popular than Shiny, and I myself use internal tools built with Streamlit. Many of you might wonder what significance there is in porting Shiny, which doesn't attract as much attention as before, to Python at this timing 🤔

###Benefits of Using Shiny for Python

The benefits of Shiny for Python might include:

• 📃 Speed improvements through WebAssembly
• 🐍 Can leverage existing Python libraries
  • Might be better than ShinyR when utilizing libraries that don't exist in R ⭕
• 📉 Detailed customization is possible
  • Shiny can use arbitrary CSS, JavaScript, and HTML, so it can somewhat compete with web frameworks like Django
  • However, CSS tends to break easily and requires more effort to build from scratch ❌
  • ※Streamlit can also use most CSS, JavaScript, and HTML, but not as freely as Shiny
• 🚄 Can create dynamic web pages
  • Can accept text and button input and dynamically execute Python code, so development cost is much lower than Django

I personally think the biggest benefit is the speed improvement through WebAssembly (WASM) compilation, but it doesn't seem to be particularly marketed that way.[ref]https://www.rstudio.com/conference/2022/keynotes/past-future-shiny/ It's revealed in a talk with Hadley.[/ref]

##Comparison Project: Streamlit × ShinyR × PyShiny

This time I personally compared Streamlit, ShinyR, and Shiny for Python to find my optimal solution for creating web apps.

So I decided to clone parts of the Ministry of Health, Labour and Welfare's "COVID-19 Information from Data", which ① includes typical graph creation, ② has dynamic App functionality, and ③ is catchy content.

The production policy was to fully utilize each framework's add-ins while creating as beautiful a page as easily as possible.

https://covid19.mhlw.go.jp/extensions/public/index.html

###Results

I'll leave the pros and cons for a future article. For now, here's what I created:

Streamlit

https://snitch0-mhlw-clone-streamlit-app-jaculn.streamlitapp.com/

I made this first. Development took about three days including weekends (about 5 hours).

Since I had been operating Streamlit apps at work recently, I was able to enjoy developing as usual.

ShinyR

https://snitch.shinyapps.io/dashboard/#section-%E3%83%AC%E3%83%99%E3%83%AB%E3%81%AE%E5%88%A4%E6%96%AD%E3%81%A7%E5%8F%82%E8%80%83%E3%81%A8%E3%81%95%E3%82%8C%E3%82%8B%E6%8C%87%E6%A8%99%E9%96%A2%E9%80%A3%E3%83%87%E3%83%BC%E3%82%BF

After Streamlit, I made it with ShinyR.

I had made some simple tools with Shiny before, but I had the preconception that creating IN/OUT mechanisms in ShinyR was troublesome, and the vanilla appearance wasn't very beautiful. So this time I used the flexdashboard package. Thanks to that, development went smoother than I expected.

Development took 1 week (about 10 hours), with most of the time spent dealing with layout collapse issues and deployment problems. These issues were stressful, but overall I think development went smoothly.

Shiny for Python

https://snitch.shinyapps.io/ipyshiny\_covid1/

This is the main topic - the Python version of Shiny that I'm introducing was made last.

Since Shiny for Python is still in α version, there are virtually no convenient third-party libraries, so the appearance was unfortunately plain. I supplemented it with a bit of custom CSS.

From here I'd like to report on how to use Shiny for Python. First, a note of caution: errors due to Python and library version dependencies occur easily. Make sure to use virtual environments like conda or venv for version control.

For conda, I recommend miniforge. You can find many resources on how to use venv by searching.

##Environment Setup

This article introduces an example using Ubuntu 20.04 LTS (WSL2). It should probably work on MacOS too, but I couldn't get the rsconnect command to work properly in a Windows conda environment. ※Except for rsconnect used for deployment, I think everything else works fine on Windows. Probably.

Also, I'll proceed assuming miniconda is installed. Since we only use pip, python venv or pipenv should work fine too.

Creating conda env

First, create the environment. For some reason deployment failed just from different Python versions, so make sure to specify versions properly.

mamba create -n shinyenv python=3.9.13

Unrelated to the main topic, but I still regularly see comments on Twitter saying "conda corrupts environments so it's bad" - but I don't think you need to worry that much. Python libraries installed via conda have been improved not to conflict with pip installations, so I rarely see environment corruption recently. Also, using the mamba command super-speeds up installation, making quick environment setup possible.

Installing Required Libraries

Next, prepare the required Python libraries.

Make sure the pip you're using is from the environment you just created.

conda activate shinyenv
which pip3
# ~/mambaforge/envs/shinyenv/bin/pip3

pip install shiny shinywidgets htmltools jupyter rsconnect-python

The following are libraries used for graph creation etc.

pip install pandas numpy altair vega

VSCode Development Environment

This is optional.

Any Python development environment works, but this time I'm using VSCode to create the App. There's a convenient extension available, so install it.

https://marketplace.visualstudio.com/items?itemName=rstudio.pyshiny

Also, it's good to set up linters etc. as per the official guide.

Testing

The environment should be properly set up now. Let's try making something and running it.

The shiny command has an API to generate an empty project, let's try it.

$ shiny create testapp
Created Shiny app at testapp/app.py

A very simple app.py was generated. The content is exactly the same as the Shiny Examples example.

https://shinylive.io/py/editor/#code=NobwRAdghgtgpmAXGKAHVA6VBPMAaMAYwHsIAXOcpMAMwCdiYACAZwAsBLCbJjmVYnTJMAgujxM6lACZw6EgK4cAOhFVpUAfSVMAvEyVYoAcziaaAGyXSAFKqYODHDF1QKymlhY6y6dyMr4TIEAcoESAAwSAIwRUUwATBEAlHj2jobE7m4eFAAeHgBucgBGUGR8-mQFgamqyaqNELI0rHLFfq7uEllkORIscCwsHKTJiOkOAAK9OZNMU1LNchj5ZPMtTNVkNuPzjpJwZAp0EEw0gRAAVAm8LEwgXWQYELtMV4kAvoFN6uh6onQNg02g4A3acgaEDAnwAukA

You can easily verify operation just by looking at the preview on the Example page, but you can get the same results using the VSCode extension.

The App starts on port 8000 with logs like below. Using the VSCode extension, the App launches inside VSCode using Simple Browser, which is very convenient.

> /home/snitch/mambaforge/envs/shinyenv/bin/python -m shiny run --port 8000 --reload "/home/snitch/R/ipyshiny_test/testapp/app.py"
INFO:     Will watch for changes in these directories: ['/home/snitch/R/ipyshiny_test/testapp']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [2233] using StatReload
INFO:     Started server process [2242]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:51224 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /?vscodeBrowserReqId=1660601926014 HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /require.min.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /bootstrap.min.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /shiny.min.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /css/ion.rangeSlider.css HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /shiny.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /jquery-3.6.0.min.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /bootstrap.bundle.min.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /js/ion.rangeSlider.min.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /strftime-min.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:51226 - "GET /shiny-autoreload.js HTTP/1.1" 200 OK
INFO:     ('127.0.0.1', 51240) - "WebSocket /websocket/" [accepted]
INFO:     connection open

##How to Create the COVID Dashboard

Next, let me explain briefly about the COVID dashboard I created 💡

Code is here. Deployment is here.

###Basic Syntax of Shiny for Python

Those who have used the R version of Shiny know that Shiny consists of two parts: UI part and server part. HTML and data input widgets go in the UI part, and computation processing like data frame transformation and graph creation is written in the server part.

The UI side is an object containing only multiple Tag class objects, while the server side is composed of free function definitions allowing free processing.

When you want to exchange data between UI and server - for example, to display a data frame called iris on the server side in the UI - you use the @render.table decorator to render the table and the @output decorator to put it on the I/O stream.

I'm impressed that this notation really tried hard to match the ShinyR writing style. It's great that using decorators results in a very clean API 👍 However, not being able to pass objects as objects feels un-Pythonic and hard to understand. ShinyR also passes things similarly, so maybe they're intentionally aligning the notation.

###UI Part

The main features implemented in the UI part are:

• Create 2 pages split by tabs
• Display cards like "New Cases" at the top of the page
• Create prefecture dropdown menu and read input
• Dynamic graph that reads "Graph Display Period"

Tab Pages

For some reason only this feature has a rich implementation 🙂

ui.navset_pill(), ui.navset_pill_list(), ui.navset_pill_card() etc. are available, and the official documentation is fairly comprehensive.

https://shiny.rstudio.com/py/docs/ui-page-layouts.html

This time I used the header option to display prefecture selection at the top like this:

app_ui = ui.page_fluid(
    ui.navset_pill(
        ui.nav(
            ui.markdown("## Page 1")
            # widget group
        ),
        ui.nav(
            ui.markdown("## Page 2")
            # widget group
        ),
        header=ui.input_select(
            id="
            prefecture",
            label="You can view by prefecture.",
            choices=list(pref.keys()),
        ),
    ),
)

What I found interesting about the Python Shiny implementation is that UI components are composed of tuples of Tag class objects. In Python, trailing commas work fine for tuple definition, so I thought it enables web app creation with somewhat the same feel as JavaScript.

Creating Metrics Boxes with Custom CSS

Vanilla Shiny, whether R or Python, doesn't provide high-quality UI. Streamlit can create beautiful UI without doing anything, and R Shiny can do roughly the same with extensions like {flexdashboard}.

However, Python Shiny is still in α version with no plugins at all. It's disadvantaged so I decided to fill the gap with custom CSS.

That said, I just borrowed the CSS from the Ministry of Health, Labour and Welfare widget and adjusted the details.

Save custom CSS as "style.css" and load it in Shiny. Keep all files under root.

 .
├──  app.py
├──  metrics_box.py
├──  plot_figure.py
├──  plot_func.py
├──  prefecture_dictionary.py
└──  style.css

Then just input the CSS content directly into the shiny.ui.tags.style() method. I used pathlib.Path.read_text() as a one-liner.

from htmltools import head_content
 
 
app_ui = ui.page_fluid(
    # head
    head_content(
        ui.tags.meta(charset="UTF-8"),
        ui.tags.style(
            (Path(__file__).parent / "style.css").read_text()
                      ),
        ui.tags.link(rel="stylesheet",
                     href="https://cdnjs.cloudflare.com/ajax/libs/normalize/5.0.0/normalize.min.css"),
        ui.tags.link(rel="stylesheet",
                     href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css"),
    ),
)

There's also code for loading meta tags and external CSS, but it's the same as regular HTML pages. I referenced the official gallery "wordle" for this writing style.

Now I want to create and display content using CSS classes, but since I need to accept input from the "Prefecture Dropdown," I need to display values interactively. Interactive values are defined in the server part explained later; here I describe the HTML tag content.

from shiny.ui import p, span, div, br
 
 
def metrics_card_item(str_title: str, num_main: int, num_sub=0):
 
    num_main_str = f'{num_main:,}'
    arrow_char = "⬆︎" if num_sub > 0 else "⬇"
 
    if num_sub:
        num_sub_str = f'{num_sub:,}'
        card = div(
            div(
                p(str_title),
                p(
                    span(num_main_str, class_="col4-pattern1_num",
                         id="curSituNewCaseKPI"),
                    span("people", class_="fontSize3"),
                    br(),
                    span("vs previous day", class_="fontSize4"),
                    span(arrow_char,
                         class_="fontSize8",
                         id="curSituNewCaseArw",
                         style="color: rgb(204, 0, 0)"),
                    span(num_sub_str, class_="fontSize6", id="curSituNewCaseDB"),
                    span("people", class_="fontSize7")
                ),
                class_="col4-pattern1_sub"
            ), class_="col4-pattern1_item"
        )

Like this, you use methods like p, span, div defined in shiny.ui. You specify HTML attributes like id=****, but since class is a reserved word in Python for class definitions, class_ argument is provided instead.

Being able to create free HTML like this expands creative possibilities ♪

Input Widgets

This time I'm getting ① prefecture input and ② graph display period input from user operations.

For ① prefecture, I used ui.input_select() and tried using Python dictionaries for data matching.

def create_pref_dict():
    en_list = [
        "ALL",
        ...,
        "kumamoto", "oita", "miyazaki", "kagoshima", "okinawa"
    ]
    en_c_list = [s.capitalize() if s.islower() else s for s in en_list]
    ja_list = [
        "All",
        ...,
        "Kumamoto", "Oita", "Miyazaki", "Kagoshima", "Okinawa"
    ]
    pref_dict = {key:[val1, val2] for key, val1, val2 in zip(
        ja_list, en_c_list, range(len(ja_list)))}
 
    return pref_dict

Using dictionary comprehension like above, you can generate a dictionary object like below, making data conversion and reference easy:

>>> pref = create_pref_dict()
>>> pref["Kumamoto"]
['Kumamoto', 43]

So the implementation for prefecture input and radio button input for graph period specification looks like this:

pref = create_pref_dict()
 
app_ui = ui.page_fluid(
    ui.input_select(
        id="prefecture",
        label="You can view by prefecture.",
        choices=list(pref.keys()),
    ),
    ui.input_radio_buttons(
        "rb1",
        "Graph Display Period",
        {
            "week": "1 week",
            "month": "1 month",
            "3months": "3 months",
            "year": "1 year"
        },
        selected="year"
    )
)
 
def server(input, output, session):
    @output
    @render.plot
    def my_ploy():
        plot = plot_func.plot_line_cases(
            url="https://covid19.mhlw.go.jp/public/opendata/newly_confirmed_cases_daily.csv",
            prefec=pref[input.prefecture()][0],
            period=input.rb1()
        )

In the UI part, you specify object names as str type (prefecture and rb1 in the example above), while in the server part they're called as methods of input. This feels like magic and isn't very Pythonic, but I think it's designed to be an easy-to-use API.

As a side note, surprisingly this notation passes mypy checks without problems. Much of the important processing must be hidden in the decorators 🤔

Displaying Graphs

The graph display method is almost the same as R Shiny. Just output the plot created on the server side.

However, the ui.render_plot() method implemented in the shiny library only accepts matplotlib Plot objects, so you can only use matplotlib or seaborn library plots 😥

So I used the py-shinywidgets library. With this, you can use all major interactive plotting frameworks like plotly, altair, and vega.

from shinywidgets import output_widget, render_widget
 
import plot_figure
 
 
app_ui = ui.page_fluid(
    ui.columns(
        output_widget(plot1_1)
    )
)
 
def server(input, output, session):
    @output
    @render_widget
    def plot1_1():
        return plot_figure.plot_new_cases(
            url="https://covid19.mhlw.go.jp/public/opendata/newly_confirmed_cases_daily.csv",
            plot_range=input.rb1(),
            ytick_space=50000,
            color="#fd6262",
            prefecture=pref[input.prefecture()][0]
        )

I'll explain the plot function definitions later, but they're defined in a separate file.

###Server Part

Here we receive values from input, create altair graphs, and pass graphs to the UI part through the output stream.

Data is received as methods of input class like input.prefecture(), but output automatically passes function return values by defining functions decorated with decorators.

This time I tried using altair graphs, which I hadn't used before.

I had used plotly many times before, but I somehow felt bokeh/vega systems are more popular lately, and I wanted to try altair, which is famous as a vega-based framework. After using it a bit, the documentation is a bit hard to understand, but I found it easy to use and quite liked it.

I liked it enough to think I'll adopt altair when making interactive plots for work.

I realized later that for simple input like "period specification," it could be completed within bokeh or altair alone without borrowing Shiny's power. I'll try that sometime 🦾

The plot I created uses pandas.DataFrame and is written with altair. I'll skip the explanation since it's not the main topic.

def plot_new_cases(plot_range: str, url: str, ytick_space: int, color: str,
                   prefecture: str):
    df = pd.read_csv(url)
    df["Date"] = pd.to_datetime(df["Date"])
    df["col"] = color
 
    chart = alt.Chart(filter_df_with_daterange(df, plot_range)).mark_area(
    ).encode(
        alt.Y(prefecture, axis=alt.Axis(
            values=[i*ytick_space for i in range(1, 6, 1)])),
        x="Date",
        color=alt.Color("col", scale=None)
    )
 
    return chart

##Impressions After Creating

So this time I tried out Shiny for Python which was just α-released. I haven't seen anyone else writing about trying it yet, so I hope this helps someone who's going to try it 🖐️

Although simple, since I made one coherent thing, I'd like to share my impressions.

###If You're Already Using R Shiny, This Might Be a Promising New Star

The big difference between Python Shiny and R version is that it's compiled to WebAssembly (WASM)[ref]In Shiny's presentation (53:45) Joe Cheng reveals this while confirming "Am I allowed to say this?"[/ref]. Using WASM makes applications faster and lighter, which is definitely a big advantage.

When Pyscript was announced it was also a hot topic - the web app industry around WASM needs to be watched going forward 🧐

Also, needless to say, Python has a vast amount of libraries as assets. Being able to utilize convenient libraries that exist in Python but not in R is a great strength.

###Honestly, Shiny Development Is Painful

This is my personal impression after trying it, but developing with Shiny wasn't very fun, whether R or Python version 😅

The reasons are:

• CSS behavior doesn't work as expected
  • ⇒ It breaks as soon as you put it in ui.tags(), and the class inheritance relationships are unclear
• Seems to use remaining cache improperly, so tests sometimes run states that shouldn't work
  • ⇒ Layout creation using columns and rows has low flexibility
  • ⇒ Might be solved with frameworks like Shinytest2
• Creating nice-looking widgets is difficult
  • ⇒ The amount of code needed to create cool UI inevitably becomes large

The first two are bug-like reasons so there's room for improvement, but I was frustrated that the situation hadn't changed much from when I used R Shiny about five years ago 💢

However, the VSCode extension is wonderful despite being simple. I wish there was an extension for Streamlit that opens in VSCode's simple browser too 🤔 [ref]But if that existed there'd really be no reason to use Shiny...[/ref][ref]I thought that but someone made one yesterday. https://discuss.streamlit.io/t/release-vscode-extension-python-string-markdown-for-streamlit/3378\[/ref\]

###For People Like Me, Streamlit Is Recommended

I don't have much web skills and don't plan to develop them[ref]I'm interested in TypeScript so I want to study it, but priority-wise I think Rust, C++ > Zig >> ts. Being a non-engineer leaves so much left to learn... 😉[/ref], so I'm resistant to frameworks that require some HTML/CSS/JS skill stack.

If you're like me and want to "focus on graph and calculation app creation without spending time on WebUI," Streamlit is overwhelmingly recommended.

Even with zero HTML knowledge, you should be able to master almost all widget and UI layout features. (Of course, markdown rendering and HTML tag insertion are also possible)

Also, Streamlit has rich input/output widgets from the start. Just combining existing ones should accomplish most of what you want to do. Community-made widgets are officially adopted and Python libraries are shared on the forum, so you should never be stuck for widgets.

##Rough Summary

• WASM is really interesting. High expectations for future developments.
• From my perspective, I currently see no reason to choose R/PyShiny over Streamlit.

##Reference Pages

Official documentation

Can't say it's comprehensive, but you can learn Shiny for Python in general.

https://shiny.rstudio.com/py/

API Reference

https://shiny.rstudio.com/py/api/

How to deploy to shinyapps.io

https://docs.rstudio.com/shinyapps.io/getting-started.html#working-with-shiny-for-python

py-shiny repository

https://github.com/rstudio/py-shiny/

rsconnect-python repository

https://github.com/rstudio/rsconnect-python

py-shinywidgets repository

https://github.com/rstudio/py-shinywidgets