Shiny Express: Advanced topics

Note

This document was written and updated during the development of Shiny Express, and some parts of it may now be out of date.

Please see the official documentation for Shiny Express.

Shiny Express is built on top of “Core” Shiny – everything that you can do with Shiny Express you can also do in Shiny Core. The reverse isn’t exactly true – there are times where you’ll hit the limits of Shiny Express and need to use Shiny Core.

This document explains some of the technical details behind how Shiny Express works. Most Shiny Express users don’t need to read this document, but if you want to understand how it works under the hood, or want to do something more sophisticated with Shiny Express, you might find this document useful.

Note

This page is still a work in progress, and some sections are not yet complete!

Special evaluation of Shiny Express app code

In a Shiny Core app, when you run shiny run app.py from the command line, it effectively runs the Python code in app.py. In that code, you must create an object named app, which is a Shiny.App object. That object also conforms to the ASGI application specification, which means that it is a web application that can be run by an ASGI web server. For Shiny, we use uvicorn as the web server.

In a Shiny Express app, you do not explicitly create an object named app; you actually do not create a Shiny.App object anywhere. That is done automatically for you, when Shiny detects that it is a Shiny Express app.

How does Shiny detect when the app is an Express app? When you call shiny run app.py, it parses the code and looks for an import statement like these at the top level of the code. If it contains one of these, then Shiny knows that it’s an Express app, and evaluates the code in a special way, which we’ll talk about more below.

# Any of these imports indicates that it is a Shiny Express app
import shiny.express
from shiny import express
from shiny.express import input

Next: - special evaluation to capture side effects - create app object

Special evaluation of Express app code: UI as display

As mentioned above, the code of an Express app is evaluated in a special way. This is needed because the code needs to be evaluated in a way that is similar to Jupyter notebooks: in a Jupyter notebook, the results of most top-level statements are rendered in the notebook. Jupyter does this by evaluating each statement and calling IPython.display.display() on the return value. In Shiny Express, we need to do something similar.

Before diving into Shiny Express, let’s talk about how Shiny Core works. In Shiny Core, you create UI by calling pure functions that return objects. Containership is expressed by passing children as arguments into the parent when the latter is being created. Eventually, the top-level parent is passed to Shiny via the App object (or returned from a @render.ui).

# Shiny Core
app_ui = ui.page_fixed(
    widget1(),
    widget2(),
    widget3(),
)

app = App(app_ui, ...)

In Shiny Express, top-level statements are evaluated in order, and any non-None expression has something like Jupyter’s display() called on it. This is intended to feel similar to a code chunk in .rmd or .qmd, or like a Jupyter notebook (with InteractiveShell.ast_node_interactivity="all").

# Shiny Express
widget1()       # Displayed
w2 = widget2()  # Not displayed
w2              # Now it's displayed
widget3()       # Displayed

Shiny Express does not actually call IPython.display.display() on each statement. Instead, it collects the result of each statement.

There’s an inherent tension in these decisions. What seems to be the case is that from a superficial level, imperative UI feels more intuitive than functional UI–that is, calling button() should immediately “output” a button at the moment it’s called, rather than returning some kind of value. And indeed, there are UI frameworks that take this approach, including Streamlit.

But imperative UI starts to run into trouble very quickly, because programming against side-effects is harder than programming against values. For example, consider a function that takes a title as an argument, which could either be a string or an HTML element. How would you pass an HTML element if the very act of constructing an <h1> causes it to be emitted as output? Or think about a function that takes a while to generate some UI; if UI is just returned objects, you can trivially cache the results in any number of naturally Pythonic ways, but if it’s side effects, you have to figure out some way of intercepting those side effects, and cache them using some custom mechanism.

The display() approach of Jupyter/Quarto/Express serves as a compromise between Shiny Core’s functional approach and Streamlit’s imperative approach. The code looks very similar to imperative code, but we’re able to re-use most of our UI components even though they use the pure functional style. (Existing container components cannot be used in Express, as explained in the next section.)

One special challenge is dealing with expressions that are not at the top-level; that is, in functions. If you define a function in a Quarto code chunk or Jupyter notebook cell, its expressions will not be displayed/printed. The same is true in Shiny Express. However, it’s clearly desirable to be able to write functions that express UI, and it’d be great to do it without having to explicitly call sys.displayhook() on every piece of UI. The @expressify decorator does this by transforming a function’s body to behave like the top level (and see also @render.express, which is sort of like @render.ui plus @expressify combined).

Express advantages:

Same/similar paradigm as Quarto chunk or Jupyter notebook cell.
Fewer commas/parentheses.

Core advantages:

Easier refactoring of complicated UI (extracting into variables, parameterized functions, caching, etc.).
Can write apps in Python modules/packages.

Containers as context managers

The prior two differences make container components a special challenge for Shiny Express. Shiny Core’s UI components are passed as function arguments, but Shiny Express’s render functions make this approach untenable; Python won’t let you declare a decorated function and pass it as a function argument at the same time.

As a result, you can’t use Shiny Core’s container UI functions with Shiny Express. Instead, we’ve created a Shiny Express port for each of our container functions.

Core syntax:

# Shiny Core
app_ui = ui.page_fixed(
    ui.card(
        ui.input_slider(),
        ui.output_plot("plot1"),
    )
)

def server(input, output, session):
    @render.plot
    def plot1():
        ...

app = App(app_ui, server)

Express syntax (notice that we don’t need to call page_fixed() because it is the default page type for Express):

# Shiny Express
from shiny.express import layout

with layout.card():
    ui.input_slider()

    @render.plot
    def plot1():
        ...

These new shiny.express.layout container functions do not return objects, but rather, Python context managers that are side-effecty (they intercept sys.displayhook while inside the with block, and display() themselves upon exit).

Express advantages:

Makes it possible to put render functions inside UI containers.
Fewer commas/parentheses.

Core advantages:

As before, easier refactoring of complicated UI (extracting into variables, parameterized functions, caching, etc.).
Container function signatures make it easier to see what arguments are allowed/required.
No difference between child and container UI functions, in terms of: how you call them, how you handle their results, and how you write your own.

`render` functions automatically create outputs

In the Core version of the previous example app, the server function contains a @render.plot; def plot1(), and the UI contains a corresponding output, ui.output_plot("plot1"). The output component is how Shiny Core knows where to put the plot on the page.

# Shiny Core
app_ui = ui.page_fixed(
    ui.card(
        ui.input_slider(),
        ui.output_plot("plot1"),
    )
)

def server(input, output, session):
    @render.plot
    def plot1():
        ...

app = App(app_ui, server)

In an Express app, you don’t need to explicitly call ui.output_plot("plot1"). Instead, when you call @render.plot; def plot1(), it automatically puts the corresponding output at that location:

# Shiny Express
from shiny.express import layout

with layout.card():
    ui.input_slider()

    @render.plot
    def plot1():
        ...

Preventing display of objects with `ui.hold()`

If for some reason, you want to create an object but not render it into the page at that location, one way to do it is to simply assign it to a variable:

x = widget()

Just like in a Jupyter notebook, assigning the value to a variable prevents it from being displayed in the Shiny app. You can later put x on a line by itself to display it there.

However, there are cases where you can’t do this with Python. If you have a @render.plot, but don’t want it to put an output right there, you can’t assign it to a variable with x = .... This simply is not valid Python code:

# NOT valid code
x = @render.plot
def _():
    ...

For cases like this, you can use with ui.hold():

with ui.hold():
    @render.plot
    def plot1():
        ...

You could also put more render functions in that same code block if you wanted.

You would have to add the corresponding output explicitly somewhere in your app:

ui.output_plot("plot1", click=True)

This can be useful when you want to add an output_plot with options that aren’t available in the render.plot() function. (Although note that in a future version of Shiny, we plan to make all of those options, like click, available in the render.plot() function.)

You can also use with ui.hold() as x, and then place x later in the page.

# Create a card here...
with ui.hold() as hello_card:
    with ui.card():
        with ui.span():
            "This is a"
            ui.span(" card", style="color: red;")


ui.h3("Some content")

# and display the card here
hello_card

`@render.text()` for plain text, `@render.code()` for code

When @render.text() is used, it defaults to displaying as plain text, and when @render.code() is used, it defaults to displaying in a monospaced font in a code block.

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 300
from shiny.express import render, ui

@render.text
def txt():
    return "This is @render.text"

ui.br()

@render.code
def code():
    return "This is @render.code"

Using Express syntax in functions with `@expressify`

If you want to write a function using Express syntax (as opposed to Core syntax), you can use @expressify. In the example below, you can compare the syntax to the version that’s written with Shiny Core syntax.

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 300
from shiny.express import expressify, ui

# The Shiny Express syntax version
@expressify
def card(i: int):
    with ui.card():
        with ui.span():
            "This is a card with @expressify: "
            ui.span(str(i), style="color: red;")

card(1)
card(2)


# The Shiny Core syntax version, with nested function calls
def card_core(i: int):
    from shiny import ui as sui

    return sui.card(
        sui.span(
            "This is a card with Shiny Core: ",
            sui.span(str(i), style="color: blue;"),
        )
    )


card_core(1)
card_core(2)

`@render.express()`

Code is run twice

Shared objects

In a Shiny Core app, each time a web browser visits the page, Shiny executes the server function once; any objects created inside of the server function (this includes reactive and regular non-reactive objects) are scoped to that user session.

Global objects – those that are created outside of the server function – are shared across all sessions. This is useful if there are slow operations that only need to be done once, like loading a large data set. Sharing large objects can also significantly reduce the memory load of the application.

# Shiny Core

# Global objects here are shared across all sessions (within a Python process)
df = pd.read_csv("big_data.csv")

def server(input, output, session):
    # Everything in this function is scoped to each user session

    db_conn = connect(...)

    @render.table
    def tbl():
        return df

app = App(app_ui, server)

In a Shiny Express app, all of the code in the app.py is run once per session. (There is actually a Shiny server function which evaluates the code within its scope, and like any Shiny server function, it is executed once per user session.) This means that code at the top level of the Shiny Express app file is not shared across sessions.

If you want to share objects across Shiny sessions, you can create a separate .py file and put the shared code in there, and then import that file into your app. For example, you could have a file called shared.py with the following:

shared.py

df = pd.read_csv("big_data.csv")

Then your app might look like this:

app.py

import shared

# This is scoped to the session
db_conn = connect(...)

@render.table
def tbl():
    return shared.df

(Note that you can’t name the file global.py because global is a reserved keyword in Python, and you can’t use import global in your app.)

Python’s module loading system is smart, and does not run the code in shared.py each time it is imported. The code in shared.py is run only once, on the first import. The objects in the shared module are then shared – if you modify an object in one Shiny session, that change will also be reflected in other Shiny sessions.

Shared reactive objects

The example above used a non-reactive object (a data frame) in shared.py. If you have any reactive objects that are in shared.py, then you must create them without an active session. As mentioned earlier, the code in shared.py is run on the first import. In Shiny Express, that first import can happen with an active user session, and that will confuse the reactive graph.

Special evaluation of Shiny Express app code

Special evaluation of Express app code: UI as display

Express advantages:

Core advantages:

Containers as context managers

Express advantages:

Core advantages:

render functions automatically create outputs

Preventing display of objects with ui.hold()

@render.text() for plain text, @render.code() for code

Using Express syntax in functions with @expressify

@render.express()

Code is run twice

Shared objects

Shared reactive objects

Entrypoint

`render` functions automatically create outputs

Preventing display of objects with `ui.hold()`

`@render.text()` for plain text, `@render.code()` for code

Using Express syntax in functions with `@expressify`

`@render.express()`