The ability to quickly develop and deploy interactive applications is invaluable. Streamlit is a powerful tool that enables data scientists and developers to create intuitive web apps with minimal code. Coupled with the Python Data Science Notebook Docker Image, which comes pre-loaded with essential data science libraries, setting up a robust environment for building Streamlit apps has never been easier.
What is Streamlit?
Streamlit is an open-source Python library that simplifies the process of creating interactive web applications for data science and machine learning projects. With Streamlit, you can transform your data scripts into shareable web apps in just a few minutes, all using pure Python. There’s no need for front-end development skills or knowledge of web frameworks like Flask or Django.
Key features of Streamlit include:
- Easy to Use: Build apps with a few lines of code using straightforward APIs.
- Interactive Widgets: Incorporate sliders, buttons, text inputs, and more to make your app interactive.
- Real-Time Updates: Automatically update app content when your data or code changes.
- Data Visualization: Seamlessly integrate with libraries like Matplotlib, Seaborn, Plotly, and Altair for rich visualizations.
- Deployment Ready: Deploy apps effortlessly on various platforms, including Streamlit Cloud, Heroku, and AWS.
Why Use Streamlit for Data Apps?
Streamlit offers several advantages that make it an ideal choice for developing data applications:
- Rapid Prototyping: Quickly turn ideas into functional apps without worrying about the underlying web infrastructure.
- Pythonic Syntax: Write apps entirely in Python, leveraging your existing skills without the need to learn HTML, CSS, or JavaScript.
- Interactive Data Exploration: Enable users to interact with data through widgets, making it easier to explore datasets and model results.
- Community and Support: Benefit from a growing community that contributes to a rich ecosystem of plugins and extensions.
- Open Source: Modify and extend the library to suit your needs, with the assurance of ongoing development and support.
By using Streamlit, data scientists can focus on data analysis and model building while providing stakeholders with interactive tools to visualize and understand the results.
Overview of the Python Data Science Notebook Docker Image
The Python Data Science Notebook Docker Image is a Docker container designed to streamline your data science workflow. Built from the minimal python:3.9-slim
base image, it includes a comprehensive suite of pre-installed libraries that cater to various aspects of data science, including data manipulation, machine learning, visualization, and database connectivity.
Key Features:
- Jupyter Notebook Access: Run and access Jupyter Notebooks through your web browser, facilitating an interactive coding environment.
- Pre-Installed Libraries:
- Data Manipulation:
pandas
,numpy
,polars
,dask
,ibis
,pyiceberg
,datafusion
,sqlframe
- Machine Learning:
scikit-learn
,tensorflow
,torch
,xgboost
,lightgbm
- Visualization:
matplotlib
,seaborn
,plotly
- Database Access:
psycopg2-binary
,mysqlclient
,sqlalchemy
,duckdb
,pyarrow
- Object Storage:
boto3
,s3fs
,minio
- Utilities:
openpyxl
,requests
,beautifulsoup4
,lxml
,pyspark
,dremio-simple-query
- Data Manipulation:
- User Configuration: Operates under the user
pydata
with the home directory set to/home/pydata
. The working directory is/home/pydata/work
. - Port Exposure: Exposes port
8888
to allow access to the Jupyter Notebook server.
Benefits:
- Consistency: Ensure a consistent development environment across different machines and team members.
- Isolation: Avoid conflicts with other projects and dependencies on your local machine.
- Portability: Easily move your development environment between systems or deploy it to a server.
- Extendable: Customize the Docker image by adding more libraries or configurations as needed.
By utilizing this Docker image, you can save time on setup and focus on developing your Streamlit applications, knowing that you have all the necessary tools and libraries at your disposal.
Setting Up the Environment
To get started with building Streamlit applications using the Python Data Science Notebook Docker Image, you’ll need to set up your environment. This involves installing Docker, pulling the Docker image, running the container, and verifying that Streamlit is installed and functioning correctly.
Installing Docker
If Docker is not already installed on your machine, follow these steps:
- Download Docker Desktop:
- Windows and macOS: Visit the Docker Desktop download page and download the installer for your operating system.
- Linux: Refer to the official Docker installation guides for Ubuntu, Debian, Fedora, or your specific distribution.
- Install Docker:
- Run the installer and follow the on-screen instructions.
- For Linux, follow the command-line instructions provided in the installation guide for your distribution.
- Verify the Installation:
Open a terminal or command prompt and run:
docker <span>--version</span>docker <span>--version</span>docker --version
Enter fullscreen mode Exit fullscreen mode
You should see the Docker version information displayed, confirming that Docker is installed.
Pulling the alexmerced/datanotebook
Docker Image
The alexmerced/datanotebook
Docker image includes a comprehensive Python environment with pre-installed data science libraries.
Pull the Docker Image:
In your terminal, execute:
docker pull alexmerced/datanotebookdocker pull alexmerced/datanotebookdocker pull alexmerced/datanotebook
Enter fullscreen mode Exit fullscreen mode
This command downloads the image from Docker Hub to your local machine.
Confirm the Image is Pulled:
List all Docker images on your system:
docker imagesdocker imagesdocker images
Enter fullscreen mode Exit fullscreen mode
You should see alexmerced/datanotebook listed among the images.
Running the Docker Container with Jupyter Notebook Access
Now, run a Docker container from the image and access the Jupyter Notebook server.
-
Navigate to Your Working Directory
-
Open a terminal and change to the directory where you want your Jupyter Notebooks and Streamlit apps to reside:
<span>cd</span> /path/to/your/project<span>cd</span> /path/to/your/projectcd /path/to/your/project
Enter fullscreen mode Exit fullscreen mode
Run the Docker Container:
Execute the following command:
docker run <span>-p</span> 8888:8888 <span>-p</span> 8501:8501 <span>-v</span> <span>$(</span><span>pwd</span><span>)</span>:/home/pydata/work alexmerced/datanotebookdocker run <span>-p</span> 8888:8888 <span>-p</span> 8501:8501 <span>-v</span> <span>$(</span><span>pwd</span><span>)</span>:/home/pydata/work alexmerced/datanotebookdocker run -p 8888:8888 -p 8501:8501 -v $(pwd):/home/pydata/work alexmerced/datanotebook
Enter fullscreen mode Exit fullscreen mode
- Port Mapping:
-p 8888:8888
maps the container’s port 8888 to your local machine, allowing access to Jupyter Notebook.-p 8501:8501
maps the container’s port 8501 to your local machine, allowing access to Streamlit apps. - Volume Mounting: –
v $(pwd):/home/pydata/work
mounts your current directory into the container, enabling file sharing between your host and the container. ### Access Jupyter Notebook:
Open your web browser and navigate to http://localhost:8888
.
You should see the Jupyter Notebook interface without needing a password or token.
Verifying the Installation of Streamlit within the Container
Ensure that Streamlit is installed and functioning properly inside the Docker container.
-
Open a New Terminal in Jupyter Notebook.
-
In the Jupyter interface, click on the New dropdown menu and select Terminal.
-
In the terminal, run:
streamlit <span>--version</span>streamlit <span>--version</span>streamlit --version
Enter fullscreen mode Exit fullscreen mode
If Streamlit is installed, the version number will be displayed.
If not installed, install it using:
pip <span>install </span>streamlitpip <span>install </span>streamlitpip install streamlit
Enter fullscreen mode Exit fullscreen mode
Create a Test Streamlit App:
In the Jupyter interface, click on New and select Text File.
Save the file as app.py in your working directory.
Add the following code to app.py:
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>Streamlit Test App</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Congratulations! Streamlit is working inside the Docker container.</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>Streamlit Test App</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Congratulations! Streamlit is working inside the Docker container.</span><span>"</span><span>)</span>import streamlit as st st.title("Streamlit Test App") st.write("Congratulations! Streamlit is working inside the Docker container.")
Enter fullscreen mode Exit fullscreen mode
Save the file.
Run the Streamlit App:
In the Jupyter terminal, execute:
streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0streamlit run app.py --server.enableCORS false --server.enableXsrfProtection false --server.port 8501 --server.address 0.0.0.0
Enter fullscreen mode Exit fullscreen mode
Server Flags Explained:
-
--server.enableCORS false:
Disables Cross-Origin Resource Sharing protection. -
--server.enableXsrfProtection false:
Disables Cross-Site Request Forgery protection. -
--server.port 8501:
Runs the app on port 8501. -
--server.address 0.0.0.0:
Makes the server accessible externally.
Access the Streamlit App:
Open a new tab in your web browser and navigate to http://localhost:8501.
You should see the Streamlit app displaying the title and message.
Optional: Keep Streamlit Running in the Background:
To keep the Streamlit app running without occupying the terminal, you can run it in the background using nohup:
<span>nohup </span>streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0 &<span>nohup </span>streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0 &nohup streamlit run app.py --server.enableCORS false --server.enableXsrfProtection false --server.port 8501 --server.address 0.0.0.0 &
Enter fullscreen mode Exit fullscreen mode
Exiting the Docker Container
In the Terminal Running the Container:
Press Ctrl + C
to stop the container.
Alternatively, Use Docker Commands:
- List running containers
docker psdocker psdocker ps
Enter fullscreen mode Exit fullscreen mode
Stop the container using its Container ID:
docker stop <container_id>docker stop <container_id>docker stop <container_id>
Enter fullscreen mode Exit fullscreen mode
Summary
You’ve successfully set up your environment:
- Installed Docker (if necessary).
- Pulled the alexmerced/datanotebook Docker image.
- Ran the Docker container with Jupyter Notebook access.
- Verified that Streamlit is installed and operational within the container.
With this setup, you’re ready to develop and run Streamlit applications in a consistent and isolated environment, leveraging the powerful tools provided by the Docker image.
Getting Started with Streamlit
With your environment set up, it’s time to dive into Streamlit and start building interactive applications. This section will guide you through creating your first Streamlit app, understanding the basic structure of a Streamlit script, and running Streamlit apps from within the Jupyter Notebook provided by the Docker container.
Creating Your First Streamlit App
Let’s begin by creating a simple Streamlit application that displays text and a chart.
- Create a New Python Script:
- In the Jupyter Notebook interface, click on
New
and selectText File
. - Save the file as
app.py
in your working directory (/home/pydata/work
).
- Write the Streamlit Code:
Open app.py
and add the following code:
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>import</span> <span>numpy</span> <span>as</span> <span>np</span><span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>My First Streamlit App</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Welcome to my first Streamlit application!</span><span>"</span><span>)</span><span># Create a random dataframe </span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>20</span><span>,</span> <span>3</span><span>),</span><span>columns</span><span>=</span><span>[</span><span>'</span><span>Column A</span><span>'</span><span>,</span> <span>'</span><span>Column B</span><span>'</span><span>,</span> <span>'</span><span>Column C</span><span>'</span><span>]</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Here is a random dataframe:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Line chart of the data:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>import</span> <span>numpy</span> <span>as</span> <span>np</span> <span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>My First Streamlit App</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Welcome to my first Streamlit application!</span><span>"</span><span>)</span> <span># Create a random dataframe </span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>20</span><span>,</span> <span>3</span><span>),</span> <span>columns</span><span>=</span><span>[</span><span>'</span><span>Column A</span><span>'</span><span>,</span> <span>'</span><span>Column B</span><span>'</span><span>,</span> <span>'</span><span>Column C</span><span>'</span><span>]</span> <span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Here is a random dataframe:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Line chart of the data:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span>import streamlit as st import pandas as pd import numpy as np st.title("My First Streamlit App") st.write("Welcome to my first Streamlit application!") # Create a random dataframe df = pd.DataFrame( np.random.randn(20, 3), columns=['Column A', 'Column B', 'Column C'] ) st.write("Here is a random dataframe:") st.dataframe(df) st.write("Line chart of the data:") st.line_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation:
- Imports necessary libraries.
- Sets the title and writes introductory text.
- Generates a random DataFrame.
- Displays the DataFrame and a line chart based on the data.
Save the Script:
- Ensure that you save app.py after adding the code.
Understanding the Basic Structure of a Streamlit Script
A Streamlit script is a standard Python script with the streamlit library functions to create interactive elements.
- Import Streamlit:
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span>import streamlit as st
Enter fullscreen mode Exit fullscreen mode
Set the Title and Headers:
<span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>App Title</span><span>"</span><span>)</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>This is a header</span><span>"</span><span>)</span><span>st</span><span>.</span><span>subheader</span><span>(</span><span>"</span><span>This is a subheader</span><span>"</span><span>)</span><span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>App Title</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>This is a header</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>subheader</span><span>(</span><span>"</span><span>This is a subheader</span><span>"</span><span>)</span>st.title("App Title") st.header("This is a header") st.subheader("This is a subheader")
Enter fullscreen mode Exit fullscreen mode
Write Text:
<span>st</span><span>.</span><span>text</span><span>(</span><span>"</span><span>This is a simple text.</span><span>"</span><span>)</span><span>st</span><span>.</span><span>markdown</span><span>(</span><span>"</span><span>This is a text with **markdown** formatting.</span><span>"</span><span>)</span><span>st</span><span>.</span><span>text</span><span>(</span><span>"</span><span>This is a simple text.</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>markdown</span><span>(</span><span>"</span><span>This is a text with **markdown** formatting.</span><span>"</span><span>)</span>st.text("This is a simple text.") st.markdown("This is a text with **markdown** formatting.")
Enter fullscreen mode Exit fullscreen mode
Display Data:
<span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span> <span># Displays an interactive table </span><span>st</span><span>.</span><span>table</span><span>(</span><span>df</span><span>)</span> <span># Displays a static table </span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span> <span># Displays an interactive table </span><span>st</span><span>.</span><span>table</span><span>(</span><span>df</span><span>)</span> <span># Displays a static table </span>st.dataframe(df) # Displays an interactive table st.table(df) # Displays a static table
Enter fullscreen mode Exit fullscreen mode
Display Charts:
<span>st</span><span>.</span><span>line_chart</span><span>(</span><span>data</span><span>)</span><span>st</span><span>.</span><span>bar_chart</span><span>(</span><span>data</span><span>)</span><span>st</span><span>.</span><span>area_chart</span><span>(</span><span>data</span><span>)</span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>data</span><span>)</span> <span>st</span><span>.</span><span>bar_chart</span><span>(</span><span>data</span><span>)</span> <span>st</span><span>.</span><span>area_chart</span><span>(</span><span>data</span><span>)</span>st.line_chart(data) st.bar_chart(data) st.area_chart(data)
Enter fullscreen mode Exit fullscreen mode
Add Interactive Widgets:
<span>name</span> <span>=</span> <span>st</span><span>.</span><span>text_input</span><span>(</span><span>"</span><span>Enter your name:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Hello, </span><span>{</span><span>name</span><span>}</span><span>!</span><span>"</span><span>)</span><span>age</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>"</span><span>Select your age:</span><span>"</span><span>,</span> <span>0</span><span>,</span> <span>100</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>You are </span><span>{</span><span>age</span><span>}</span><span> years old.</span><span>"</span><span>)</span><span>name</span> <span>=</span> <span>st</span><span>.</span><span>text_input</span><span>(</span><span>"</span><span>Enter your name:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Hello, </span><span>{</span><span>name</span><span>}</span><span>!</span><span>"</span><span>)</span> <span>age</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>"</span><span>Select your age:</span><span>"</span><span>,</span> <span>0</span><span>,</span> <span>100</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>You are </span><span>{</span><span>age</span><span>}</span><span> years old.</span><span>"</span><span>)</span>name = st.text_input("Enter your name:") st.write(f"Hello, {name}!") age = st.slider("Select your age:", 0, 100) st.write(f"You are {age} years old.")
Enter fullscreen mode Exit fullscreen mode
Layout Elements:
<span>with</span> <span>st</span><span>.</span><span>sidebar</span><span>:</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is the sidebar.</span><span>"</span><span>)</span><span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span><span>col1</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in column 1</span><span>"</span><span>)</span><span>col2</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in column 2</span><span>"</span><span>)</span><span>with</span> <span>st</span><span>.</span><span>sidebar</span><span>:</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is the sidebar.</span><span>"</span><span>)</span> <span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span> <span>col1</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in column 1</span><span>"</span><span>)</span> <span>col2</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in column 2</span><span>"</span><span>)</span>with st.sidebar: st.write("This is the sidebar.") col1, col2 = st.columns(2) col1.write("Content in column 1") col2.write("Content in column 2")
Enter fullscreen mode Exit fullscreen mode
Running Streamlit Apps from Within the Jupyter Notebook
To run your Streamlit app within the Docker container and access it from your host machine:
Open a Terminal in Jupyter Notebook:
In the Jupyter interface, click on New and select Terminal.
Navigate to the Working Directory:
<span>cd</span> /home/pydata/work<span>cd</span> /home/pydata/workcd /home/pydata/work
Enter fullscreen mode Exit fullscreen mode
Run the Streamlit App:
Execute the following command:
streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0streamlit run app.py --server.enableCORS false --server.enableXsrfProtection false --server.port 8501 --server.address 0.0.0.0
Enter fullscreen mode Exit fullscreen mode
Explanation of Flags:
-
--server.enableCORS false:
Disables Cross-Origin Resource Sharing protection. -
--server.enableXsrfProtection false:
Disables Cross-Site Request Forgery protection. -
--server.port 8501:
Sets the port to 8501. -
--server.address 0.0.0.0:
Makes the app accessible externally.
Access the Streamlit App:
- Open your web browser and navigate to
http://localhost:8501
.
You should see your Streamlit app running.
Interact with the App:
- Modify
app.py
to add more features or interactive elements. - Save the changes, and the app will automatically reload in the browser.
Tips for Running Streamlit in Docker
Expose the Correct Port:
When running the Docker container, ensure you expose the port used by Streamlit. If you use port 8501, run the container with:
docker run <span>-p</span> 8888:8888 <span>-p</span> 8501:8501 <span>-v</span> <span>$(</span><span>pwd</span><span>)</span>:/home/pydata/work alexmerced/datanotebookdocker run <span>-p</span> 8888:8888 <span>-p</span> 8501:8501 <span>-v</span> <span>$(</span><span>pwd</span><span>)</span>:/home/pydata/work alexmerced/datanotebookdocker run -p 8888:8888 -p 8501:8501 -v $(pwd):/home/pydata/work alexmerced/datanotebook
Enter fullscreen mode Exit fullscreen mode
Running Multiple Apps:
Use different ports for each app and expose them accordingly.
Background Execution:
To run the Streamlit app without tying up the terminal, use:
<span>nohup </span>streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0 &<span>nohup </span>streamlit run app.py <span>--server</span>.enableCORS <span>false</span> <span>--server</span>.enableXsrfProtection <span>false</span> <span>--server</span>.port 8501 <span>--server</span>.address 0.0.0.0 &nohup streamlit run app.py --server.enableCORS false --server.enableXsrfProtection false --server.port 8501 --server.address 0.0.0.0 &
Enter fullscreen mode Exit fullscreen mode
This runs the app in the background and outputs logs to nohup.out.
Summary
In this section, you:
- Created your first Streamlit app using the pre-configured Docker environment.
- Learned about the basic structure and components of a Streamlit script.
- Ran the Streamlit app from within the Jupyter Notebook environment.
- Accessed and interacted with the app via your web browser.
With these foundational skills, you’re ready to explore more advanced features of Streamlit to build sophisticated data applications.
Building Interactive Data Visualizations
Data visualization is a crucial aspect of data analysis and communication. Streamlit simplifies the process of creating interactive and dynamic visualizations that can help users explore and understand data more effectively. In this section, we’ll explore how to use Streamlit’s built-in functions and integrate popular visualization libraries to build interactive data visualizations.
Using Streamlit’s Built-in Chart Functions
Streamlit provides easy-to-use functions for creating basic charts directly from data structures like Pandas DataFrames and NumPy arrays.
Line Chart
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>import</span> <span>numpy</span> <span>as</span> <span>np</span><span># Generate random data </span><span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>100</span><span>,</span> <span>3</span><span>)</span><span>columns</span> <span>=</span> <span>[</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>'</span><span>Feature C</span><span>'</span><span>]</span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span><span># Display line chart </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>import</span> <span>numpy</span> <span>as</span> <span>np</span> <span># Generate random data </span><span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>100</span><span>,</span> <span>3</span><span>)</span> <span>columns</span> <span>=</span> <span>[</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>'</span><span>Feature C</span><span>'</span><span>]</span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span> <span># Display line chart </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span>import streamlit as st import pandas as pd import numpy as np # Generate random data data = np.random.randn(100, 3) columns = ['Feature A', 'Feature B', 'Feature C'] df = pd.DataFrame(data, columns=columns) # Display line chart st.line_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: The st.line_chart() function takes a DataFrame or array-like object and renders an interactive line chart.
Bar Chart
<span># Display bar chart </span><span>st</span><span>.</span><span>bar_chart</span><span>(</span><span>df</span><span>)</span><span># Display bar chart </span><span>st</span><span>.</span><span>bar_chart</span><span>(</span><span>df</span><span>)</span># Display bar chart st.bar_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: st.bar_chart() displays a bar chart. It’s useful for categorical data or comparing different groups.
Area Chart
<span># Display area chart </span><span>st</span><span>.</span><span>area_chart</span><span>(</span><span>df</span><span>)</span><span># Display area chart </span><span>st</span><span>.</span><span>area_chart</span><span>(</span><span>df</span><span>)</span># Display area chart st.area_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: st.area_chart() creates an area chart, which is similar to a line chart but with the area below the line filled.
Customizing Charts with Altair
For more advanced visualizations, Streamlit supports libraries like Altair, which provides a declarative statistical visualization library for Python.
Creating an Altair Chart
<span>import</span> <span>altair</span> <span>as</span> <span>alt</span><span># Create an Altair chart </span><span>chart</span> <span>=</span> <span>alt</span><span>.</span><span>Chart</span><span>(</span><span>df</span><span>.</span><span>reset_index</span><span>()).</span><span>mark_circle</span><span>(</span><span>size</span><span>=</span><span>60</span><span>).</span><span>encode</span><span>(</span><span>x</span><span>=</span><span>'</span><span>index</span><span>'</span><span>,</span><span>y</span><span>=</span><span>'</span><span>Feature A</span><span>'</span><span>,</span><span>color</span><span>=</span><span>'</span><span>Feature B</span><span>'</span><span>,</span><span>tooltip</span><span>=</span><span>[</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>'</span><span>Feature C</span><span>'</span><span>]</span><span>).</span><span>interactive</span><span>()</span><span>st</span><span>.</span><span>altair_chart</span><span>(</span><span>chart</span><span>,</span> <span>use_container_width</span><span>=</span><span>True</span><span>)</span><span>import</span> <span>altair</span> <span>as</span> <span>alt</span> <span># Create an Altair chart </span><span>chart</span> <span>=</span> <span>alt</span><span>.</span><span>Chart</span><span>(</span><span>df</span><span>.</span><span>reset_index</span><span>()).</span><span>mark_circle</span><span>(</span><span>size</span><span>=</span><span>60</span><span>).</span><span>encode</span><span>(</span> <span>x</span><span>=</span><span>'</span><span>index</span><span>'</span><span>,</span> <span>y</span><span>=</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>color</span><span>=</span><span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>tooltip</span><span>=</span><span>[</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>'</span><span>Feature C</span><span>'</span><span>]</span> <span>).</span><span>interactive</span><span>()</span> <span>st</span><span>.</span><span>altair_chart</span><span>(</span><span>chart</span><span>,</span> <span>use_container_width</span><span>=</span><span>True</span><span>)</span>import altair as alt # Create an Altair chart chart = alt.Chart(df.reset_index()).mark_circle(size=60).encode( x='index', y='Feature A', color='Feature B', tooltip=['Feature A', 'Feature B', 'Feature C'] ).interactive() st.altair_chart(chart, use_container_width=True)
Enter fullscreen mode Exit fullscreen mode
Explanation: This code creates an interactive scatter plot using Altair, where you can hover over points to see tooltips.
Interactive Widgets for User Input
Streamlit allows you to add widgets that enable users to interact with your visualizations.
Adding a Slider
<span># Slider to select number of data points </span><span>num_points</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>'</span><span>Select number of data points</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>10</span><span>,</span> <span>max_value</span><span>=</span><span>100</span><span>,</span> <span>value</span><span>=</span><span>50</span><span>)</span><span># Generate data based on slider </span><span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>num_points</span><span>,</span> <span>3</span><span>)</span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span><span># Display updated chart </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span><span># Slider to select number of data points </span><span>num_points</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>'</span><span>Select number of data points</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>10</span><span>,</span> <span>max_value</span><span>=</span><span>100</span><span>,</span> <span>value</span><span>=</span><span>50</span><span>)</span> <span># Generate data based on slider </span><span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>num_points</span><span>,</span> <span>3</span><span>)</span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span> <span># Display updated chart </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span># Slider to select number of data points num_points = st.slider('Select number of data points', min_value=10, max_value=100, value=50) # Generate data based on slider data = np.random.randn(num_points, 3) df = pd.DataFrame(data, columns=columns) # Display updated chart st.line_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: The slider widget allows users to select the number of data points, and the chart updates accordingly.
Selectbox for Options
<span># Selectbox to choose a feature </span><span>feature</span> <span>=</span> <span>st</span><span>.</span><span>selectbox</span><span>(</span><span>'</span><span>Select a feature to display</span><span>'</span><span>,</span> <span>columns</span><span>)</span><span># Display the selected feature </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>[</span><span>feature</span><span>])</span><span># Selectbox to choose a feature </span><span>feature</span> <span>=</span> <span>st</span><span>.</span><span>selectbox</span><span>(</span><span>'</span><span>Select a feature to display</span><span>'</span><span>,</span> <span>columns</span><span>)</span> <span># Display the selected feature </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>[</span><span>feature</span><span>])</span># Selectbox to choose a feature feature = st.selectbox('Select a feature to display', columns) # Display the selected feature st.line_chart(df[feature])
Enter fullscreen mode Exit fullscreen mode
Explanation: The selectbox lets users choose which feature to visualize.
Integrating Plotly for Advanced Visualizations
Plotly is another powerful library for creating interactive graphs.
Plotly Example
<span>import</span> <span>plotly.express</span> <span>as</span> <span>px</span><span># Create a Plotly figure </span><span>fig</span> <span>=</span> <span>px</span><span>.</span><span>scatter</span><span>(</span><span>df</span><span>,</span> <span>x</span><span>=</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>y</span><span>=</span><span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>size</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>,</span> <span>color</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>,</span> <span>hover_name</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>)</span><span># Display the Plotly figure in Streamlit </span><span>st</span><span>.</span><span>plotly_chart</span><span>(</span><span>fig</span><span>,</span> <span>use_container_width</span><span>=</span><span>True</span><span>)</span><span>import</span> <span>plotly.express</span> <span>as</span> <span>px</span> <span># Create a Plotly figure </span><span>fig</span> <span>=</span> <span>px</span><span>.</span><span>scatter</span><span>(</span><span>df</span><span>,</span> <span>x</span><span>=</span><span>'</span><span>Feature A</span><span>'</span><span>,</span> <span>y</span><span>=</span><span>'</span><span>Feature B</span><span>'</span><span>,</span> <span>size</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>,</span> <span>color</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>,</span> <span>hover_name</span><span>=</span><span>'</span><span>Feature C</span><span>'</span><span>)</span> <span># Display the Plotly figure in Streamlit </span><span>st</span><span>.</span><span>plotly_chart</span><span>(</span><span>fig</span><span>,</span> <span>use_container_width</span><span>=</span><span>True</span><span>)</span>import plotly.express as px # Create a Plotly figure fig = px.scatter(df, x='Feature A', y='Feature B', size='Feature C', color='Feature C', hover_name='Feature C') # Display the Plotly figure in Streamlit st.plotly_chart(fig, use_container_width=True)
Enter fullscreen mode Exit fullscreen mode
Explanation: This code creates an interactive scatter plot with Plotly, which includes zooming, panning, and tooltips.
Combining Widgets and Visualizations
You can combine multiple widgets and charts to create a rich interactive experience.
Example: Interactive Data Filtering
<span># Multiselect to choose features </span><span>selected_features</span> <span>=</span> <span>st</span><span>.</span><span>multiselect</span><span>(</span><span>'</span><span>Select features to visualize</span><span>'</span><span>,</span> <span>columns</span><span>,</span> <span>default</span><span>=</span><span>columns</span><span>)</span><span># Checkbox to toggle data normalization </span><span>normalize</span> <span>=</span> <span>st</span><span>.</span><span>checkbox</span><span>(</span><span>'</span><span>Normalize data</span><span>'</span><span>)</span><span># Process data based on user input </span><span>if</span> <span>normalize</span><span>:</span><span>df_normalized</span> <span>=</span> <span>(</span><span>df</span> <span>-</span> <span>df</span><span>.</span><span>mean</span><span>())</span> <span>/</span> <span>df</span><span>.</span><span>std</span><span>()</span><span>data_to_plot</span> <span>=</span> <span>df_normalized</span><span>[</span><span>selected_features</span><span>]</span><span>else</span><span>:</span><span>data_to_plot</span> <span>=</span> <span>df</span><span>[</span><span>selected_features</span><span>]</span><span># Display line chart of selected features </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>data_to_plot</span><span>)</span><span># Multiselect to choose features </span><span>selected_features</span> <span>=</span> <span>st</span><span>.</span><span>multiselect</span><span>(</span><span>'</span><span>Select features to visualize</span><span>'</span><span>,</span> <span>columns</span><span>,</span> <span>default</span><span>=</span><span>columns</span><span>)</span> <span># Checkbox to toggle data normalization </span><span>normalize</span> <span>=</span> <span>st</span><span>.</span><span>checkbox</span><span>(</span><span>'</span><span>Normalize data</span><span>'</span><span>)</span> <span># Process data based on user input </span><span>if</span> <span>normalize</span><span>:</span> <span>df_normalized</span> <span>=</span> <span>(</span><span>df</span> <span>-</span> <span>df</span><span>.</span><span>mean</span><span>())</span> <span>/</span> <span>df</span><span>.</span><span>std</span><span>()</span> <span>data_to_plot</span> <span>=</span> <span>df_normalized</span><span>[</span><span>selected_features</span><span>]</span> <span>else</span><span>:</span> <span>data_to_plot</span> <span>=</span> <span>df</span><span>[</span><span>selected_features</span><span>]</span> <span># Display line chart of selected features </span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>data_to_plot</span><span>)</span># Multiselect to choose features selected_features = st.multiselect('Select features to visualize', columns, default=columns) # Checkbox to toggle data normalization normalize = st.checkbox('Normalize data') # Process data based on user input if normalize: df_normalized = (df - df.mean()) / df.std() data_to_plot = df_normalized[selected_features] else: data_to_plot = df[selected_features] # Display line chart of selected features st.line_chart(data_to_plot)
Enter fullscreen mode Exit fullscreen mode
Explanation: Users can select which features to visualize and whether to normalize the data, and the chart updates accordingly.
Best Practices for Interactive Visualizations
- Limit Data Size: Large datasets can slow down your app. Consider sampling or aggregating data.
- Use Caching: Use @st.cache_data decorator to cache data loading and computation functions.
- Provide Instructions: Use st.markdown() or st.write() to guide users on how to interact with your app.
- Optimize Layout: Organize widgets and charts using columns and expanders for a clean interface.
Example of Layout Optimization
<span># Create columns </span><span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span><span>with</span> <span>col1</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>'</span><span>User Inputs</span><span>'</span><span>)</span><span># Add widgets here </span> <span>num_points</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>'</span><span>Number of points</span><span>'</span><span>,</span> <span>10</span><span>,</span> <span>100</span><span>,</span> <span>50</span><span>)</span><span>feature</span> <span>=</span> <span>st</span><span>.</span><span>selectbox</span><span>(</span><span>'</span><span>Feature</span><span>'</span><span>,</span> <span>columns</span><span>)</span><span>with</span> <span>col2</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>'</span><span>Visualization</span><span>'</span><span>)</span><span># Generate and display chart </span> <span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>num_points</span><span>,</span> <span>len</span><span>(</span><span>columns</span><span>))</span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>[</span><span>feature</span><span>])</span><span># Create columns </span><span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span> <span>with</span> <span>col1</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>'</span><span>User Inputs</span><span>'</span><span>)</span> <span># Add widgets here </span> <span>num_points</span> <span>=</span> <span>st</span><span>.</span><span>slider</span><span>(</span><span>'</span><span>Number of points</span><span>'</span><span>,</span> <span>10</span><span>,</span> <span>100</span><span>,</span> <span>50</span><span>)</span> <span>feature</span> <span>=</span> <span>st</span><span>.</span><span>selectbox</span><span>(</span><span>'</span><span>Feature</span><span>'</span><span>,</span> <span>columns</span><span>)</span> <span>with</span> <span>col2</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>'</span><span>Visualization</span><span>'</span><span>)</span> <span># Generate and display chart </span> <span>data</span> <span>=</span> <span>np</span><span>.</span><span>random</span><span>.</span><span>randn</span><span>(</span><span>num_points</span><span>,</span> <span>len</span><span>(</span><span>columns</span><span>))</span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>(</span><span>data</span><span>,</span> <span>columns</span><span>=</span><span>columns</span><span>)</span> <span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>[</span><span>feature</span><span>])</span># Create columns col1, col2 = st.columns(2) with col1: st.header('User Inputs') # Add widgets here num_points = st.slider('Number of points', 10, 100, 50) feature = st.selectbox('Feature', columns) with col2: st.header('Visualization') # Generate and display chart data = np.random.randn(num_points, len(columns)) df = pd.DataFrame(data, columns=columns) st.line_chart(df[feature])
Enter fullscreen mode Exit fullscreen mode
Explanation: This layout separates user inputs and visualizations into two columns, making the app more organized.
Summary
In this section, you’ve learned how to:
- Use Streamlit’s built-in chart functions to create quick visualizations.
- Customize charts using Altair and Plotly for more advanced visualizations.
- Add interactive widgets like sliders and selectboxes to make your visualizations dynamic.
- Combine widgets and charts to build a user-friendly data exploration tool.
By leveraging these features, you can create powerful interactive applications that make data exploration and analysis more accessible to your audience.
Advanced Streamlit Features
As you become more familiar with Streamlit, you’ll discover a wealth of advanced features that allow you to build more sophisticated and powerful applications. In this section, we’ll delve into some of these capabilities, including state management, dynamic content creation, file handling, and performance optimization through caching.
State Management with st.session_state
Streamlit runs your script from top to bottom every time a user interacts with a widget. To maintain state across these reruns, you can use st.session_state
, which is a dictionary-like object that persists throughout the user’s session.
Example: Counter Application
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span># Initialize counter in session state </span><span>if</span> <span>'</span><span>counter</span><span>'</span> <span>not</span> <span>in</span> <span>st</span><span>.</span><span>session_state</span><span>:</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>=</span> <span>0</span><span># Increment counter on button click </span><span>if</span> <span>st</span><span>.</span><span>button</span><span>(</span><span>'</span><span>Increment</span><span>'</span><span>):</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>+=</span> <span>1</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Counter value: </span><span>{</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span><span>}</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span># Initialize counter in session state </span><span>if</span> <span>'</span><span>counter</span><span>'</span> <span>not</span> <span>in</span> <span>st</span><span>.</span><span>session_state</span><span>:</span> <span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>=</span> <span>0</span> <span># Increment counter on button click </span><span>if</span> <span>st</span><span>.</span><span>button</span><span>(</span><span>'</span><span>Increment</span><span>'</span><span>):</span> <span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>+=</span> <span>1</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Counter value: </span><span>{</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span><span>}</span><span>"</span><span>)</span>import streamlit as st # Initialize counter in session state if 'counter' not in st.session_state: st.session_state.counter = 0 # Increment counter on button click if st.button('Increment'): st.session_state.counter += 1 st.write(f"Counter value: {st.session_state.counter}")
Enter fullscreen mode Exit fullscreen mode
Explanation: The counter value is stored in st.session_state.counter, ensuring it persists across interactions.
Dynamic Content with st.expander and st.tabs
Streamlit provides layout elements to organize content and improve user experience.
Using st.expander
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is visible content</span><span>"</span><span>)</span><span>with</span> <span>st</span><span>.</span><span>expander</span><span>(</span><span>"</span><span>Click to expand</span><span>"</span><span>):</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This content is hidden by default</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is visible content</span><span>"</span><span>)</span> <span>with</span> <span>st</span><span>.</span><span>expander</span><span>(</span><span>"</span><span>Click to expand</span><span>"</span><span>):</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This content is hidden by default</span><span>"</span><span>)</span>import streamlit as st st.write("This is visible content") with st.expander("Click to expand"): st.write("This content is hidden by default")
Enter fullscreen mode Exit fullscreen mode
Explanation: st.expander creates a collapsible section that users can expand or collapse.
Using st.tabs
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>tab1</span><span>,</span> <span>tab2</span> <span>=</span> <span>st</span><span>.</span><span>tabs</span><span>([</span><span>"</span><span>Tab 1</span><span>"</span><span>,</span> <span>"</span><span>Tab 2</span><span>"</span><span>])</span><span>with</span> <span>tab1</span><span>:</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in Tab 1</span><span>"</span><span>)</span><span>with</span> <span>tab2</span><span>:</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in Tab 2</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>tab1</span><span>,</span> <span>tab2</span> <span>=</span> <span>st</span><span>.</span><span>tabs</span><span>([</span><span>"</span><span>Tab 1</span><span>"</span><span>,</span> <span>"</span><span>Tab 2</span><span>"</span><span>])</span> <span>with</span> <span>tab1</span><span>:</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in Tab 1</span><span>"</span><span>)</span> <span>with</span> <span>tab2</span><span>:</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content in Tab 2</span><span>"</span><span>)</span>import streamlit as st tab1, tab2 = st.tabs(["Tab 1", "Tab 2"]) with tab1: st.write("Content in Tab 1") with tab2: st.write("Content in Tab 2")
Enter fullscreen mode Exit fullscreen mode
Explanation: st.tabs allows you to organize content into tabs for better navigation.
Uploading and Handling Files with st.file_uploader
Allow users to upload files directly into your app for processing.
Example: CSV File Uploader
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Choose a CSV file</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>"</span><span>csv</span><span>"</span><span>)</span><span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>uploaded_file</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Uploaded Data:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Choose a CSV file</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>"</span><span>csv</span><span>"</span><span>)</span> <span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>uploaded_file</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Uploaded Data:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span>import streamlit as st import pandas as pd uploaded_file = st.file_uploader("Choose a CSV file", type="csv") if uploaded_file is not None: df = pd.read_csv(uploaded_file) st.write("Uploaded Data:") st.dataframe(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: Users can upload a CSV file, which the app reads and displays as a DataFrame.
Caching with @st.cache_data for Performance Optimization
Heavy computations or data loading can slow down your app. Use caching to store results and avoid redundant processing.
Using @st.cache_data
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>@st.cache_data</span><span>def</span> <span>load_data</span><span>(</span><span>url</span><span>):</span><span>return</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>url</span><span>)</span><span>data_url</span> <span>=</span> <span>'</span><span>https://path-to-large-dataset.csv</span><span>'</span><span>df</span> <span>=</span> <span>load_data</span><span>(</span><span>data_url</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data loaded successfully</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>.</span><span>head</span><span>())</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>@st.cache_data</span> <span>def</span> <span>load_data</span><span>(</span><span>url</span><span>):</span> <span>return</span> <span>pd</span><span>.</span><span>read_csv</span><span>(</span><span>url</span><span>)</span> <span>data_url</span> <span>=</span> <span>'</span><span>https://path-to-large-dataset.csv</span><span>'</span> <span>df</span> <span>=</span> <span>load_data</span><span>(</span><span>data_url</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data loaded successfully</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>.</span><span>head</span><span>())</span>import streamlit as st import pandas as pd @st.cache_data def load_data(url): return pd.read_csv(url) data_url = 'https://path-to-large-dataset.csv' df = load_data(data_url) st.write("Data loaded successfully") st.dataframe(df.head())
Enter fullscreen mode Exit fullscreen mode
Explanation: The @st.cache_data decorator caches the load_data function’s output, improving performance on subsequent runs.
Customizing the App Layout
Enhance user experience by customizing your app’s layout and appearance.
Setting Page Configuration
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>st</span><span>.</span><span>set_page_config</span><span>(</span><span>page_title</span><span>=</span><span>"</span><span>Advanced Streamlit Features</span><span>"</span><span>,</span><span>page_icon</span><span>=</span><span>"</span><span></span><span>"</span><span>,</span><span>layout</span><span>=</span><span>"</span><span>wide</span><span>"</span><span>,</span><span>initial_sidebar_state</span><span>=</span><span>"</span><span>expanded</span><span>"</span><span>,</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>st</span><span>.</span><span>set_page_config</span><span>(</span> <span>page_title</span><span>=</span><span>"</span><span>Advanced Streamlit Features</span><span>"</span><span>,</span> <span>page_icon</span><span>=</span><span>"</span><span></span><span>"</span><span>,</span> <span>layout</span><span>=</span><span>"</span><span>wide</span><span>"</span><span>,</span> <span>initial_sidebar_state</span><span>=</span><span>"</span><span>expanded</span><span>"</span><span>,</span> <span>)</span>import streamlit as st st.set_page_config( page_title="Advanced Streamlit Features", page_icon="", layout="wide", initial_sidebar_state="expanded", )
Enter fullscreen mode Exit fullscreen mode
Explanation: st.set_page_config sets global configurations like the page title, icon, layout, and sidebar state.
Using Columns and Containers
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span><span>with</span> <span>col1</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Column 1</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content for the first column</span><span>"</span><span>)</span><span>with</span> <span>col2</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Column 2</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content for the second column</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span> <span>with</span> <span>col1</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Column 1</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content for the first column</span><span>"</span><span>)</span> <span>with</span> <span>col2</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Column 2</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Content for the second column</span><span>"</span><span>)</span>import streamlit as st col1, col2 = st.columns(2) with col1: st.header("Column 1") st.write("Content for the first column") with col2: st.header("Column 2") st.write("Content for the second column")
Enter fullscreen mode Exit fullscreen mode
Explanation: Columns help organize content side by side.
Theming and Styling
Apply custom themes to match your app’s branding or preferred aesthetics.
Applying a Custom Theme
Create a .streamlit/config.toml file in your app directory with the following content:
<span>[theme]</span><span>primaryColor</span><span>=</span><span>"#d33682"</span><span>backgroundColor</span><span>=</span><span>"#002b36"</span><span>secondaryBackgroundColor</span><span>=</span><span>"#586e75"</span><span>textColor</span><span>=</span><span>"#ffffff"</span><span>font</span><span>=</span><span>"sans serif"</span><span>[theme]</span> <span>primaryColor</span><span>=</span><span>"#d33682"</span> <span>backgroundColor</span><span>=</span><span>"#002b36"</span> <span>secondaryBackgroundColor</span><span>=</span><span>"#586e75"</span> <span>textColor</span><span>=</span><span>"#ffffff"</span> <span>font</span><span>=</span><span>"sans serif"</span>[theme] primaryColor="#d33682" backgroundColor="#002b36" secondaryBackgroundColor="#586e75" textColor="#ffffff" font="sans serif"
Enter fullscreen mode Exit fullscreen mode
Explanation: The theme settings adjust the app’s color scheme and font.
Interactive Widgets for Advanced User Input
Streamlit offers a variety of widgets for complex user interactions.
Date Input and Time Input
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>date</span> <span>=</span> <span>st</span><span>.</span><span>date_input</span><span>(</span><span>"</span><span>Select a date</span><span>"</span><span>)</span><span>time</span> <span>=</span> <span>st</span><span>.</span><span>time_input</span><span>(</span><span>"</span><span>Select a time</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>You selected </span><span>{</span><span>date</span><span>}</span><span> at </span><span>{</span><span>time</span><span>}</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>date</span> <span>=</span> <span>st</span><span>.</span><span>date_input</span><span>(</span><span>"</span><span>Select a date</span><span>"</span><span>)</span> <span>time</span> <span>=</span> <span>st</span><span>.</span><span>time_input</span><span>(</span><span>"</span><span>Select a time</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>You selected </span><span>{</span><span>date</span><span>}</span><span> at </span><span>{</span><span>time</span><span>}</span><span>"</span><span>)</span>import streamlit as st date = st.date_input("Select a date") time = st.time_input("Select a time") st.write(f"You selected {date} at {time}")
Enter fullscreen mode Exit fullscreen mode
Explanation: Allows users to input dates and times.
Color Picker
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>color</span> <span>=</span> <span>st</span><span>.</span><span>color_picker</span><span>(</span><span>'</span><span>Pick A Color</span><span>'</span><span>,</span> <span>'</span><span>#00f900</span><span>'</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>'</span><span>The current color is</span><span>'</span><span>,</span> <span>color</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>color</span> <span>=</span> <span>st</span><span>.</span><span>color_picker</span><span>(</span><span>'</span><span>Pick A Color</span><span>'</span><span>,</span> <span>'</span><span>#00f900</span><span>'</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>'</span><span>The current color is</span><span>'</span><span>,</span> <span>color</span><span>)</span>import streamlit as st color = st.color_picker('Pick A Color', '#00f900') st.write('The current color is', color)
Enter fullscreen mode Exit fullscreen mode
Explanation: Users can select a color, which can be used in visualizations or styling.
Advanced Callbacks and Event Handling
Respond to user interactions with callbacks.
Using Button Callbacks
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>def</span> <span>on_button_click</span><span>():</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Button was clicked!</span><span>"</span><span>)</span><span>st</span><span>.</span><span>button</span><span>(</span><span>"</span><span>Click Me</span><span>"</span><span>,</span> <span>on_click</span><span>=</span><span>on_button_click</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>def</span> <span>on_button_click</span><span>():</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Button was clicked!</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>button</span><span>(</span><span>"</span><span>Click Me</span><span>"</span><span>,</span> <span>on_click</span><span>=</span><span>on_button_click</span><span>)</span>import streamlit as st def on_button_click(): st.write("Button was clicked!") st.button("Click Me", on_click=on_button_click)
Enter fullscreen mode Exit fullscreen mode
Explanation: The on_click parameter specifies a function to execute when the button is clicked.
Integrating with External APIs
Fetch and display data from external sources.
Example: Fetching Data from an API
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>requests</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Fetch data from an API</span><span>"</span><span>)</span><span>response</span> <span>=</span> <span>requests</span><span>.</span><span>get</span><span>(</span><span>'</span><span>https://api.example.com/data</span><span>'</span><span>)</span><span>if</span> <span>response</span><span>.</span><span>status_code</span> <span>==</span> <span>200</span><span>:</span><span>data</span> <span>=</span> <span>response</span><span>.</span><span>json</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>data</span><span>)</span><span>else</span><span>:</span><span>st</span><span>.</span><span>error</span><span>(</span><span>"</span><span>Failed to fetch data</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>requests</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Fetch data from an API</span><span>"</span><span>)</span> <span>response</span> <span>=</span> <span>requests</span><span>.</span><span>get</span><span>(</span><span>'</span><span>https://api.example.com/data</span><span>'</span><span>)</span> <span>if</span> <span>response</span><span>.</span><span>status_code</span> <span>==</span> <span>200</span><span>:</span> <span>data</span> <span>=</span> <span>response</span><span>.</span><span>json</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>data</span><span>)</span> <span>else</span><span>:</span> <span>st</span><span>.</span><span>error</span><span>(</span><span>"</span><span>Failed to fetch data</span><span>"</span><span>)</span>import streamlit as st import requests st.write("Fetch data from an API") response = requests.get('https://api.example.com/data') if response.status_code == 200: data = response.json() st.write(data) else: st.error("Failed to fetch data")
Enter fullscreen mode Exit fullscreen mode
Explanation: Uses the requests library to fetch data from an API and display it.
Real-Time Data Updates with WebSockets
Streamlit supports bi-directional communication for real-time updates.
Using st.experimental_get_query_params
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>params</span> <span>=</span> <span>st</span><span>.</span><span>experimental_get_query_params</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Query parameters:</span><span>"</span><span>,</span> <span>params</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>params</span> <span>=</span> <span>st</span><span>.</span><span>experimental_get_query_params</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Query parameters:</span><span>"</span><span>,</span> <span>params</span><span>)</span>import streamlit as st params = st.experimental_get_query_params() st.write("Query parameters:", params)
Enter fullscreen mode Exit fullscreen mode
Explanation: Access query parameters from the URL to control app behavior dynamically.
Modularizing Code with Components
Break down your app into reusable components.
Creating a Custom Component
<span># components.py </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>def</span> <span>display_header</span><span>():</span><span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>Advanced Streamlit Features</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is a custom component</span><span>"</span><span>)</span><span># main app </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>from</span> <span>components</span> <span>import</span> <span>display_header</span><span>display_header</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Main app content</span><span>"</span><span>)</span><span># components.py </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>def</span> <span>display_header</span><span>():</span> <span>st</span><span>.</span><span>title</span><span>(</span><span>"</span><span>Advanced Streamlit Features</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>This is a custom component</span><span>"</span><span>)</span> <span># main app </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>from</span> <span>components</span> <span>import</span> <span>display_header</span> <span>display_header</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Main app content</span><span>"</span><span>)</span># components.py import streamlit as st def display_header(): st.title("Advanced Streamlit Features") st.write("This is a custom component") # main app import streamlit as st from components import display_header display_header() st.write("Main app content")
Enter fullscreen mode Exit fullscreen mode
Explanation: Organize code by splitting it into modules for better maintainability.
Localization and Internationalization
Make your app accessible to a global audience.
Setting the Language
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Hello, World!</span><span>"</span><span>)</span><span># Use gettext or other localization libraries for translations </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Hello, World!</span><span>"</span><span>)</span> <span># Use gettext or other localization libraries for translations </span>import streamlit as st st.write("Hello, World!") # Use gettext or other localization libraries for translations
Enter fullscreen mode Exit fullscreen mode
Explanation: While Streamlit doesn’t provide built-in localization, you can use Python’s localization libraries.
Accessibility Features
Ensure your app is usable by people with disabilities.
-
Use Semantic HTML: Streamlit automatically generates accessible HTML elements.
-
Provide Alt Text: When displaying images, use the caption parameter.
<span>st</span><span>.</span><span>image</span><span>(</span><span>'</span><span>image.png</span><span>'</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Descriptive text</span><span>'</span><span>)</span><span>st</span><span>.</span><span>image</span><span>(</span><span>'</span><span>image.png</span><span>'</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Descriptive text</span><span>'</span><span>)</span>st.image('image.png', caption='Descriptive text')
Enter fullscreen mode Exit fullscreen mode
Summary
In this section, we’ve explored several advanced features of Streamlit that empower you to build more interactive and efficient applications:
- State Management: Use st.session_state to preserve data across user interactions.
- Dynamic Layouts: Organize content with expanders, tabs, columns, and containers.
- File Handling: Allow users to upload and interact with files directly in the app.
- Performance Optimization: Improve app speed with caching decorators like @st.cache_data.
- Customization: Enhance the look and feel with custom themes and page configurations.
- Advanced Widgets: Utilize a variety of input widgets for richer user interactions.
- External Integrations: Connect your app to external APIs and services.
- Code Organization: Modularize your code for better readability and maintenance.
- Global Reach: Consider localization and accessibility to reach a wider audience.
By mastering these advanced features, you can create sophisticated Streamlit applications that provide a seamless and engaging user experience.
Integrating Machine Learning Models
Streamlit excels at making machine learning models accessible through interactive web applications. In this section, we’ll explore how to integrate machine learning models into your Streamlit apps using the pre-installed libraries in the Python Data Science Notebook Docker Image, such as TensorFlow, PyTorch, and scikit-learn.
Loading Pre-trained Models with TensorFlow and PyTorch
The Docker image comes with TensorFlow and PyTorch installed, allowing you to work with complex neural network models.
Using TensorFlow
Loading a Pre-trained Model
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>tensorflow</span> <span>as</span> <span>tf</span><span># Load a pre-trained model, e.g., MobileNetV2 </span><span>model</span> <span>=</span> <span>tf</span><span>.</span><span>keras</span><span>.</span><span>applications</span><span>.</span><span>MobileNetV2</span><span>(</span><span>weights</span><span>=</span><span>'</span><span>imagenet</span><span>'</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model loaded successfully.</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>tensorflow</span> <span>as</span> <span>tf</span> <span># Load a pre-trained model, e.g., MobileNetV2 </span><span>model</span> <span>=</span> <span>tf</span><span>.</span><span>keras</span><span>.</span><span>applications</span><span>.</span><span>MobileNetV2</span><span>(</span><span>weights</span><span>=</span><span>'</span><span>imagenet</span><span>'</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model loaded successfully.</span><span>"</span><span>)</span>import streamlit as st import tensorflow as tf # Load a pre-trained model, e.g., MobileNetV2 model = tf.keras.applications.MobileNetV2(weights='imagenet') st.write("Model loaded successfully.")
Enter fullscreen mode Exit fullscreen mode
Making Predictions
<span>from</span> <span>tensorflow.keras.applications.mobilenet_v2</span> <span>import</span> <span>preprocess_input</span><span>,</span> <span>decode_predictions</span><span>from</span> <span>PIL</span> <span>import</span> <span>Image</span><span>import</span> <span>numpy</span> <span>as</span> <span>np</span><span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Upload an image</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>[</span><span>"</span><span>jpg</span><span>"</span><span>,</span> <span>"</span><span>jpeg</span><span>"</span><span>,</span> <span>"</span><span>png</span><span>"</span><span>])</span><span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span><span># Load and preprocess the image </span> <span>image</span> <span>=</span> <span>Image</span><span>.</span><span>open</span><span>(</span><span>uploaded_file</span><span>)</span><span>st</span><span>.</span><span>image</span><span>(</span><span>image</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Uploaded Image</span><span>'</span><span>,</span> <span>use_column_width</span><span>=</span><span>True</span><span>)</span><span>img</span> <span>=</span> <span>image</span><span>.</span><span>resize</span><span>((</span><span>224</span><span>,</span> <span>224</span><span>))</span><span>img_array</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>(</span><span>img</span><span>)</span><span>img_array</span> <span>=</span> <span>preprocess_input</span><span>(</span><span>img_array</span><span>)</span><span>img_array</span> <span>=</span> <span>np</span><span>.</span><span>expand_dims</span><span>(</span><span>img_array</span><span>,</span> <span>axis</span><span>=</span><span>0</span><span>)</span><span># Make prediction </span> <span>predictions</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>img_array</span><span>)</span><span>results</span> <span>=</span> <span>decode_predictions</span><span>(</span><span>predictions</span><span>,</span> <span>top</span><span>=</span><span>3</span><span>)[</span><span>0</span><span>]</span><span># Display predictions </span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Top Predictions:</span><span>"</span><span>)</span><span>for</span> <span>i</span><span>,</span> <span>res</span> <span>in</span> <span>enumerate</span><span>(</span><span>results</span><span>):</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>{</span><span>i</span><span>+</span><span>1</span><span>}</span><span>. </span><span>{</span><span>res</span><span>[</span><span>1</span><span>]</span><span>}</span><span>: </span><span>{</span><span>round</span><span>(</span><span>res</span><span>[</span><span>2</span><span>]</span><span>*</span><span>100</span><span>,</span> <span>2</span><span>)</span><span>}</span><span>%</span><span>"</span><span>)</span><span>from</span> <span>tensorflow.keras.applications.mobilenet_v2</span> <span>import</span> <span>preprocess_input</span><span>,</span> <span>decode_predictions</span> <span>from</span> <span>PIL</span> <span>import</span> <span>Image</span> <span>import</span> <span>numpy</span> <span>as</span> <span>np</span> <span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Upload an image</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>[</span><span>"</span><span>jpg</span><span>"</span><span>,</span> <span>"</span><span>jpeg</span><span>"</span><span>,</span> <span>"</span><span>png</span><span>"</span><span>])</span> <span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span> <span># Load and preprocess the image </span> <span>image</span> <span>=</span> <span>Image</span><span>.</span><span>open</span><span>(</span><span>uploaded_file</span><span>)</span> <span>st</span><span>.</span><span>image</span><span>(</span><span>image</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Uploaded Image</span><span>'</span><span>,</span> <span>use_column_width</span><span>=</span><span>True</span><span>)</span> <span>img</span> <span>=</span> <span>image</span><span>.</span><span>resize</span><span>((</span><span>224</span><span>,</span> <span>224</span><span>))</span> <span>img_array</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>(</span><span>img</span><span>)</span> <span>img_array</span> <span>=</span> <span>preprocess_input</span><span>(</span><span>img_array</span><span>)</span> <span>img_array</span> <span>=</span> <span>np</span><span>.</span><span>expand_dims</span><span>(</span><span>img_array</span><span>,</span> <span>axis</span><span>=</span><span>0</span><span>)</span> <span># Make prediction </span> <span>predictions</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>img_array</span><span>)</span> <span>results</span> <span>=</span> <span>decode_predictions</span><span>(</span><span>predictions</span><span>,</span> <span>top</span><span>=</span><span>3</span><span>)[</span><span>0</span><span>]</span> <span># Display predictions </span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Top Predictions:</span><span>"</span><span>)</span> <span>for</span> <span>i</span><span>,</span> <span>res</span> <span>in</span> <span>enumerate</span><span>(</span><span>results</span><span>):</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>{</span><span>i</span><span>+</span><span>1</span><span>}</span><span>. </span><span>{</span><span>res</span><span>[</span><span>1</span><span>]</span><span>}</span><span>: </span><span>{</span><span>round</span><span>(</span><span>res</span><span>[</span><span>2</span><span>]</span><span>*</span><span>100</span><span>,</span> <span>2</span><span>)</span><span>}</span><span>%</span><span>"</span><span>)</span>from tensorflow.keras.applications.mobilenet_v2 import preprocess_input, decode_predictions from PIL import Image import numpy as np uploaded_file = st.file_uploader("Upload an image", type=["jpg", "jpeg", "png"]) if uploaded_file is not None: # Load and preprocess the image image = Image.open(uploaded_file) st.image(image, caption='Uploaded Image', use_column_width=True) img = image.resize((224, 224)) img_array = np.array(img) img_array = preprocess_input(img_array) img_array = np.expand_dims(img_array, axis=0) # Make prediction predictions = model.predict(img_array) results = decode_predictions(predictions, top=3)[0] # Display predictions st.write("Top Predictions:") for i, res in enumerate(results): st.write(f"{i+1}. {res[1]}: {round(res[2]*100, 2)}%")
Enter fullscreen mode Exit fullscreen mode
Explanation: Users can upload an image, and the app displays the top predictions from the pre-trained MobileNetV2 model.
Using PyTorch
Loading a Pre-trained Model
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>import</span> <span>torch</span><span>from</span> <span>torchvision</span> <span>import</span> <span>models</span><span>,</span> <span>transforms</span><span># Load a pre-trained ResNet model </span><span>model</span> <span>=</span> <span>models</span><span>.</span><span>resnet18</span><span>(</span><span>pretrained</span><span>=</span><span>True</span><span>)</span><span>model</span><span>.</span><span>eval</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>PyTorch model loaded successfully.</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>import</span> <span>torch</span> <span>from</span> <span>torchvision</span> <span>import</span> <span>models</span><span>,</span> <span>transforms</span> <span># Load a pre-trained ResNet model </span><span>model</span> <span>=</span> <span>models</span><span>.</span><span>resnet18</span><span>(</span><span>pretrained</span><span>=</span><span>True</span><span>)</span> <span>model</span><span>.</span><span>eval</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>PyTorch model loaded successfully.</span><span>"</span><span>)</span>import streamlit as st import torch from torchvision import models, transforms # Load a pre-trained ResNet model model = models.resnet18(pretrained=True) model.eval() st.write("PyTorch model loaded successfully.")
Enter fullscreen mode Exit fullscreen mode
Making Predictions
<span>from</span> <span>PIL</span> <span>import</span> <span>Image</span><span>import</span> <span>torchvision.transforms</span> <span>as</span> <span>T</span><span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Upload an image</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>[</span><span>"</span><span>jpg</span><span>"</span><span>,</span> <span>"</span><span>jpeg</span><span>"</span><span>,</span> <span>"</span><span>png</span><span>"</span><span>])</span><span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span><span># Load and preprocess the image </span> <span>image</span> <span>=</span> <span>Image</span><span>.</span><span>open</span><span>(</span><span>uploaded_file</span><span>)</span><span>st</span><span>.</span><span>image</span><span>(</span><span>image</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Uploaded Image</span><span>'</span><span>,</span> <span>use_column_width</span><span>=</span><span>True</span><span>)</span><span>preprocess</span> <span>=</span> <span>T</span><span>.</span><span>Compose</span><span>([</span><span>T</span><span>.</span><span>Resize</span><span>(</span><span>256</span><span>),</span><span>T</span><span>.</span><span>CenterCrop</span><span>(</span><span>224</span><span>),</span><span>T</span><span>.</span><span>ToTensor</span><span>(),</span><span>T</span><span>.</span><span>Normalize</span><span>(</span><span>mean</span><span>=</span><span>[</span><span>0.485</span><span>,</span> <span>0.456</span><span>,</span> <span>0.406</span><span>],</span><span>std</span><span>=</span><span>[</span><span>0.229</span><span>,</span> <span>0.224</span><span>,</span> <span>0.225</span><span>]</span><span>)</span><span>])</span><span>img_t</span> <span>=</span> <span>preprocess</span><span>(</span><span>image</span><span>)</span><span>batch_t</span> <span>=</span> <span>torch</span><span>.</span><span>unsqueeze</span><span>(</span><span>img_t</span><span>,</span> <span>0</span><span>)</span><span># Make prediction </span> <span>with</span> <span>torch</span><span>.</span><span>no_grad</span><span>():</span><span>out</span> <span>=</span> <span>model</span><span>(</span><span>batch_t</span><span>)</span><span>probabilities</span> <span>=</span> <span>torch</span><span>.</span><span>nn</span><span>.</span><span>functional</span><span>.</span><span>softmax</span><span>(</span><span>out</span><span>[</span><span>0</span><span>],</span> <span>dim</span><span>=</span><span>0</span><span>)</span><span># Load labels </span> <span>with</span> <span>open</span><span>(</span><span>"</span><span>imagenet_classes.txt</span><span>"</span><span>)</span> <span>as</span> <span>f</span><span>:</span><span>labels</span> <span>=</span> <span>[</span><span>line</span><span>.</span><span>strip</span><span>()</span> <span>for</span> <span>line</span> <span>in</span> <span>f</span><span>.</span><span>readlines</span><span>()]</span><span># Show top 3 predictions </span> <span>top3_prob</span><span>,</span> <span>top3_catid</span> <span>=</span> <span>torch</span><span>.</span><span>topk</span><span>(</span><span>probabilities</span><span>,</span> <span>3</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Top Predictions:</span><span>"</span><span>)</span><span>for</span> <span>i</span> <span>in</span> <span>range</span><span>(</span><span>top3_prob</span><span>.</span><span>size</span><span>(</span><span>0</span><span>)):</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>{</span><span>i</span><span>+</span><span>1</span><span>}</span><span>. </span><span>{</span><span>labels</span><span>[</span><span>top3_catid</span><span>[</span><span>i</span><span>]]</span><span>}</span><span>: </span><span>{</span><span>round</span><span>(</span><span>top3_prob</span><span>[</span><span>i</span><span>].</span><span>item</span><span>()</span><span>*</span><span>100</span><span>,</span> <span>2</span><span>)</span><span>}</span><span>%</span><span>"</span><span>)</span><span>from</span> <span>PIL</span> <span>import</span> <span>Image</span> <span>import</span> <span>torchvision.transforms</span> <span>as</span> <span>T</span> <span>uploaded_file</span> <span>=</span> <span>st</span><span>.</span><span>file_uploader</span><span>(</span><span>"</span><span>Upload an image</span><span>"</span><span>,</span> <span>type</span><span>=</span><span>[</span><span>"</span><span>jpg</span><span>"</span><span>,</span> <span>"</span><span>jpeg</span><span>"</span><span>,</span> <span>"</span><span>png</span><span>"</span><span>])</span> <span>if</span> <span>uploaded_file</span> <span>is</span> <span>not</span> <span>None</span><span>:</span> <span># Load and preprocess the image </span> <span>image</span> <span>=</span> <span>Image</span><span>.</span><span>open</span><span>(</span><span>uploaded_file</span><span>)</span> <span>st</span><span>.</span><span>image</span><span>(</span><span>image</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Uploaded Image</span><span>'</span><span>,</span> <span>use_column_width</span><span>=</span><span>True</span><span>)</span> <span>preprocess</span> <span>=</span> <span>T</span><span>.</span><span>Compose</span><span>([</span> <span>T</span><span>.</span><span>Resize</span><span>(</span><span>256</span><span>),</span> <span>T</span><span>.</span><span>CenterCrop</span><span>(</span><span>224</span><span>),</span> <span>T</span><span>.</span><span>ToTensor</span><span>(),</span> <span>T</span><span>.</span><span>Normalize</span><span>(</span> <span>mean</span><span>=</span><span>[</span><span>0.485</span><span>,</span> <span>0.456</span><span>,</span> <span>0.406</span><span>],</span> <span>std</span><span>=</span><span>[</span><span>0.229</span><span>,</span> <span>0.224</span><span>,</span> <span>0.225</span><span>]</span> <span>)</span> <span>])</span> <span>img_t</span> <span>=</span> <span>preprocess</span><span>(</span><span>image</span><span>)</span> <span>batch_t</span> <span>=</span> <span>torch</span><span>.</span><span>unsqueeze</span><span>(</span><span>img_t</span><span>,</span> <span>0</span><span>)</span> <span># Make prediction </span> <span>with</span> <span>torch</span><span>.</span><span>no_grad</span><span>():</span> <span>out</span> <span>=</span> <span>model</span><span>(</span><span>batch_t</span><span>)</span> <span>probabilities</span> <span>=</span> <span>torch</span><span>.</span><span>nn</span><span>.</span><span>functional</span><span>.</span><span>softmax</span><span>(</span><span>out</span><span>[</span><span>0</span><span>],</span> <span>dim</span><span>=</span><span>0</span><span>)</span> <span># Load labels </span> <span>with</span> <span>open</span><span>(</span><span>"</span><span>imagenet_classes.txt</span><span>"</span><span>)</span> <span>as</span> <span>f</span><span>:</span> <span>labels</span> <span>=</span> <span>[</span><span>line</span><span>.</span><span>strip</span><span>()</span> <span>for</span> <span>line</span> <span>in</span> <span>f</span><span>.</span><span>readlines</span><span>()]</span> <span># Show top 3 predictions </span> <span>top3_prob</span><span>,</span> <span>top3_catid</span> <span>=</span> <span>torch</span><span>.</span><span>topk</span><span>(</span><span>probabilities</span><span>,</span> <span>3</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Top Predictions:</span><span>"</span><span>)</span> <span>for</span> <span>i</span> <span>in</span> <span>range</span><span>(</span><span>top3_prob</span><span>.</span><span>size</span><span>(</span><span>0</span><span>)):</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>{</span><span>i</span><span>+</span><span>1</span><span>}</span><span>. </span><span>{</span><span>labels</span><span>[</span><span>top3_catid</span><span>[</span><span>i</span><span>]]</span><span>}</span><span>: </span><span>{</span><span>round</span><span>(</span><span>top3_prob</span><span>[</span><span>i</span><span>].</span><span>item</span><span>()</span><span>*</span><span>100</span><span>,</span> <span>2</span><span>)</span><span>}</span><span>%</span><span>"</span><span>)</span>from PIL import Image import torchvision.transforms as T uploaded_file = st.file_uploader("Upload an image", type=["jpg", "jpeg", "png"]) if uploaded_file is not None: # Load and preprocess the image image = Image.open(uploaded_file) st.image(image, caption='Uploaded Image', use_column_width=True) preprocess = T.Compose([ T.Resize(256), T.CenterCrop(224), T.ToTensor(), T.Normalize( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] ) ]) img_t = preprocess(image) batch_t = torch.unsqueeze(img_t, 0) # Make prediction with torch.no_grad(): out = model(batch_t) probabilities = torch.nn.functional.softmax(out[0], dim=0) # Load labels with open("imagenet_classes.txt") as f: labels = [line.strip() for line in f.readlines()] # Show top 3 predictions top3_prob, top3_catid = torch.topk(probabilities, 3) st.write("Top Predictions:") for i in range(top3_prob.size(0)): st.write(f"{i+1}. {labels[top3_catid[i]]}: {round(top3_prob[i].item()*100, 2)}%")
Enter fullscreen mode Exit fullscreen mode
Note: Ensure that the imagenet_classes.txt file is available in your working directory.
Building a Simple Prediction App with scikit-learn
Let’s build a simple regression app using scikit-learn.
Training a Model
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>from</span> <span>sklearn.datasets</span> <span>import</span> <span>load_boston</span><span>from</span> <span>sklearn.ensemble</span> <span>import</span> <span>RandomForestRegressor</span><span>from</span> <span>sklearn.model_selection</span> <span>import</span> <span>train_test_split</span><span># Load dataset </span><span>data</span> <span>=</span> <span>load_boston</span><span>()</span><span>X</span> <span>=</span> <span>data</span><span>.</span><span>data</span><span>y</span> <span>=</span> <span>data</span><span>.</span><span>target</span><span># Split data </span><span>X_train</span><span>,</span> <span>X_test</span><span>,</span> <span>y_train</span><span>,</span> <span>y_test</span> <span>=</span> <span>train_test_split</span><span>(</span><span>X</span><span>,</span> <span>y</span><span>,</span> <span>test_size</span><span>=</span><span>0.2</span><span>)</span><span># Train model </span><span>model</span> <span>=</span> <span>RandomForestRegressor</span><span>()</span><span>model</span><span>.</span><span>fit</span><span>(</span><span>X_train</span><span>,</span> <span>y_train</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model trained successfully.</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>from</span> <span>sklearn.datasets</span> <span>import</span> <span>load_boston</span> <span>from</span> <span>sklearn.ensemble</span> <span>import</span> <span>RandomForestRegressor</span> <span>from</span> <span>sklearn.model_selection</span> <span>import</span> <span>train_test_split</span> <span># Load dataset </span><span>data</span> <span>=</span> <span>load_boston</span><span>()</span> <span>X</span> <span>=</span> <span>data</span><span>.</span><span>data</span> <span>y</span> <span>=</span> <span>data</span><span>.</span><span>target</span> <span># Split data </span><span>X_train</span><span>,</span> <span>X_test</span><span>,</span> <span>y_train</span><span>,</span> <span>y_test</span> <span>=</span> <span>train_test_split</span><span>(</span><span>X</span><span>,</span> <span>y</span><span>,</span> <span>test_size</span><span>=</span><span>0.2</span><span>)</span> <span># Train model </span><span>model</span> <span>=</span> <span>RandomForestRegressor</span><span>()</span> <span>model</span><span>.</span><span>fit</span><span>(</span><span>X_train</span><span>,</span> <span>y_train</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model trained successfully.</span><span>"</span><span>)</span>import streamlit as st from sklearn.datasets import load_boston from sklearn.ensemble import RandomForestRegressor from sklearn.model_selection import train_test_split # Load dataset data = load_boston() X = data.data y = data.target # Split data X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) # Train model model = RandomForestRegressor() model.fit(X_train, y_train) st.write("Model trained successfully.")
Enter fullscreen mode Exit fullscreen mode
Making Predictions with User Input
<span>import</span> <span>numpy</span> <span>as</span> <span>np</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Boston Housing Price Prediction</span><span>"</span><span>)</span><span># Feature input sliders </span><span>CRIM</span> <span>=</span> <span>st</span><span>.</span><span>number_input</span><span>(</span><span>'</span><span>Per capita crime rate by town</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>0.0</span><span>,</span> <span>value</span><span>=</span><span>0.1</span><span>)</span><span>ZN</span> <span>=</span> <span>st</span><span>.</span><span>number_input</span><span>(</span><span>'</span><span>Proportion of residential land zoned for lots over 25,000 sq.ft.</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>0.0</span><span>,</span> <span>value</span><span>=</span><span>0.0</span><span>)</span><span># ... add inputs for other features </span><span># For brevity, we'll use default values for the rest of the features </span><span>input_features</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([[</span><span>CRIM</span><span>,</span> <span>ZN</span><span>]</span> <span>+</span> <span>[</span><span>0</span><span>]</span><span>*</span><span>(</span><span>X</span><span>.</span><span>shape</span><span>[</span><span>1</span><span>]</span><span>-</span><span>2</span><span>)])</span><span># Predict </span><span>prediction</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>input_features</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Predicted median value of owner-occupied homes: $</span><span>{</span><span>prediction</span><span>[</span><span>0</span><span>]</span><span>*</span><span>1000</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span><span>import</span> <span>numpy</span> <span>as</span> <span>np</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Boston Housing Price Prediction</span><span>"</span><span>)</span> <span># Feature input sliders </span><span>CRIM</span> <span>=</span> <span>st</span><span>.</span><span>number_input</span><span>(</span><span>'</span><span>Per capita crime rate by town</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>0.0</span><span>,</span> <span>value</span><span>=</span><span>0.1</span><span>)</span> <span>ZN</span> <span>=</span> <span>st</span><span>.</span><span>number_input</span><span>(</span><span>'</span><span>Proportion of residential land zoned for lots over 25,000 sq.ft.</span><span>'</span><span>,</span> <span>min_value</span><span>=</span><span>0.0</span><span>,</span> <span>value</span><span>=</span><span>0.0</span><span>)</span> <span># ... add inputs for other features </span> <span># For brevity, we'll use default values for the rest of the features </span><span>input_features</span> <span>=</span> <span>np</span><span>.</span><span>array</span><span>([[</span><span>CRIM</span><span>,</span> <span>ZN</span><span>]</span> <span>+</span> <span>[</span><span>0</span><span>]</span><span>*</span><span>(</span><span>X</span><span>.</span><span>shape</span><span>[</span><span>1</span><span>]</span><span>-</span><span>2</span><span>)])</span> <span># Predict </span><span>prediction</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>input_features</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Predicted median value of owner-occupied homes: $</span><span>{</span><span>prediction</span><span>[</span><span>0</span><span>]</span><span>*</span><span>1000</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span>import numpy as np st.header("Boston Housing Price Prediction") # Feature input sliders CRIM = st.number_input('Per capita crime rate by town', min_value=0.0, value=0.1) ZN = st.number_input('Proportion of residential land zoned for lots over 25,000 sq.ft.', min_value=0.0, value=0.0) # ... add inputs for other features # For brevity, we'll use default values for the rest of the features input_features = np.array([[CRIM, ZN] + [0]*(X.shape[1]-2)]) # Predict prediction = model.predict(input_features) st.write(f"Predicted median value of owner-occupied homes: ${prediction[0]*1000:.2f}")
Enter fullscreen mode Exit fullscreen mode
Explanation: Users can input values for features, and the app predicts housing prices.
Visualizing Model Outputs and Performance Metrics
Visualizations help in understanding model performance.
Displaying Metrics
<span>from</span> <span>sklearn.metrics</span> <span>import</span> <span>mean_squared_error</span><span>,</span> <span>r2_score</span><span># Predict on test set </span><span>y_pred</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>X_test</span><span>)</span><span># Calculate metrics </span><span>mse</span> <span>=</span> <span>mean_squared_error</span><span>(</span><span>y_test</span><span>,</span> <span>y_pred</span><span>)</span><span>r2</span> <span>=</span> <span>r2_score</span><span>(</span><span>y_test</span><span>,</span> <span>y_pred</span><span>)</span><span># Display metrics </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model Performance on Test Set:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Mean Squared Error: </span><span>{</span><span>mse</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>R² Score: </span><span>{</span><span>r2</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span><span>from</span> <span>sklearn.metrics</span> <span>import</span> <span>mean_squared_error</span><span>,</span> <span>r2_score</span> <span># Predict on test set </span><span>y_pred</span> <span>=</span> <span>model</span><span>.</span><span>predict</span><span>(</span><span>X_test</span><span>)</span> <span># Calculate metrics </span><span>mse</span> <span>=</span> <span>mean_squared_error</span><span>(</span><span>y_test</span><span>,</span> <span>y_pred</span><span>)</span> <span>r2</span> <span>=</span> <span>r2_score</span><span>(</span><span>y_test</span><span>,</span> <span>y_pred</span><span>)</span> <span># Display metrics </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Model Performance on Test Set:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Mean Squared Error: </span><span>{</span><span>mse</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>R² Score: </span><span>{</span><span>r2</span><span>:</span><span>.</span><span>2</span><span>f</span><span>}</span><span>"</span><span>)</span>from sklearn.metrics import mean_squared_error, r2_score # Predict on test set y_pred = model.predict(X_test) # Calculate metrics mse = mean_squared_error(y_test, y_pred) r2 = r2_score(y_test, y_pred) # Display metrics st.write("Model Performance on Test Set:") st.write(f"Mean Squared Error: {mse:.2f}") st.write(f"R² Score: {r2:.2f}")
Enter fullscreen mode Exit fullscreen mode
Plotting Actual vs. Predicted Values
<span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>({</span><span>'</span><span>Actual</span><span>'</span><span>:</span> <span>y_test</span><span>,</span> <span>'</span><span>Predicted</span><span>'</span><span>:</span> <span>y_pred</span><span>})</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Actual vs. Predicted Values</span><span>"</span><span>)</span><span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span>df</span> <span>=</span> <span>pd</span><span>.</span><span>DataFrame</span><span>({</span><span>'</span><span>Actual</span><span>'</span><span>:</span> <span>y_test</span><span>,</span> <span>'</span><span>Predicted</span><span>'</span><span>:</span> <span>y_pred</span><span>})</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Actual vs. Predicted Values</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>line_chart</span><span>(</span><span>df</span><span>)</span>import pandas as pd df = pd.DataFrame({'Actual': y_test, 'Predicted': y_pred}) st.write("Actual vs. Predicted Values") st.line_chart(df)
Enter fullscreen mode Exit fullscreen mode
Explanation: The line chart shows how closely the model’s predictions match the actual values.
Tips for Integrating Machine Learning Models in Streamlit
Model Serialization: For complex models, consider saving and loading models using joblib or pickle to avoid retraining every time.
<span>import</span> <span>joblib</span><span># Save model </span><span>joblib</span><span>.</span><span>dump</span><span>(</span><span>model</span><span>,</span> <span>'</span><span>model.joblib</span><span>'</span><span>)</span><span># Load model </span><span>model</span> <span>=</span> <span>joblib</span><span>.</span><span>load</span><span>(</span><span>'</span><span>model.joblib</span><span>'</span><span>)</span><span>import</span> <span>joblib</span> <span># Save model </span><span>joblib</span><span>.</span><span>dump</span><span>(</span><span>model</span><span>,</span> <span>'</span><span>model.joblib</span><span>'</span><span>)</span> <span># Load model </span><span>model</span> <span>=</span> <span>joblib</span><span>.</span><span>load</span><span>(</span><span>'</span><span>model.joblib</span><span>'</span><span>)</span>import joblib # Save model joblib.dump(model, 'model.joblib') # Load model model = joblib.load('model.joblib')
Enter fullscreen mode Exit fullscreen mode
Use Caching for Models: Cache the model loading or training functions to improve performance.
<span>@st.cache_resource</span><span>def</span> <span>load_model</span><span>():</span><span># Load or train model </span> <span>return</span> <span>model</span><span>@st.cache_resource</span> <span>def</span> <span>load_model</span><span>():</span> <span># Load or train model </span> <span>return</span> <span>model</span>@st.cache_resource def load_model(): # Load or train model return model
Enter fullscreen mode Exit fullscreen mode
Handle Large Models: Be mindful of resource limitations. Use efficient data structures and consider offloading heavy computations.
Provide Clear Instructions: Guide users on how to interact with the app, especially when expecting specific input formats.
Summary
In this section, you’ve learned how to:
- Load and use pre-trained models with TensorFlow and PyTorch in your Streamlit apps.
- Build a simple prediction app using scikit-learn.
- Collect user input to make predictions and display results.
- Visualize model outputs and performance metrics to evaluate model effectiveness.
By integrating machine learning models into your Streamlit applications, you can create powerful tools that make complex models accessible to end-users in an interactive and user-friendly manner.
Database Connectivity
In many data science projects, interacting with databases is essential for retrieving, processing, and storing data. Streamlit, combined with the powerful libraries included in the Python Data Science Notebook Docker Image, makes it straightforward to connect to various databases and integrate them into your applications. In this section, we’ll explore how to connect to databases using sqlalchemy
, psycopg2
, and specifically how to interact with Dremio using the dremio-simple-query
library.
Connecting to Dremio Using dremio-simple-query
Dremio is a data lakehouse platform that enables you to govern, join, and accelerate queries across various data sources such as Iceberg, Delta Lake, S3, JSON, CSV, RDBMS, and more. The dremio-simple-query
library simplifies querying a Dremio source using Apache Arrow Flight, providing performant data retrieval for analytics.
Installing the dremio-simple-query
Library
First, ensure that the dremio-simple-query
library is installed in your environment:
pip <span>install </span>dremio-simple-querypip <span>install </span>dremio-simple-querypip install dremio-simple-query
Enter fullscreen mode Exit fullscreen mode
Setting Up the Connection to Dremio
To connect to Dremio, you’ll need to obtain your Dremio Arrow Flight endpoint and an authentication token.
Obtaining the Arrow Flight Endpoint
- Dremio Cloud (NA):
grpc+tls://data.dremio.cloud:443
- Dremio Cloud (EU):
grpc+tls://data.eu.dremio.cloud:443
- Dremio Software (SSL):
grpc+tls://<ip-address>:32010
- Dremio Software (No SSL):
grpc://<ip-address>:32010
Getting Your Authentication Token
- Dremio Cloud: Obtain the token from the Dremio interface or via the REST API.
- Dremio Software: Obtain the token using the REST API.
You can use the get_token
function from the dremio-simple-query
library to retrieve the token programmatically.
Connecting to Dremio
<span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>from</span> <span>dremio_simple_query.connect</span> <span>import</span> <span>get_token</span><span>,</span> <span>DremioConnection</span><span>from</span> <span>os</span> <span>import</span> <span>getenv</span><span>from</span> <span>dotenv</span> <span>import</span> <span>load_dotenv</span><span># Load environment variables from a .env file (optional) </span><span>load_dotenv</span><span>()</span><span># Retrieve Dremio credentials and endpoints </span><span>username</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_username</span><span>"</span><span>]</span><span>password</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_password</span><span>"</span><span>]</span><span>arrow_endpoint</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_arrow_endpoint</span><span>"</span><span>]</span> <span># e.g., "grpc+tls://data.dremio.cloud:443" </span><span>login_endpoint</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_login_endpoint</span><span>"</span><span>]</span> <span># e.g., "https://your-dremio-server:9047/apiv2/login" </span><span># Get authentication token </span><span>payload</span> <span>=</span> <span>{</span><span>"</span><span>userName</span><span>"</span><span>:</span> <span>username</span><span>,</span><span>"</span><span>password</span><span>"</span><span>:</span> <span>password</span><span>}</span><span>token</span> <span>=</span> <span>get_token</span><span>(</span><span>uri</span><span>=</span><span>login_endpoint</span><span>,</span> <span>payload</span><span>=</span><span>payload</span><span>)</span><span># Establish connection to Dremio </span><span>dremio</span> <span>=</span> <span>DremioConnection</span><span>(</span><span>token</span><span>,</span> <span>arrow_endpoint</span><span>)</span><span># Test the connection </span><span>try</span><span>:</span><span>st</span><span>.</span><span>success</span><span>(</span><span>"</span><span>Successfully connected to Dremio.</span><span>"</span><span>)</span><span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span><span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>Failed to connect to Dremio: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>from</span> <span>dremio_simple_query.connect</span> <span>import</span> <span>get_token</span><span>,</span> <span>DremioConnection</span> <span>from</span> <span>os</span> <span>import</span> <span>getenv</span> <span>from</span> <span>dotenv</span> <span>import</span> <span>load_dotenv</span> <span># Load environment variables from a .env file (optional) </span><span>load_dotenv</span><span>()</span> <span># Retrieve Dremio credentials and endpoints </span><span>username</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_username</span><span>"</span><span>]</span> <span>password</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_password</span><span>"</span><span>]</span> <span>arrow_endpoint</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_arrow_endpoint</span><span>"</span><span>]</span> <span># e.g., "grpc+tls://data.dremio.cloud:443" </span><span>login_endpoint</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>dremio_login_endpoint</span><span>"</span><span>]</span> <span># e.g., "https://your-dremio-server:9047/apiv2/login" </span> <span># Get authentication token </span><span>payload</span> <span>=</span> <span>{</span> <span>"</span><span>userName</span><span>"</span><span>:</span> <span>username</span><span>,</span> <span>"</span><span>password</span><span>"</span><span>:</span> <span>password</span> <span>}</span> <span>token</span> <span>=</span> <span>get_token</span><span>(</span><span>uri</span><span>=</span><span>login_endpoint</span><span>,</span> <span>payload</span><span>=</span><span>payload</span><span>)</span> <span># Establish connection to Dremio </span><span>dremio</span> <span>=</span> <span>DremioConnection</span><span>(</span><span>token</span><span>,</span> <span>arrow_endpoint</span><span>)</span> <span># Test the connection </span><span>try</span><span>:</span> <span>st</span><span>.</span><span>success</span><span>(</span><span>"</span><span>Successfully connected to Dremio.</span><span>"</span><span>)</span> <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span> <span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>Failed to connect to Dremio: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span>import streamlit as st from dremio_simple_query.connect import get_token, DremioConnection from os import getenv from dotenv import load_dotenv # Load environment variables from a .env file (optional) load_dotenv() # Retrieve Dremio credentials and endpoints username = st.secrets["dremio_username"] password = st.secrets["dremio_password"] arrow_endpoint = st.secrets["dremio_arrow_endpoint"] # e.g., "grpc+tls://data.dremio.cloud:443" login_endpoint = st.secrets["dremio_login_endpoint"] # e.g., "https://your-dremio-server:9047/apiv2/login" # Get authentication token payload = { "userName": username, "password": password } token = get_token(uri=login_endpoint, payload=payload) # Establish connection to Dremio dremio = DremioConnection(token, arrow_endpoint) # Test the connection try: st.success("Successfully connected to Dremio.") except Exception as e: st.error(f"Failed to connect to Dremio: {e}")
Enter fullscreen mode Exit fullscreen mode
Note: Ensure that you securely manage your credentials using Streamlit’s secrets management or environment variables.
Querying Data from Dremio
You can now query data from Dremio and retrieve it in various formats.
Retrieving Data as an Arrow Table
<span># Query data and get a FlightStreamReader object </span><span>stream</span> <span>=</span> <span>dremio</span><span>.</span><span>toArrow</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span><span># Convert the stream to an Arrow Table </span><span>arrow_table</span> <span>=</span> <span>stream</span><span>.</span><span>read_all</span><span>()</span><span># Optionally, display the data in Streamlit </span><span>df</span> <span>=</span> <span>arrow_table</span><span>.</span><span>to_pandas</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span><span># Query data and get a FlightStreamReader object </span><span>stream</span> <span>=</span> <span>dremio</span><span>.</span><span>toArrow</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span> <span># Convert the stream to an Arrow Table </span><span>arrow_table</span> <span>=</span> <span>stream</span><span>.</span><span>read_all</span><span>()</span> <span># Optionally, display the data in Streamlit </span><span>df</span> <span>=</span> <span>arrow_table</span><span>.</span><span>to_pandas</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span># Query data and get a FlightStreamReader object stream = dremio.toArrow("SELECT * FROM your_space.your_table LIMIT 100") # Convert the stream to an Arrow Table arrow_table = stream.read_all() # Optionally, display the data in Streamlit df = arrow_table.to_pandas() st.write("Data from Dremio:") st.dataframe(df)
Enter fullscreen mode Exit fullscreen mode
Retrieving Data as a Pandas DataFrame
<span># Directly get a Pandas DataFrame </span><span>df</span> <span>=</span> <span>dremio</span><span>.</span><span>toPandas</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span><span># Directly get a Pandas DataFrame </span><span>df</span> <span>=</span> <span>dremio</span><span>.</span><span>toPandas</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span># Directly get a Pandas DataFrame df = dremio.toPandas("SELECT * FROM your_space.your_table LIMIT 100") st.write("Data from Dremio:") st.dataframe(df)
Enter fullscreen mode Exit fullscreen mode
Retrieving Data as a Polars DataFrame
<span># Get a Polars DataFrame </span><span>df_polars</span> <span>=</span> <span>dremio</span><span>.</span><span>toPolars</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio (Polars DataFrame):</span><span>"</span><span>)</span><span>st</span><span>.</span><span>write</span><span>(</span><span>df_polars</span><span>)</span><span># Get a Polars DataFrame </span><span>df_polars</span> <span>=</span> <span>dremio</span><span>.</span><span>toPolars</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from Dremio (Polars DataFrame):</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>df_polars</span><span>)</span># Get a Polars DataFrame df_polars = dremio.toPolars("SELECT * FROM your_space.your_table LIMIT 100") st.write("Data from Dremio (Polars DataFrame):") st.write(df_polars)
Enter fullscreen mode Exit fullscreen mode
Querying with DuckDB
You can leverage DuckDB for in-memory analytics on the data retrieved from Dremio.
Using the DuckDB Relation API
<span># Retrieve data as a DuckDB relation </span><span>duck_rel</span> <span>=</span> <span>dremio</span><span>.</span><span>toDuckDB</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span><span># Perform queries on the DuckDB relation </span><span>result</span> <span>=</span> <span>duck_rel</span><span>.</span><span>filter</span><span>(</span><span>"</span><span>column_name > 50</span><span>"</span><span>).</span><span>df</span><span>()</span><span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data using DuckDB:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span><span># Retrieve data as a DuckDB relation </span><span>duck_rel</span> <span>=</span> <span>dremio</span><span>.</span><span>toDuckDB</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span> <span># Perform queries on the DuckDB relation </span><span>result</span> <span>=</span> <span>duck_rel</span><span>.</span><span>filter</span><span>(</span><span>"</span><span>column_name > 50</span><span>"</span><span>).</span><span>df</span><span>()</span> <span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data using DuckDB:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span># Retrieve data as a DuckDB relation duck_rel = dremio.toDuckDB("SELECT * FROM your_space.your_table LIMIT 100") # Perform queries on the DuckDB relation result = duck_rel.filter("column_name > 50").df() # Display the result st.write("Filtered Data using DuckDB:") st.dataframe(result)
Enter fullscreen mode Exit fullscreen mode
Querying Arrow Objects with DuckDB
Alternatively, you can query Arrow Tables using DuckDB:
<span>import</span> <span>duckdb</span><span># Get data from Dremio as an Arrow Table </span><span>stream</span> <span>=</span> <span>dremio</span><span>.</span><span>toArrow</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span><span>arrow_table</span> <span>=</span> <span>stream</span><span>.</span><span>read_all</span><span>()</span><span># Create a DuckDB connection </span><span>con</span> <span>=</span> <span>duckdb</span><span>.</span><span>connect</span><span>()</span><span># Register the Arrow Table with DuckDB </span><span>con</span><span>.</span><span>register</span><span>(</span><span>"</span><span>dremio_table</span><span>"</span><span>,</span> <span>arrow_table</span><span>)</span><span># Perform SQL queries using DuckDB </span><span>query</span> <span>=</span> <span>"</span><span>SELECT * FROM dremio_table WHERE column_name > 50</span><span>"</span><span>result</span> <span>=</span> <span>con</span><span>.</span><span>execute</span><span>(</span><span>query</span><span>).</span><span>fetch_df</span><span>()</span><span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data using DuckDB on Arrow Table:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span><span>import</span> <span>duckdb</span> <span># Get data from Dremio as an Arrow Table </span><span>stream</span> <span>=</span> <span>dremio</span><span>.</span><span>toArrow</span><span>(</span><span>"</span><span>SELECT * FROM your_space.your_table LIMIT 100</span><span>"</span><span>)</span> <span>arrow_table</span> <span>=</span> <span>stream</span><span>.</span><span>read_all</span><span>()</span> <span># Create a DuckDB connection </span><span>con</span> <span>=</span> <span>duckdb</span><span>.</span><span>connect</span><span>()</span> <span># Register the Arrow Table with DuckDB </span><span>con</span><span>.</span><span>register</span><span>(</span><span>"</span><span>dremio_table</span><span>"</span><span>,</span> <span>arrow_table</span><span>)</span> <span># Perform SQL queries using DuckDB </span><span>query</span> <span>=</span> <span>"</span><span>SELECT * FROM dremio_table WHERE column_name > 50</span><span>"</span> <span>result</span> <span>=</span> <span>con</span><span>.</span><span>execute</span><span>(</span><span>query</span><span>).</span><span>fetch_df</span><span>()</span> <span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data using DuckDB on Arrow Table:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span>import duckdb # Get data from Dremio as an Arrow Table stream = dremio.toArrow("SELECT * FROM your_space.your_table LIMIT 100") arrow_table = stream.read_all() # Create a DuckDB connection con = duckdb.connect() # Register the Arrow Table with DuckDB con.register("dremio_table", arrow_table) # Perform SQL queries using DuckDB query = "SELECT * FROM dremio_table WHERE column_name > 50" result = con.execute(query).fetch_df() # Display the result st.write("Filtered Data using DuckDB on Arrow Table:") st.dataframe(result)
Enter fullscreen mode Exit fullscreen mode
Best Practices for Using Dremio with Streamlit
- Secure Credentials: Always handle your Dremio credentials securely. Use Streamlit’s secrets management or environment variables to avoid hardcoding sensitive information.
- Efficient Data Retrieval: Optimize your SQL queries to retrieve only the necessary data. Use LIMIT clauses and filters to reduce data transfer and improve performance.
- Error Handling: Implement try-except blocks to manage exceptions and provide informative error messages to users.
- Environment Configuration: Ensure that your arrow_endpoint and login_endpoint are correctly configured based on your Dremio deployment (Cloud or Software, with or without SSL).
Connecting to Databases Using sqlalchemy and psycopg2
In addition to Dremio, you might need to connect to other databases like PostgreSQL or MySQL. The Docker image comes with sqlalchemy, psycopg2-binary, and other database drivers pre-installed.
Setting Up a Connection to a PostgreSQL Database
<span>from</span> <span>sqlalchemy</span> <span>import</span> <span>create_engine</span><span>import</span> <span>pandas</span> <span>as</span> <span>pd</span><span># Database connection parameters </span><span>DB_USER</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_user</span><span>"</span><span>]</span><span>DB_PASSWORD</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_password</span><span>"</span><span>]</span><span>DB_HOST</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_host</span><span>"</span><span>]</span><span>DB_PORT</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_port</span><span>"</span><span>]</span><span>DB_NAME</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_name</span><span>"</span><span>]</span><span># Create a database engine </span><span>engine</span> <span>=</span> <span>create_engine</span><span>(</span><span>f</span><span>'</span><span>postgresql+psycopg2://</span><span>{</span><span>DB_USER</span><span>}</span><span>:</span><span>{</span><span>DB_PASSWORD</span><span>}</span><span>@</span><span>{</span><span>DB_HOST</span><span>}</span><span>:</span><span>{</span><span>DB_PORT</span><span>}</span><span>/</span><span>{</span><span>DB_NAME</span><span>}</span><span>'</span><span>)</span><span># Test the connection </span><span>try</span><span>:</span><span>with</span> <span>engine</span><span>.</span><span>connect</span><span>()</span> <span>as</span> <span>connection</span><span>:</span><span>st</span><span>.</span><span>success</span><span>(</span><span>"</span><span>Successfully connected to the PostgreSQL database.</span><span>"</span><span>)</span><span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span><span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>Failed to connect to the database: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span><span>from</span> <span>sqlalchemy</span> <span>import</span> <span>create_engine</span> <span>import</span> <span>pandas</span> <span>as</span> <span>pd</span> <span># Database connection parameters </span><span>DB_USER</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_user</span><span>"</span><span>]</span> <span>DB_PASSWORD</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_password</span><span>"</span><span>]</span> <span>DB_HOST</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_host</span><span>"</span><span>]</span> <span>DB_PORT</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_port</span><span>"</span><span>]</span> <span>DB_NAME</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>db_name</span><span>"</span><span>]</span> <span># Create a database engine </span><span>engine</span> <span>=</span> <span>create_engine</span><span>(</span><span>f</span><span>'</span><span>postgresql+psycopg2://</span><span>{</span><span>DB_USER</span><span>}</span><span>:</span><span>{</span><span>DB_PASSWORD</span><span>}</span><span>@</span><span>{</span><span>DB_HOST</span><span>}</span><span>:</span><span>{</span><span>DB_PORT</span><span>}</span><span>/</span><span>{</span><span>DB_NAME</span><span>}</span><span>'</span><span>)</span> <span># Test the connection </span><span>try</span><span>:</span> <span>with</span> <span>engine</span><span>.</span><span>connect</span><span>()</span> <span>as</span> <span>connection</span><span>:</span> <span>st</span><span>.</span><span>success</span><span>(</span><span>"</span><span>Successfully connected to the PostgreSQL database.</span><span>"</span><span>)</span> <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span> <span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>Failed to connect to the database: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span>from sqlalchemy import create_engine import pandas as pd # Database connection parameters DB_USER = st.secrets["db_user"] DB_PASSWORD = st.secrets["db_password"] DB_HOST = st.secrets["db_host"] DB_PORT = st.secrets["db_port"] DB_NAME = st.secrets["db_name"] # Create a database engine engine = create_engine(f'postgresql+psycopg2://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}') # Test the connection try: with engine.connect() as connection: st.success("Successfully connected to the PostgreSQL database.") except Exception as e: st.error(f"Failed to connect to the database: {e}")
Enter fullscreen mode Exit fullscreen mode
Querying Data from the Database
<span># Sample query </span><span>query</span> <span>=</span> <span>"</span><span>SELECT * FROM your_table LIMIT 10</span><span>"</span><span># Execute the query and load data into a DataFrame </span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>read_sql</span><span>(</span><span>query</span><span>,</span> <span>engine</span><span>)</span><span># Display the data </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from PostgreSQL:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span><span># Sample query </span><span>query</span> <span>=</span> <span>"</span><span>SELECT * FROM your_table LIMIT 10</span><span>"</span> <span># Execute the query and load data into a DataFrame </span><span>df</span> <span>=</span> <span>pd</span><span>.</span><span>read_sql</span><span>(</span><span>query</span><span>,</span> <span>engine</span><span>)</span> <span># Display the data </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Data from PostgreSQL:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>df</span><span>)</span># Sample query query = "SELECT * FROM your_table LIMIT 10" # Execute the query and load data into a DataFrame df = pd.read_sql(query, engine) # Display the data st.write("Data from PostgreSQL:") st.dataframe(df)
Enter fullscreen mode Exit fullscreen mode
Handling Large Datasets with Dask
When dealing with large datasets, performance can become an issue. Dask is a parallel computing library that integrates with Pandas to handle larger-than-memory datasets efficiently.
Using Dask to Query Large Tables
<span>import</span> <span>dask.dataframe</span> <span>as</span> <span>dd</span><span># Read data from SQL using Dask </span><span>df</span> <span>=</span> <span>dd</span><span>.</span><span>read_sql_table</span><span>(</span><span>table</span><span>=</span><span>'</span><span>large_table</span><span>'</span><span>,</span><span>uri</span><span>=</span><span>f</span><span>'</span><span>postgresql+psycopg2://</span><span>{</span><span>DB_USER</span><span>}</span><span>:</span><span>{</span><span>DB_PASSWORD</span><span>}</span><span>@</span><span>{</span><span>DB_HOST</span><span>}</span><span>:</span><span>{</span><span>DB_PORT</span><span>}</span><span>/</span><span>{</span><span>DB_NAME</span><span>}</span><span>'</span><span>,</span><span>index_col</span><span>=</span><span>'</span><span>id</span><span>'</span><span>)</span><span># Perform computations with Dask DataFrame </span><span>filtered_df</span> <span>=</span> <span>df</span><span>[</span><span>df</span><span>[</span><span>'</span><span>value</span><span>'</span><span>]</span> <span>></span> <span>100</span><span>]</span><span># Compute the result and convert to Pandas DataFrame </span><span>result</span> <span>=</span> <span>filtered_df</span><span>.</span><span>compute</span><span>()</span><span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data:</span><span>"</span><span>)</span><span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span><span>import</span> <span>dask.dataframe</span> <span>as</span> <span>dd</span> <span># Read data from SQL using Dask </span><span>df</span> <span>=</span> <span>dd</span><span>.</span><span>read_sql_table</span><span>(</span> <span>table</span><span>=</span><span>'</span><span>large_table</span><span>'</span><span>,</span> <span>uri</span><span>=</span><span>f</span><span>'</span><span>postgresql+psycopg2://</span><span>{</span><span>DB_USER</span><span>}</span><span>:</span><span>{</span><span>DB_PASSWORD</span><span>}</span><span>@</span><span>{</span><span>DB_HOST</span><span>}</span><span>:</span><span>{</span><span>DB_PORT</span><span>}</span><span>/</span><span>{</span><span>DB_NAME</span><span>}</span><span>'</span><span>,</span> <span>index_col</span><span>=</span><span>'</span><span>id</span><span>'</span> <span>)</span> <span># Perform computations with Dask DataFrame </span><span>filtered_df</span> <span>=</span> <span>df</span><span>[</span><span>df</span><span>[</span><span>'</span><span>value</span><span>'</span><span>]</span> <span>></span> <span>100</span><span>]</span> <span># Compute the result and convert to Pandas DataFrame </span><span>result</span> <span>=</span> <span>filtered_df</span><span>.</span><span>compute</span><span>()</span> <span># Display the result </span><span>st</span><span>.</span><span>write</span><span>(</span><span>"</span><span>Filtered Data:</span><span>"</span><span>)</span> <span>st</span><span>.</span><span>dataframe</span><span>(</span><span>result</span><span>)</span>import dask.dataframe as dd # Read data from SQL using Dask df = dd.read_sql_table( table='large_table', uri=f'postgresql+psycopg2://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}', index_col='id' ) # Perform computations with Dask DataFrame filtered_df = df[df['value'] > 100] # Compute the result and convert to Pandas DataFrame result = filtered_df.compute() # Display the result st.write("Filtered Data:") st.dataframe(result)
Enter fullscreen mode Exit fullscreen mode
Best Practices for Database Connectivity
- Secure Credentials: Use Streamlit’s secrets management or environment variables to store sensitive information.
- Parameterized Queries: Always use parameterized queries to prevent SQL injection.
- Connection Management: Use context managers (with statements) to ensure connections are properly closed.
- Error Handling: Implement try-except blocks to handle exceptions and provide user-friendly error messages.
- Limit Data Fetching: When displaying data in the app, limit the number of rows fetched to prevent performance issues.
Summary
In this section, you’ve learned how to:
- Connect to Dremio using the dremio-simple-query library and retrieve data efficiently using Apache Arrow Flight.
- Query data from Dremio and convert it into various formats such as Arrow Tables, Pandas DataFrames, Polars DataFrames, and DuckDB relations.
- Utilize DuckDB for in-memory analytics on data retrieved from Dremio.
- Connect to other databases like PostgreSQL using sqlalchemy and psycopg2.
- Handle large datasets efficiently using Dask. Implement best practices for secure and efficient database connectivity.
By integrating Dremio and other data systems into your Streamlit applications, you can create powerful data-driven apps that interact with live data sources, enabling real-time analysis and insights.
Deploying Streamlit Apps
With your Streamlit app developed and tested within the Docker environment, the next step is to deploy it so that others can access and use it. Deploying Streamlit apps can be done in several ways, including running the app locally, containerizing it with Docker, and deploying it to cloud platforms like Streamlit Community Cloud, Heroku, AWS, or other hosting services.
In this section, we’ll explore how to:
- Run your Streamlit app outside of Jupyter Notebook
- Containerize your Streamlit app with Docker
- Deploy your app to cloud platforms
Running Streamlit Apps Outside of Jupyter Notebook
While developing within Jupyter Notebook is convenient, deploying your app typically involves running it as a standalone script.
Steps to Run the App Locally
- Ensure Streamlit is Installed
If you followed the previous sections, Streamlit should already be installed in your Docker container. If not, install it using:
pip <span>install </span>streamlitpip <span>install </span>streamlitpip install streamlit
Enter fullscreen mode Exit fullscreen mode
Exit the Jupyter Notebook Environment
-
Stop the Jupyter Notebook server if it’s still running.
-
Navigate to Your App Directory
-
Open a terminal and navigate to the directory containing your app.py file:
<span>cd</span> /home/pydata/work<span>cd</span> /home/pydata/workcd /home/pydata/work
Enter fullscreen mode Exit fullscreen mode
Run the Streamlit App
Execute the following command:
streamlit run app.pystreamlit run app.pystreamlit run app.py
Enter fullscreen mode Exit fullscreen mode
This command starts the Streamlit server and serves your app at http://localhost:8501 by default.
Access the App in Your Browser
Open your web browser and navigate to http://localhost:8501 to interact with your app.
Containerizing Your Streamlit App with Docker
Containerizing your app ensures consistency across different environments and simplifies deployment.
Creating a Dockerfile for Your Streamlit App
-
Create a Dockerfile
-
In your app directory, create a file named Dockerfile with the following content:
<span># Use the official Python image as base</span><span>FROM</span><span> python:3.9-slim</span><span># Set the working directory</span><span>WORKDIR</span><span> /app</span><span># Copy the requirements file</span><span>COPY</span><span> requirements.txt .</span><span># Install dependencies</span><span>RUN </span>pip <span>install</span> <span>--no-cache-dir</span> <span>-r</span> requirements.txt<span># Copy the rest of the application code</span><span>COPY</span><span> . .</span><span># Expose the port Streamlit uses</span><span>EXPOSE</span><span> 8501</span><span># Run the Streamlit app</span><span>CMD</span><span> ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]</span><span># Use the official Python image as base</span> <span>FROM</span><span> python:3.9-slim</span> <span># Set the working directory</span> <span>WORKDIR</span><span> /app</span> <span># Copy the requirements file</span> <span>COPY</span><span> requirements.txt .</span> <span># Install dependencies</span> <span>RUN </span>pip <span>install</span> <span>--no-cache-dir</span> <span>-r</span> requirements.txt <span># Copy the rest of the application code</span> <span>COPY</span><span> . .</span> <span># Expose the port Streamlit uses</span> <span>EXPOSE</span><span> 8501</span> <span># Run the Streamlit app</span> <span>CMD</span><span> ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]</span># Use the official Python image as base FROM python:3.9-slim # Set the working directory WORKDIR /app # Copy the requirements file COPY requirements.txt . # Install dependencies RUN pip install --no-cache-dir -r requirements.txt # Copy the rest of the application code COPY . . # Expose the port Streamlit uses EXPOSE 8501 # Run the Streamlit app CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]
Enter fullscreen mode Exit fullscreen mode
Create a requirements.txt File
List all your Python dependencies in a file named requirements.txt:
streamlitpandasnumpy# Add any other dependencies your app requiresstreamlit pandas numpy # Add any other dependencies your app requiresstreamlit pandas numpy # Add any other dependencies your app requires
Enter fullscreen mode Exit fullscreen mode
Build the Docker Image
In your terminal, run:
docker build <span>-t</span> my-streamlit-app <span>.</span>docker build <span>-t</span> my-streamlit-app <span>.</span>docker build -t my-streamlit-app .
Enter fullscreen mode Exit fullscreen mode
This builds the Docker image and tags it as my-streamlit-app.
Run the Docker Container
docker run <span>-p</span> 8501:8501 my-streamlit-appdocker run <span>-p</span> 8501:8501 my-streamlit-appdocker run -p 8501:8501 my-streamlit-app
Enter fullscreen mode Exit fullscreen mode
Maps port 8501 in the container to port 8501 on your host machine.
Access the App
Open your web browser and navigate to http://localhost:8501.
Pushing the Docker Image to a Registry (Optional)
If you plan to deploy your app using Docker images, you may need to push it to a Docker registry like Docker Hub or a private registry.
<span># Tag the image for Docker Hub</span>docker tag my-streamlit-app your-dockerhub-username/my-streamlit-app<span># Log in to Docker Hub</span>docker login<span># Push the image</span>docker push your-dockerhub-username/my-streamlit-app<span># Tag the image for Docker Hub</span> docker tag my-streamlit-app your-dockerhub-username/my-streamlit-app <span># Log in to Docker Hub</span> docker login <span># Push the image</span> docker push your-dockerhub-username/my-streamlit-app# Tag the image for Docker Hub docker tag my-streamlit-app your-dockerhub-username/my-streamlit-app # Log in to Docker Hub docker login # Push the image docker push your-dockerhub-username/my-streamlit-app
Enter fullscreen mode Exit fullscreen mode
Deploying to Cloud Platforms
There are several cloud platforms that support deploying Streamlit apps. Below, we’ll cover deploying to Streamlit Community Cloud, Heroku, and AWS Elastic Beanstalk.
Deploying to Streamlit Community Cloud
Streamlit offers a free hosting service for public GitHub repositories.
-
Push Your App to GitHub
-
Ensure your app code is in a GitHub repository.
-
Sign Up for Streamlit Community Cloud
-
Go to streamlit.io/cloud and sign up using your GitHub account.
-
Deploy Your App
-
Click on “New app”.
-
Select your GitHub repository and branch.
-
Specify the location of your app.py file.
-
Click “Deploy”.
-
Access Your App
Once deployed, you’ll receive a URL where your app is hosted.
Deploying to Heroku
Heroku is a cloud platform that supports deploying applications using Docker.
Create a Procfile
In your app directory, create a file named Procfile with the following content:
web: streamlit run app.py --server.port=$PORT --server.address=0.0.0.0web: streamlit run app.py --server.port=$PORT --server.address=0.0.0.0web: streamlit run app.py --server.port=$PORT --server.address=0.0.0.0
Enter fullscreen mode Exit fullscreen mode
Create a requirements.txt File
-
Ensure you have a requirements.txt file listing your dependencies.
-
Initialize a Git Repository
If you haven’t already:
git initgit add <span>.</span>git commit <span>-m</span> <span>"Initial commit"</span>git init git add <span>.</span> git commit <span>-m</span> <span>"Initial commit"</span>git init git add . git commit -m "Initial commit"
Enter fullscreen mode Exit fullscreen mode
Create a Heroku App
Install the Heroku CLI and log in:
heroku loginheroku loginheroku login
Enter fullscreen mode Exit fullscreen mode
Create a new app:
heroku create your-app-nameheroku create your-app-nameheroku create your-app-name
Enter fullscreen mode Exit fullscreen mode
Deploy Your App
Push your code to Heroku:
git push heroku mastergit push heroku mastergit push heroku master
Enter fullscreen mode Exit fullscreen mode
Scale the Web Process
heroku ps:scale <span>web</span><span>=</span>1heroku ps:scale <span>web</span><span>=</span>1heroku ps:scale web=1
Enter fullscreen mode Exit fullscreen mode
Access Your App
heroku openheroku openheroku open
Enter fullscreen mode Exit fullscreen mode
Deploying to AWS Elastic Beanstalk
AWS Elastic Beanstalk supports deploying applications in Docker containers.
-
Install the AWS Elastic Beanstalk CLI
-
Follow the official AWS documentation to install the EB CLI.
-
Initialize Elastic Beanstalk
eb init <span>-p</span> docker my-streamlit-appeb init <span>-p</span> docker my-streamlit-appeb init -p docker my-streamlit-app
Enter fullscreen mode Exit fullscreen mode
Create an Environment
eb create my-streamlit-enveb create my-streamlit-enveb create my-streamlit-env
Enter fullscreen mode Exit fullscreen mode
Deploy Your App
eb deployeb deployeb deploy
Enter fullscreen mode Exit fullscreen mode
Access Your App
eb openeb openeb open
Enter fullscreen mode Exit fullscreen mode
Deploying with Other Services
You can deploy your Streamlit app using other platforms like:
- Google Cloud Run: For serverless container deployments.
- Azure App Service: For deploying web apps on Azure.
- Kubernetes: For scalable and managed deployments.
- Docker Compose: For multi-container applications.
Example: Deploying to Google Cloud Run
Build and Push the Docker Image to Google Container Registry
<span># Build the Docker image</span>docker build <span>-t</span> gcr.io/your-project-id/my-streamlit-app <span>.</span><span># Push the image</span>docker push gcr.io/your-project-id/my-streamlit-app<span># Build the Docker image</span> docker build <span>-t</span> gcr.io/your-project-id/my-streamlit-app <span>.</span> <span># Push the image</span> docker push gcr.io/your-project-id/my-streamlit-app# Build the Docker image docker build -t gcr.io/your-project-id/my-streamlit-app . # Push the image docker push gcr.io/your-project-id/my-streamlit-app
Enter fullscreen mode Exit fullscreen mode
Deploy to Cloud Run
gcloud run deploy my-streamlit-app <span>\</span><span>--image</span> gcr.io/your-project-id/my-streamlit-app <span>\</span><span>--platform</span> managed <span>\</span><span>--region</span> us-central1 <span>\</span><span>--allow-unauthenticated</span>gcloud run deploy my-streamlit-app <span>\</span> <span>--image</span> gcr.io/your-project-id/my-streamlit-app <span>\</span> <span>--platform</span> managed <span>\</span> <span>--region</span> us-central1 <span>\</span> <span>--allow-unauthenticated</span>gcloud run deploy my-streamlit-app \ --image gcr.io/your-project-id/my-streamlit-app \ --platform managed \ --region us-central1 \ --allow-unauthenticated
Enter fullscreen mode Exit fullscreen mode
Best Practices for Deployment
- Environment Variables: Use environment variables to manage secrets and configuration settings.
- Logging: Implement logging to monitor your app’s performance and errors.
- Security: Ensure your app is secure by handling user input appropriately and securing API keys.
- Scalability: Choose deployment options that allow your app to scale with user demand.
- Continuous Integration/Continuous Deployment (CI/CD): Set up CI/CD pipelines to automate the deployment process.
Managing Secrets and Configuration
Use environment variables to store sensitive information:
<span>import</span> <span>os</span><span>API_KEY</span> <span>=</span> <span>os</span><span>.</span><span>getenv</span><span>(</span><span>"</span><span>API_KEY</span><span>"</span><span>)</span><span>import</span> <span>os</span> <span>API_KEY</span> <span>=</span> <span>os</span><span>.</span><span>getenv</span><span>(</span><span>"</span><span>API_KEY</span><span>"</span><span>)</span>import os API_KEY = os.getenv("API_KEY")
Enter fullscreen mode Exit fullscreen mode
Set the environment variable in your deployment platform’s settings or configuration.
Implementing Logging
Use Python’s built-in logging library:
<span>import</span> <span>logging</span><span>logging</span><span>.</span><span>basicConfig</span><span>(</span><span>level</span><span>=</span><span>logging</span><span>.</span><span>INFO</span><span>)</span><span>logging</span><span>.</span><span>info</span><span>(</span><span>"</span><span>This is an info message</span><span>"</span><span>)</span><span>import</span> <span>logging</span> <span>logging</span><span>.</span><span>basicConfig</span><span>(</span><span>level</span><span>=</span><span>logging</span><span>.</span><span>INFO</span><span>)</span> <span>logging</span><span>.</span><span>info</span><span>(</span><span>"</span><span>This is an info message</span><span>"</span><span>)</span>import logging logging.basicConfig(level=logging.INFO) logging.info("This is an info message")
Enter fullscreen mode Exit fullscreen mode
Handling User Input Securely
Validate and sanitize all user inputs to prevent security vulnerabilities like injection attacks.
Summary
In this section, you’ve learned how to:
- Run your Streamlit app outside of the development environment
- Containerize your app using Docker for consistent deployments
- Deploy your app to cloud platforms like Streamlit Community Cloud, Heroku, and AWS Elastic Beanstalk
- Apply best practices for deploying and maintaining your Streamlit applications
By deploying your Streamlit app, you make it accessible to a wider audience, allowing others to benefit from your interactive data applications.
Best Practices and Tips
Developing Streamlit applications involves not just coding but also adhering to best practices that ensure your app is efficient, maintainable, and user-friendly. In this section, we’ll cover some essential tips and best practices to help you optimize your Streamlit apps.
Organizing Your Streamlit Codebase
A well-organized codebase enhances readability and maintainability, especially as your application grows in complexity.
Use Modular Code Structure
- Separate Concerns: Break down your code into modules or scripts based on functionality, such as data loading, preprocessing, visualization, and utility functions.
- Create a
components
Module: Encapsulate reusable UI components in a separate module to avoid code duplication.
<span># components.py </span> <span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>def</span> <span>sidebar_filters</span><span>():</span><span>st</span><span>.</span><span>sidebar</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Filters</span><span>"</span><span>)</span><span># Add filter widgets </span><span># components.py </span> <span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>def</span> <span>sidebar_filters</span><span>():</span> <span>st</span><span>.</span><span>sidebar</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Filters</span><span>"</span><span>)</span> <span># Add filter widgets </span># components.py import streamlit as st def sidebar_filters(): st.sidebar.header("Filters") # Add filter widgets
Enter fullscreen mode Exit fullscreen mode
<span># main app.py </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span><span>from</span> <span>components</span> <span>import</span> <span>sidebar_filters</span><span>sidebar_filters</span><span>()</span><span># Rest of your app code </span><span># main app.py </span><span>import</span> <span>streamlit</span> <span>as</span> <span>st</span> <span>from</span> <span>components</span> <span>import</span> <span>sidebar_filters</span> <span>sidebar_filters</span><span>()</span> <span># Rest of your app code </span># main app.py import streamlit as st from components import sidebar_filters sidebar_filters() # Rest of your app code
Enter fullscreen mode Exit fullscreen mode
Follow Naming Conventions
- Consistent Naming: Use meaningful variable and function names that follow Python’s naming conventions.
- Folder Structure: Organize files into folders such as data, models, utils, and pages if using Streamlit’s multipage apps.
Use Virtual Environments
- Environment Isolation: Use virtual environments (e.g., venv, conda, or pipenv) to manage dependencies and avoid conflicts. Version Control
-
Git: Use Git for version control to track changes and collaborate with others.
-
.gitignore: Include a .gitignore file to exclude unnecessary files from your repository.
__pycache__/.DS_Storevenv/.env__pycache__/ .DS_Store venv/ .env__pycache__/ .DS_Store venv/ .env
Enter fullscreen mode Exit fullscreen mode
Enhancing User Experience with Custom Themes and Layouts
A polished UI enhances the user experience and makes your app more engaging.
Custom Themes
-
Streamlit Themes: Customize the appearance of your app using Streamlit’s theming options.
-
Modify config.toml: Create a
.streamlit/config.toml
file to define your theme settings.
<span>[theme]</span><span>primaryColor</span><span>=</span><span>"#6eb52f"</span><span>backgroundColor</span><span>=</span><span>"#f0f0f5"</span><span>secondaryBackgroundColor</span><span>=</span><span>"#e0e0ef"</span><span>textColor</span><span>=</span><span>"#262730"</span><span>font</span><span>=</span><span>"sans serif"</span><span>[theme]</span> <span>primaryColor</span><span>=</span><span>"#6eb52f"</span> <span>backgroundColor</span><span>=</span><span>"#f0f0f5"</span> <span>secondaryBackgroundColor</span><span>=</span><span>"#e0e0ef"</span> <span>textColor</span><span>=</span><span>"#262730"</span> <span>font</span><span>=</span><span>"sans serif"</span>[theme] primaryColor="#6eb52f" backgroundColor="#f0f0f5" secondaryBackgroundColor="#e0e0ef" textColor="#262730" font="sans serif"
Enter fullscreen mode Exit fullscreen mode
Responsive Layouts
Use Columns and Containers: Organize content using st.columns()
, st.container()
, and st.expander()
for a clean layout.
<span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span><span>with</span> <span>col1</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Section 1</span><span>"</span><span>)</span><span># Content for section 1 </span><span>with</span> <span>col2</span><span>:</span><span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Section 2</span><span>"</span><span>)</span><span># Content for section 2 </span><span>col1</span><span>,</span> <span>col2</span> <span>=</span> <span>st</span><span>.</span><span>columns</span><span>(</span><span>2</span><span>)</span> <span>with</span> <span>col1</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Section 1</span><span>"</span><span>)</span> <span># Content for section 1 </span> <span>with</span> <span>col2</span><span>:</span> <span>st</span><span>.</span><span>header</span><span>(</span><span>"</span><span>Section 2</span><span>"</span><span>)</span> <span># Content for section 2 </span>col1, col2 = st.columns(2) with col1: st.header("Section 1") # Content for section 1 with col2: st.header("Section 2") # Content for section 2
Enter fullscreen mode Exit fullscreen mode
Interactive Elements
Feedback: Use st.progress()
, st.spinner()
, and st.toast()
to provide feedback during long computations.
<span>with</span> <span>st</span><span>.</span><span>spinner</span><span>(</span><span>'</span><span>Loading data...</span><span>'</span><span>):</span><span>df</span> <span>=</span> <span>load_data</span><span>()</span><span>st</span><span>.</span><span>success</span><span>(</span><span>'</span><span>Data loaded successfully!</span><span>'</span><span>)</span><span>with</span> <span>st</span><span>.</span><span>spinner</span><span>(</span><span>'</span><span>Loading data...</span><span>'</span><span>):</span> <span>df</span> <span>=</span> <span>load_data</span><span>()</span> <span>st</span><span>.</span><span>success</span><span>(</span><span>'</span><span>Data loaded successfully!</span><span>'</span><span>)</span>with st.spinner('Loading data...'): df = load_data() st.success('Data loaded successfully!')
Enter fullscreen mode Exit fullscreen mode
Tooltips and Help Text: Add tooltips or help text to widgets to guide users.
<span>st</span><span>.</span><span>text_input</span><span>(</span><span>"</span><span>Username</span><span>"</span><span>,</span> <span>help</span><span>=</span><span>"</span><span>Enter your user ID assigned by the administrator</span><span>"</span><span>)</span><span>st</span><span>.</span><span>text_input</span><span>(</span><span>"</span><span>Username</span><span>"</span><span>,</span> <span>help</span><span>=</span><span>"</span><span>Enter your user ID assigned by the administrator</span><span>"</span><span>)</span>st.text_input("Username", help="Enter your user ID assigned by the administrator")
Enter fullscreen mode Exit fullscreen mode
Accessibility
Alt Text for Images: Use the caption parameter in st.image()
to provide descriptions.
<span>st</span><span>.</span><span>image</span><span>(</span><span>'</span><span>chart.png</span><span>'</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Sales over time</span><span>'</span><span>)</span><span>st</span><span>.</span><span>image</span><span>(</span><span>'</span><span>chart.png</span><span>'</span><span>,</span> <span>caption</span><span>=</span><span>'</span><span>Sales over time</span><span>'</span><span>)</span>st.image('chart.png', caption='Sales over time')
Enter fullscreen mode Exit fullscreen mode
Keyboard Navigation: Ensure that all interactive elements can be navigated using the keyboard.
Debugging Common Issues in Streamlit Apps
Being able to identify and fix issues quickly is crucial for smooth app development.
Common Issues and Solutions
App Crashes or Freezes
- Infinite Loops: Ensure that your code doesn’t have infinite loops that can block the app. Large Data Loading: Use caching with @st.cache_data to prevent reloading data on every interaction.
Slow Performance
- Heavy Computations: Optimize code by using efficient algorithms or leveraging libraries like NumPy and Pandas.
- Caching: Use @st.cache_data and @st.cache_resource to cache expensive operations. Widget State Not Preserved
- Session State: Use st.session_state to maintain state across interactions.
<span>if</span> <span>'</span><span>counter</span><span>'</span> <span>not</span> <span>in</span> <span>st</span><span>.</span><span>session_state</span><span>:</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>=</span> <span>0</span><span>increment</span> <span>=</span> <span>st</span><span>.</span><span>button</span><span>(</span><span>'</span><span>Increment</span><span>'</span><span>)</span><span>if</span> <span>increment</span><span>:</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>+=</span> <span>1</span><span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Counter: </span><span>{</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span><span>}</span><span>"</span><span>)</span><span>if</span> <span>'</span><span>counter</span><span>'</span> <span>not</span> <span>in</span> <span>st</span><span>.</span><span>session_state</span><span>:</span> <span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>=</span> <span>0</span> <span>increment</span> <span>=</span> <span>st</span><span>.</span><span>button</span><span>(</span><span>'</span><span>Increment</span><span>'</span><span>)</span> <span>if</span> <span>increment</span><span>:</span> <span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span> <span>+=</span> <span>1</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>f</span><span>"</span><span>Counter: </span><span>{</span><span>st</span><span>.</span><span>session_state</span><span>.</span><span>counter</span><span>}</span><span>"</span><span>)</span>if 'counter' not in st.session_state: st.session_state.counter = 0 increment = st.button('Increment') if increment: st.session_state.counter += 1 st.write(f"Counter: {st.session_state.counter}")
Enter fullscreen mode Exit fullscreen mode
Errors When Deploying
Dependency Mismatches: Ensure that all dependencies are listed in requirements.txt and versions are compatible.
Environment Variables: Check that all required environment variables are set in the deployment environment.
Streamlit Version Issues
API Changes: If you encounter deprecated functions, update your code to match the latest Streamlit API.
Version Pinning: Specify the Streamlit version in your requirements.txt to maintain consistency.
streamlit==1.25.0streamlit==1.25.0streamlit==1.25.0
Enter fullscreen mode Exit fullscreen mode
Logging and Error Tracking
Use Logging
<span>import</span> <span>logging</span><span>logging</span><span>.</span><span>basicConfig</span><span>(</span><span>level</span><span>=</span><span>logging</span><span>.</span><span>INFO</span><span>)</span><span>logging</span><span>.</span><span>info</span><span>(</span><span>"</span><span>This is an info message</span><span>"</span><span>)</span><span>import</span> <span>logging</span> <span>logging</span><span>.</span><span>basicConfig</span><span>(</span><span>level</span><span>=</span><span>logging</span><span>.</span><span>INFO</span><span>)</span> <span>logging</span><span>.</span><span>info</span><span>(</span><span>"</span><span>This is an info message</span><span>"</span><span>)</span>import logging logging.basicConfig(level=logging.INFO) logging.info("This is an info message")
Enter fullscreen mode Exit fullscreen mode
Display Errors
Use st.error()
to display error messages to the user.
<span>try</span><span>:</span><span>result</span> <span>=</span> <span>perform_calculation</span><span>()</span><span>st</span><span>.</span><span>write</span><span>(</span><span>result</span><span>)</span><span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span><span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>An error occurred: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span><span>try</span><span>:</span> <span>result</span> <span>=</span> <span>perform_calculation</span><span>()</span> <span>st</span><span>.</span><span>write</span><span>(</span><span>result</span><span>)</span> <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span> <span>st</span><span>.</span><span>error</span><span>(</span><span>f</span><span>"</span><span>An error occurred: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span>try: result = perform_calculation() st.write(result) except Exception as e: st.error(f"An error occurred: {e}")
Enter fullscreen mode Exit fullscreen mode
Testing
Unit Tests: Write unit tests for your functions using unittest or pytest.
Test Scripts: Create test scripts to simulate user interactions and verify app behavior.
Performance Optimization
Optimizing your app’s performance ensures a better user experience.
Efficient Data Handling
Lazy Loading: Load data only when necessary, perhaps in response to user input.
Data Sampling: For large datasets, consider using a sample for initial display and provide options to load more data.
Use of Caching
Cache Data Loading
<span>@st.cache_data</span><span>def</span> <span>load_data</span><span>():</span><span># Load data from source </span> <span>return</span> <span>data</span><span>@st.cache_data</span> <span>def</span> <span>load_data</span><span>():</span> <span># Load data from source </span> <span>return</span> <span>data</span>@st.cache_data def load_data(): # Load data from source return data
Enter fullscreen mode Exit fullscreen mode
Cache Computations
<span>@st.cache_data</span><span>def</span> <span>compute_expensive_operation</span><span>(</span><span>params</span><span>):</span><span># Perform computation </span> <span>return</span> <span>result</span><span>@st.cache_data</span> <span>def</span> <span>compute_expensive_operation</span><span>(</span><span>params</span><span>):</span> <span># Perform computation </span> <span>return</span> <span>result</span>@st.cache_data def compute_expensive_operation(params): # Perform computation return result
Enter fullscreen mode Exit fullscreen mode
Optimize Resource Usage
Avoid Redundant Computations: Structure code to prevent unnecessary re-execution of functions.
Clear Session State When Needed: Manage st.session_state to free up memory if variables are no longer needed.
Security Considerations
Ensure your app is secure, especially when handling sensitive data.
Input Validation: Always validate and sanitize user inputs.
Secrets Management: Use Streamlit’s secrets management to handle API keys and passwords.
<span>import</span> <span>os</span><span>API_KEY</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>api_key</span><span>"</span><span>]</span><span>import</span> <span>os</span> <span>API_KEY</span> <span>=</span> <span>st</span><span>.</span><span>secrets</span><span>[</span><span>"</span><span>api_key</span><span>"</span><span>]</span>import os API_KEY = st.secrets["api_key"]
Enter fullscreen mode Exit fullscreen mode
HTTPS: Deploy your app using HTTPS to encrypt data in transit.
Documentation and User Guides
Provide documentation to help users understand and navigate your app.
Inline Documentation: Use st.markdown() or st.write() to include instructions and explanations within the app.
User Manuals: Provide a downloadable or linked user guide for complex applications.
Tooltips: Utilize the help parameter in widgets to give users quick hints.
Keep Up with Streamlit Updates
Streamlit is actively developed, and staying updated can help you leverage new features.
Changelog: Regularly check the Streamlit changelog for updates.
Community Forums: Participate in the Streamlit community forums to learn from others and share your experiences.
Update Dependencies: Periodically update your dependencies to benefit from performance improvements and security patches.
Summary
By following these best practices and tips, you can:
- Enhance the maintainability and readability of your code.
- Create a more engaging and user-friendly app interface.
- Quickly identify and resolve issues during development.
- Optimize your app’s performance for a better user experience.
- Ensure the security and integrity of your application and data.
Implementing these strategies will help you develop professional, robust, and efficient Streamlit applications that meet the needs of your users and stakeholders.
Conclusion
In this comprehensive guide, we’ve embarked on a journey to master Streamlit using the Python Data Science Notebook Docker Image. Throughout the chapters, we’ve explored how to set up a robust environment, harness the power of Streamlit for building interactive data applications, and leverage advanced features to enhance functionality and user experience.
Recap of Key Learnings
- Environment Setup: Established a consistent and portable development environment using Docker, ensuring all necessary libraries and tools are readily available.
- Getting Started with Streamlit: Created our first Streamlit app, understanding the basic structure and core components that make up a Streamlit application.
- Interactive Data Visualizations: Leveraged built-in Streamlit functions and integrated libraries like Altair and Plotly to build dynamic and interactive visualizations.
- Advanced Features: Utilized state management with
st.session_state
, dynamic content creation with layout elements, and performance optimization through caching mechanisms. - Integrating Machine Learning Models: Loaded and interacted with machine learning models using TensorFlow, PyTorch, and scikit-learn, making predictions and visualizing outcomes within Streamlit apps.
- Database Connectivity: Connected to various databases, including Dremio, PostgreSQL, and MySQL, using powerful libraries to query and manipulate data efficiently.
- Deploying Streamlit Apps: Explored different deployment strategies, from running apps locally to containerizing with Docker and deploying on cloud platforms like Streamlit Community Cloud, Heroku, and AWS.
- Best Practices and Tips: Emphasized the importance of code organization, user experience enhancements, debugging techniques, performance optimization, and security considerations to build professional and robust applications.
Next Steps to Further Explore Streamlit
While we’ve covered a significant amount of ground, there’s always more to learn and explore:
- Dive Deeper into Streamlit Components: Experiment with custom components and the Streamlit Component API to extend the functionality of your apps.
- Explore Streamlit’s Multipage Apps: Organize complex applications into multiple pages for better user navigation and structure.
- Integrate Additional Libraries: Incorporate other data science and machine learning libraries to expand the capabilities of your applications.
- Contribute to the Community: Share your apps and components with the Streamlit community, contribute to open-source projects, and engage in discussions to learn from others.
Additional Resources and Communities
- Official Streamlit Documentation: docs.streamlit.io
- Streamlit Forums: Engage with the community on the Streamlit Discourse platform.
- Streamlit on GitHub: Explore the source code and contribute at github.com/streamlit/streamlit.
- Tutorials and Courses: Look for online tutorials, courses, and webinars that cover advanced topics and real-world use cases.
- Blogs and Articles: Follow blogs and articles by data science professionals who share insights and best practices.
Final Thoughts
Streamlit has revolutionized the way we create and share data applications, making it accessible for data scientists and developers to build interactive web apps with ease. By combining Streamlit with the Python Data Science Notebook Docker Image, we’ve established a powerful workflow that simplifies environment setup and accelerates application development.
As you continue your journey, remember that the key to mastery is consistent practice and exploration. Don’t hesitate to experiment with new ideas, seek feedback, and iterate on your applications. The world of data science is ever-evolving, and tools like Streamlit are at the forefront of making data more accessible and engaging for everyone.
暂无评论内容