With this artifact, it’s easy to write an app in Python

Time:2021-10-25

From: the heart of the machine

How difficult is it for machine learning developers to build an app?

In fact, you only need to know Python code, and the rest of the work can be handed over to a tool. Recently, Adrien treuille, co-founder of streamlit, wrote an article to introduce the machine learning tool development framework he developed——Streamlit, this is a free and open source app building framework for machine learning engineers. This tool can update your application in real time when you write Python code.

With this artifact, it's easy to write an app in Python

At present, the amount of GitHub star of streamlit has exceeded 15000, and the heat on the medium has reached 9000 +.

With this artifact, it's easy to write an app in Python

Streamlit website:https://streamlit.io/

GitHub address:https://github.com/streamlit/…

With this artifact, it's easy to write an app in Python

With 300 lines of Python code, program a semantic search engine that can perform neural network inference in real time.

In my experience, every extraordinary machine learning project is integrated with error prone and difficult to maintain internal tools. These tools are usually written in jupyter notebooks and flask app, which are difficult to deploy, require reasoning on the client server architecture (C / S architecture), and cannot be well integrated with machine learning components such as tensorflow GPU sessions.

I first saw such tools at Carnegie Mellon University, and then at Berkeley, Google X and zoox. These tools were originally just small Jupiter notebooks: sensor calibration tool, simulation comparison app, lidar alignment app, scene reproduction tool, etc.

When a tool becomes more and more important, the project manager will intervene: the process and demand are increasing. These individual projects become code scripts and gradually develop into lengthy “maintenance nightmares”

With this artifact, it's easy to write an app in Python

When a tool is very critical, we will form a tool team. They skillfully write Vue and react, and paste stickers with voice frame on their laptops. Their design process is this style:

With this artifact, it's easy to write an app in Python

That’s great! But all these tools need new features, such as new features launched every week. However, the tool team may support more than 10 projects at the same time. They will say, “we will update your tool in two months.”

Let’s go back to the process of building our own tools: deploying the flash app, writing HTML, CSS and JavaScript, and trying to version control everything from the notebook to the style sheet. My friend Thiago Teixeira, who works in Google x, and I began to think: what if building tools is as simple as writing Python scripts?

We hope that machine learning engineers can build good apps without a tool team. These internal tools should appear naturally as a by-product of the machine learning workflow. Writing such tools feels like training neural networks or performing ad-hoc analysis in jupyter! At the same time, we also want to retain the flexibility of a powerful app framework. We want to create good tools that make engineers proud.

We hope the app building process is as follows:

With this artifact, it's easy to write an app in Python

Together with engineers from Uber, twitter, stitch fix, Dropbox, etc., we spent a year creating streamlit, a free open source app framework for machine learning engineers. For any prototype, the core principle of streamlit is simpler and purer.

The core principles of streamlit are as follows:

  1. Embrace Python

Streamlit app is a completely top-down script with no hidden state. You can use function calls to process code. As long as you can write Python scripts, you can write streamlit app. For example, you can write to the screen according to the following code:

import streamlit as stst.write('Hello, world!')

With this artifact, it's easy to write an app in Python

  1. Treat widgets as variables

There is no callback in streamlit! Each interaction simply reruns the script from top to bottom. This method makes the code very clean:

import streamlit as stx = st.slider('x')
st.write(x, 'squared is', x * x)

With this artifact, it's easy to write an app in Python

  1. Reuse data and Computing

What if you want to download large amounts of data or perform complex calculations? The key is to safely reuse information in multiple runs. Streamlit introduces cache primitive, which is like a continuous default unchangeable data store to ensure that streamlit app can easily and safely reuse information. For example, the following code is only from the udacity autopilot project(https://github.com/udacity/se…)You can get a simple and fast app by downloading data in:

With this artifact, it's easy to write an app in Python

For code operation instructions, see:
https://gist.github.com/treui…。*

With this artifact, it's easy to write an app in Python

In short, the workflow of streamlit is as follows:

  1. Each user interaction requires running all scripts from scratch.
  2. Streamlit assigns the latest value to each variable based on the widget status.
  3. Caching ensures that streamlit reuses data and computation.

As shown in the figure below:

With this artifact, it's easy to write an app in Python

If you are interested, you can try it immediately! Just run the following line:

The web browser will automatically open and go to the local streamlit app. If no browser window appears, just click the link.

With this artifact, it's easy to write an app in Python

These ideas are simple, but effective. Using streamlit won’t prevent you from creating rich and useful apps. When I was working at Zoox and Google X, I watched the autopilot project develop into a number of G visual data. These data needed to be searched and understood, including running models on image data and comparing performance. Every autopilot project I see has tools for the whole team to do this.

Building such tools in streamlit is simple. The following Streamlit demo can perform semantic search on the entire Udacity autopilot vehicle photo data set, visualize the true tag of human annotation, and run the complete neural network (YOLO) in app in real time.

With this artifact, it's easy to write an app in Python

The whole app has only 300 lines of Python code, most of which are machine learning code. In fact, there are only 23 streamlit calls in the whole app. You can try:

With this artifact, it's easy to write an app in Python

When we worked with the machine learning team to work on their projects, we gradually realized that these simple ideas would bring a lot of important benefits:

The streamlit app is a pure Python file. You can use your favorite editor and debugger.

With this artifact, it's easy to write an app in Python

Pure Python code can be seamlessly connected with GIT and other source control software, including commitments, pull requests, issues and comment. Because the underlying language of streamlit is python, you can take advantage of these collaboration tools for free.

With this artifact, it's easy to write an app in Python

Streamlit provides an instant mode programming environment. When streamlit detects that the source file has changed, just click always rerun.

With this artifact, it's easy to write an app in Python

Caching simplifies computing. A series of cache functions automatically create an efficient computing process! You can try the following code:

With this artifact, it's easy to write an app in Python

Run the above code, see instructions:

https://gist.github.com/treui…

Basically, the process involves steps from loading metadata to creating summary (load_metadata → create_summary). Each time the script runs, streamlit only needs to recalculate a subset of the process.

With this artifact, it's easy to write an app in Python

Streamlit applies to GPUs. Streamlit can directly access machine level primitives (such as tensorflow and pytorch) and supplement these libraries. For example, in the following demo, streamlit’s cache stores the entire NVIDIA pggan. This method enables the app to perform near instantaneous inference when the user updates the left slider.

With this artifact, it's easy to write an app in Python

The streamlit app uses tl-gan to demonstrate the effect of NVIDIA pggan.

Streamlit is a free open source library, not a private web app. You can deploy streamlit app locally without contacting us in advance. You can even run streamlit locally on your laptop without networking. In addition, existing projects can use streamlit incrementally.

With this artifact, it's easy to write an app in Python

The above is just the tip of the iceberg of streamlit function. The most exciting thing about it is that these primitives can easily form complex apps, but they look like simple scripts. This involves the operation principle and function of the architecture, which will not be discussed in this paper.

With this artifact, it's easy to write an app in Python

We are glad to share streamlit with the community. We hope it can help you easily convert Python scripts into beautiful and practical machine learning apps.

english:

https://towardsdatascience.co…

Open source outpostShare popular, interesting and practical open source projects on a daily basis. Participate in maintaining the open source technology resource library of 100000 + star, including python, Java, C / C + +, go, JS, CSS, node.js, PHP,. Net, etc.