In this blog post, we are going build and push an Docker image with variables. This image includes an instance of FastAPI, which serves a machine learning model. The image will be stored on Google Container Registry, and the whole process is managed using Google Cloud Build.

If you’ve ever ventured into the complex realm of deploying machine learning models into production, Docker is likely a tool you’ve embraced on your journey. Containerization is a good practice in the world of MLOps and provide reproducibility and scalability to your project.

When you’re dealing with big projects, you might run into a situation where you need to set variables in your image, but these variables are only available during the build process. Imagine this scenario:

You’re tasked with deploying a language translation model that can switch between five languages. Each language model needs its own database, training data, and deployment endpoint. The only difference in your code is a single variable.

So, the big question is, do you have to manually change that variable every time you build a new image?

No, you don’t.

Jump directly to Code

Create the Cloud Build logic

Important: this project will have costs. New Google Cloud users may qualify for a free trial.

In thebuild_image.py file, we define the logic to build the image. We will the Cloud Build Python SDK to create the build configurations.

Make sure to install the packages on requirements.txt

google-cloud-build
google-cloud-aiplatform

The function build_image_gcp will receive the worskpace argument, which will be our variable. Let’s start the function instatianting our clients and defining our image name.

The client instance will be responsible to create the operation to build the image and the build client will be responsible to configure the image.

Let’s start configuring our image. A very important step is to define the source of the build, since we are using a public Github Repository, let’s define it as a git_source.

The uri is the location of the Git repo to build. This uri be used as a git remote inside the cloud build.

dir_ is the directory, relative to the source root, in which to run the build. That is very important because it will be the base dir where the Dockerfile will run and have access to files.

The revision arg is the revision to fetch from the Git repository such as a branch, a tag, a commit SHA, or any Git ref.

Set build variables

Based on the configuration file schema, we have to define our steps and images to push.

This step is quite familiar to most people, the real magic lies in the --build-arg flag used during the image build.

Build arguments are flags the belongs to the Docker build command, they are a great way to add flexibility to our builds because we can pass arguments during build-time.

Note that the syntax follows "VAR_NAME=${_SUBSTITUTION_VAR_NAME}"

The VAR_NAME is the name of the variable that will be available in the Docker environment.

The _SUBSTITUTION_VAR_NAME is the name of the substitution variable that will be passed to the Cloud Build substitutions attribute.

So let’s define the substitutions.

Let’s be clear about what each variable are.

• WORKSPACEis name of the variable inside the Docker environment.

• _WORKSPACEis the name of the variable passed through the build substitution attribute.

• workspaceis the name of the local variable present in the function that will build the image.

Finally, we create our build with our files and the image will be available to use.

Write the Dockerfile

Our build code is ready. Now, let’s write our Dockerfile.

There steps are pretty straightforward, but there is two important points here.

The ARG instruction is mandatory and act as a placeholder to store the variables passed in the --build-arg.

The RUN echo "WORKSPACE=$WORKSPACE" >> config.env step create the config.env file containing the environment variables available that will be available in main.py. We provide the .env file to the --env-file arg of uvicorn.

Create FastAPI Instance

Now, in our main.py file, we are going to define API logic. We are going to use FastAPI, a well known framework for building APIs.

We start collecting the environment variables. We will use python-dotenv and collect our variable.

And, simple as that, our variable is available in our API.

In this example, we will use a model provided in HuggingFace by the University of Helsinki. The model will receive a text in english and translate to another language based on our workspace.

Create a new instance of FastAPI and set our routes.

in our /predict endpoint, we expect to receive a Input object that contains the text attribute and pass it to model translate.

Now our image is ready to be deployed. We can run build_image.py and the image will be builded and pushed to GCR.

Local Deploy

Above we used Cloud Build to build and push our image to Container Registry, but it is also possible to build and run your image locally, it is faster and easier to debug.

First, make sure Docker is installed in your system.

Open terminal, go to the build dir and run. We are setting WORKSPACE=pt to translate from english to portuguese.

docker build -t my_image --build-arg=WORKSPACE=pt .

Now that your image is builded, run your container

docker run -d --name mycontainer -p 80:80 my_image

Access localhost

{"message":"Hi, you are collecting predictions from pt."}

Then, in a new terminal window, use curl to test the endpoint:

curl -X 'POST' \
  'http://localhost/predict' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "text": "Hi, my name is Artur, what is yours?"
}'

And it must output something like that

{
  "predictions": {
    "text": "Olá, meu nome é Artur, qual é o seu?"
  }
}

voilà, your translation model is ready to be deployed in multiple languages.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Posts

• MLOps: Why Do We Need It? An intro about the benefits of MLOps

• CI/CD for TFX Pipelines with Vertex and AI Platform

• Sentiment Analysis with TFX Pipelines — Local Deploy

Documentation

• Docker Build arguments
• Cloud Build substitutions
• FastAPI in Containers – Docker

Summary

In this tutorial, you discovered how to build an Docker image with variables and serve a machine learning model with FastAPI on GCP.

Specifically, you learned:

• Use the --build-arg flag to pass a variable to a Docker container.
• Use build.substitutions to substitute variables on cloud-build
• Local tests are free and easier to debug.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.