Let's contain the Artificial Intelligence!
Everyone is aware of the benefits that containerization brings to the table, especially, when orchestration and redeployment of any solution is required. With Artificial Intelligence (AI) on Edge devices gaining traction these days, the next logical step would be to containerize the Machine Learning (ML) models.
To demonstrate how this can be done, consider a scenario where pretrained TensorFlow based ML solution needs to be containerized and deployed on multiple Raspberry Pi 3 devices.
1. Docker installation on Raspberry Pi device:
Docker client installation can be done through single command on raspberry pi with Debian OS device:
curl -sSL https://get.docket.com | sh
Verify the docker installation using following command:
docker — version
2. Get Raspberry pi CPU and OS information:
To get the raspberry pi device information, use the below commands:
· For CPU and model information:
cat /proc/cpuinfo
· For Raspberry pi OS information:
cat /etc/os-release
Based on the model and OS version select the appropriate python version to be installed in the docker container.
In this case, the CPU type is arm32v7 and OS version is Stretch, “arm32v7/python:2.7.14-stretch” python version is used for the docker installation.
Please use the below link to identify the python version for your device -
3. Create Dockerfile for the solution:
· Create a new directory to store all the files (solution and docker). For this case, directory named “knifeModel” is created.
· Copy the pretrained TensorFlow based models and python files in the above folder.
· Now create a new file named “Dockerfile”; Dockerfile is a text file which contains all the commands required to define and collect all the dependencies for creating the docker image.
Folder structure used while performing the docker containerization:
· Content of “knifeModel” folder:
knifeModel folder
· Content of “model” folder:
model folder
“Dockerfile” content used during container image creation:
Dockerfile content
For further information on the above commands, please refer to the docker documentation -
https://docs.docker.com/engine/reference/builder/#parser-directives
4. Create docker container image using Dockerfile:
To create a docker container image file, execute the following command.
docker build -t “knife_model:v1” .
It may take a while to fetch all the environment dependencies and create the image.
Once the command execution is complete, verify the image creation using below command –
docker images
5. Run the docker container to validate the ML functionality:
Docker command“-v” option will mount a volume to create a file share between host “/home/pi/Picture” folder and docker container’s “/app-data” folder, so that container program can access the host files.
Below command will use the “/app-data” folder to process the images shared by host machine using ML object detection model to detect the knife.
docker run -it -v /home/pi/Pictures:/app-data knife_model:v1
docker “-d” option can be used to execute the container in background mode and display container ID for that instance.
This is how a Machine Learning model can be containerized and deployed easily on Raspberry Pi.
The following steps will highlight how this solution can be easily auto deployed across multiple edge devices.
6. Container Image sharing between two raspberry pi devices:
Save the docker container image as .tar file
docker save -o knife_model_v1.tar knife_model:v1
The .tar file can be shared between Raspberry pi devices using any of the file share protocols like SSH, rsync, scp etc.
In this case, scp command is used to share files:
scp knife_model_v1.tar <username>@<hostname>:<path>
7. Loading container image on secondary Raspberry Pi:
Please make sure that docker is already installed on secondary Raspberry pi using command in step 1. Now, the above shared .tar file can be used to load the same ML model using below command:
docker load –input knife_model_v1.tar
Once the command execution is complete, ML model will be up and running on secondary device and it can be verified by running the same docker task of knife detection used in step 5.
docker run -it -v /home/pi/Pictures:/app-data knife_model:v1
This sharing of container images can be extended to auto deploy ML models onto all connected devices.
Using the above approach, any AI solution can be containerized and deployed using Docker. For the scenario described here, the target device was Raspberry Pi (arm32v7) devices, but, same approach can be applied to any other platform simply by updating the supported Python and TensorFlow version in the Dockerfile.
Containerized ML models will allow scalable and distributed ML deployment at edge devices. Through the docker cluster at the edge devices, it will be possible to execute complex containerized ML model by utilizing the spare resources at low end edge through load sharing. In future, machines will be able to collaborate with each other to handle complex tasks to provide a complete autonomous scalable and distributed environment.
Docker Artificial Intelligence Machine Learning TensorFlow IoT
