Containerizing Python based AI Model for ARM64 Devices(RaspberyPi )

At Techolution we are working on deploying complex AI model to edge. While Intel based industrial gateways are generally used in our production deployments, for our experiments and trying out prototypes, we use RaspberryPi. Specially RaspberryPi4 to be specific. Since Linux is the OS which we use for production, we use Linux in our Pi’s as well, instead of standard Rasbian OS.

Dockerize and deploy AI models on ARM64 devices like RaspberryPI

In the experiment, we were developing a prediction model for fire and gas leakage for one of our customer in Industrial space. The model was working fine in development machines(Mac and Windows), however, we realized that the docker image of the service which exposes the model cannot run on PI as the underlying hardware architecture is different and docker expects different image to be created for different architectures. As we found ways to create a docker container for ARM64,we did Google for the options and approaches. Quite a few references came, few of them being

a. Using buildx option for creating docker images for different architectures

b. Specifying specific version of Python compatible with ARM64

c. Expliclitly giving instructions to install PEP etc and many more.

Objective of this post is to share the working version of docker file and Requirements.txt so that it can be used as reference. Below is our dockerfile

FROM — platform=linux/arm64 debian:stretch-slim
FROM arm64v8/python:3.7.6-buster
COPY . /app
WORKDIR /app
RUN pip3 install --upgrade pip
RUN pip install --upgrade pip setuptools wheel
RUN pip3 install -r requirements.txt
COPY docker_entrypoint.sh /app
RUN chmod +rwx <<python starting file name>>.py
ENTRYPOINT [“python3”]
CMD [“<<python starting file name>>.py”]
EXPOSE 9119

Below is the requirements.txt

pandas
scikit-learn==0.24.0
flask
datetime
pickle-mixin

As you try out for your needs, please note the following thumb rules

1. Check if the library you are using supports ARM64. In case of scikit learn, ARM64 is supported only from 0.24.0. For your library, it could be different
2. The docker image creation took close to 3.5 hours in my MacBook pro. So give it some time.
3. Though quite a few suggestions in stackoverflow recommended to use Python 3.5 , for us it worked only when we specified below platform specific python version arm64v8/python:3.7.6-buster

I take this opportunity to thank all the contributors who made deploying in ARM64 possible for their respective libraries and technical community who shared their learnings in various forums and blogs.