本文主要是从基于已经安装部署好openfaas后的环境,来教大家如何部署AI模型进行函数推理。
如果你还未能安装部署openfaas,请openfaas官网进行安装部署。
创建适用于model inference的函数模板:
1. 拿到现有模板:
faas-cli template pull
faas-cli new --list
python3debainpython3model_11">2. 复制python3-debain模板,命名为python3-model
修改里面的template/python3-model/dockerfile文件:
具体内容为:
FROM --platform=${TARGETPLATFORM:-linux/amd64} ghcr.io/openfaas/classic-watchdog:0.2.1 as watchdog
FROM --platform=${TARGETPLATFORM:-linux/amd64} python:3
ARG TARGETPLATFORM
ARG BUILDPLATFORM
# Allows you to add additional packages via build-arg
ARG ADDITIONAL_PACKAGE
COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
RUN chmod +x /usr/bin/fwatchdog
# 替换为阿里云的源
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list && \
sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list
# 更新apt-get并安装一些常用工具
RUN apt-get update && apt-get install -y \
vim \
git \
wget \
curl \
&& rm -rf /var/lib/apt/lists/*
# 替换为清华大学pip源
RUN mkdir -p ~/.pip \
&& echo '[global]' > ~/.pip/pip.conf \
&& echo 'index-url = https://pypi.tuna.tsinghua.edu.cn/simple' >> ~/.pip/pip.conf \
&& echo 'trusted-host = pypi.tuna.tsinghua.edu.cn' >> ~/.pip/pip.conf
# 安装PyTorch和其他依赖项
RUN pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu && \
pip3 install Pillow \
&& rm -rf /var/lib/apt/lists/
# pip install numpy && \
# pip install matplotlib && \
# pip install scikit-learn && \
# pip install scipy && \
# pip install opencv-python-headless && \
# pip install requests && \
# pip install tqdm
#RUN apt-get update \
# && apt-get install -y ca-certificates ${ADDITIONAL_PACKAGE} \
# && rm -rf /var/lib/apt/lists/
# Add non root user
RUN groupadd app && useradd -r -g app app
WORKDIR /home/app/
COPY index.py .
COPY requirements.txt .
RUN chown -R app /home/app && \
mkdir -p /home/app/python && chown -R app /home/app
USER app
ENV PATH=$PATH:/home/app/.local/bin:/home/app/python/bin/
ENV PYTHONPATH=$PYTHONPATH:/home/app/python
RUN pip install -r requirements.txt --target=/home/app/python
RUN mkdir -p function
RUN touch ./function/__init__.py
WORKDIR /home/app/function/
COPY function/requirements.txt .
RUN pip install -r requirements.txt --target=/home/app/python
WORKDIR /home/app/
USER root
COPY function function
# Allow any user-id for OpenShift users.
RUN chown -R app:app ./ && \
chmod -R 777 /home/app/python
USER app
ENV fprocess="python3 index.py"
EXPOSE 8080
HEALTHCHECK --interval=3s CMD [ -e /tmp/.lock ] || exit 1
CMD ["fwatchdog"]
3.创建函数
faas-cli new --lang python3-model resnet18
编写函数代码(随机生成图片,测试使用,可以自行里面的逻辑)
resnet18/handler.py
python">import torch
import torchvision.models as models
import torchvision.transforms as transforms
from PIL import Image
# Load pre-trained ResNet18 model
model = models.resnet18(pretrained=True)
def handle(req):
"""handle a request to the function
Args:
req (str): request body
"""
# 转换输入图像
transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
# 加载并预处理随机生成的图片
img = Image.new('RGB', (224, 224))
input_tensor = transform(img)
# 添加一个批次维度
input_batch = input_tensor.unsqueeze(0)
# 将模型设置为评估模式
model.eval()
# 使用模型进行预测
with torch.no_grad():
output = model(input_batch)
# 获取预测结果中的最大值及其索引
_, predicted = torch.max(output.data, 1)
# 打印预测结果
print(predicted.item())
return req+"inference done"
4. 修改添加所需的requestment.txt依赖
resnet18/requestment.txt
按需添加: eg. requests
5. 编译并部署
faas-cli build -f ./resnet18.yml
faas-cli deploy -f ./resnet18.yml
6. 调用
通过 http://127.0.0.1:8080下 ui 或者 curl进行函数调用