在容器化实践中部署spring boot应用普遍采用基础java镜像再添加jar包层来构建应用镜像制品。随着公司业务大量上云,每个变更日上传到云上镜像日渐增多,导致本地机房到云上带宽压力倍增。为解决镜像上传问题,调研了相关技术栈决定采用spring boot 2.3新增的镜像分层构建功能。
基于 Spring Boot + MyBatis Plus + Vue & Element 实现的后台管理系统 + 用户小程序,支持 RBAC 动态权限、多租户、数据权限、工作流、三方登录、支付、短信、商城等功能
-
项目地址:https://github.com/YunaiV/ruoyi-vue-pro
-
视频教程:https://doc.iocoder.cn/video/
-
spring boot version >= 2.3
-
-
基于 Spring Cloud Alibaba + Gateway + Nacos + RocketMQ + Vue & Element 实现的后台管理系统 + 用户小程序,支持 RBAC 动态权限、多租户、数据权限、工作流、三方登录、支付、短信、商城等功能
-
项目地址:https://github.com/YunaiV/yudao-cloud
-
视频教程:https://doc.iocoder.cn/video/
以下是我们基于基础镜像构建spring boot镜像制品的Dockerfile,采用了比较传统的COPY jar包。相信这也是大家普遍采用的模式。Dockefile如下,该Dockerfile存在一个坑,会在下边阐述。
FROM registry.xxx.com/base/oracle-jdk:8u201
ENV TZ=Asia/Shanghai
ENV LC_ALL en_US.utf8
WORKDIR /app
ADD ./xxx-1.0.0.jar /app/lib/xxx-1.0.0.jar
ADD ./entrypoint.sh /app/bin/entrypoint.sh
ENTRYPOINT exec bash /app/bin/entrypoint.sh
RUN ln -s /app/logs /app/log && chown 1001.1001 -R /app
我们base镜像主要分三层添加了相关常用的工具如telnet netstat等以及安全修复
docker image inspect registry.xxx.com/base/oracle-jdk:8u201
分层信息:
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:174f5685490326fc0a1c0f5570b8663732189b327007e47ff13d2ca59673db02",
"sha256:2105884a1756425b2188084cc1738d6a6b13293773af56f6727af1be979858ed",
"sha256:c33745b350978d855171c996779055195b09e227844b7a03a1d795aee803dbb9"
]
}
镜像大小 622MB 不算太小,对比友商有些基础镜像竟然1-2G😂 简直有点离谱。
registry.xxx.com/base/oracle-jdk 8u201 1869b73a8999 2 years ago 622MB
从构建历史可以看出占用空间主要分为三层分别是
docker image history registry.xxx.com/base/oracle-jdk:8u201
IMAGE CREATED CREATED BY SIZE COMMENT
1869b73a8999 2 years ago /bin/sh -c set -ex; cd /tmp; curl -fsSLO… 302MB
2 years ago /bin/sh -c #(nop) ENV JAVA_VERSION=8 JAVA_U… 0B
2 years ago /bin/sh -c #(nop) ENV TZ=Asia/Shanghai LANG… 0B
2 years ago /bin/sh -c set -ex; yum update -y; yum i… 116MB
2 years ago /bin/sh -c #(nop) LABEL maintainer=… 0B
2 years ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
2 years ago /bin/sh -c #(nop) LABEL org.label-schema.sc… 0B
2 years ago /bin/sh -c #(nop) ADD file:b3ebbe8bd304723d4… 204MB
docker build -t registry.xxx.com/layer/build:old .
Sending build context to Docker daemon 67.45MB
Step 1/8 : FROM registry.xxx.com/base/oracle-jdk:8u201
---> 1869b73a8999
Step 2/8 : ENV TZ=Asia/Shanghai
---> Running in a67423d7246b
Removing intermediate container a67423d7246b
---> 9304102c278e
Step 3/8 : ENV LC_ALL en_US.utf8
---> Running in 05bf14c10efd
Removing intermediate container 05bf14c10efd
---> 63fb6c823893
Step 4/8 : WORKDIR /app
---> Running in 095764328d18
Removing intermediate container 095764328d18
---> c057ab4a7d18
Step 5/8 : ADD ./xxx-1.0.0.jar /app/lib/xxx-1.0.0.jar
---> 08928447fafa
Step 6/8 : ADD ./entrypoint.sh /app/bin/entrypoint.sh
---> 0902efa62157
Step 7/8 : ENTRYPOINT exec bash /app/bin/entrypoint.sh
---> Running in dc0eb814cfff
Removing intermediate container dc0eb814cfff
---> 20679a170252
Step 8/8 : RUN ln -s /app/logs /app/log && chown 1001.1001 -R /app
---> Running in 34159dca7464
Removing intermediate container 34159dca7464
---> b6043465c072
Successfully built b6043465c072
Successfully tagged registry.xxx.com/layer/build:old
构建后比我们预期的 jar包加基础镜像所占用的空间大😢 why?
#jar包大小
ls -lh xxx-1.0.0.jar
-rw------- 1 root root 65M Aug 16 10:33 xxx-1.0.0.jar
#镜像大小
docker image ls |grep registry.xxx.com/layer/build
registry.xxx.com/layer/build old b6043465c072 59 seconds ago 757MB
查看构建成品的镜像层发现竟然有两层一样大小,也会被推送了两遍。🥲离大谱。通过分析Dockerfile是因为
RUN ln -s /app/logs /app/log && chown 1001.1001 -R /app
导致的。为保证镜像安全,生产环境运行统一采用了1001账号运行。为了保证1001对workdir有绝对的读写权限。对目录递归授权导致文件属性发生改变。
在构建时认为文件发生变化产生一样大小的两层导致的。(这还是我自己当时优化引入的屎山😊)具体原因我也不太清楚docker构建时如何判定文件发生变化,我原本以为是基于MD5码。实际上在对文件属主属组改变时MD5码是不对变的。有知道的大神可以在评论区留言交流。
docker image history registry.xxx.com/layer/build:old
IMAGE CREATED CREATED BY SIZE COMMENT
b6043465c072 6 minutes ago /bin/sh -c ln -s /app/logs /app/log && chown… 67.4MB
20679a170252 6 minutes ago /bin/sh -c #(nop) ENTRYPOINT ["/bin/sh" "-c… 0B
0902efa62157 6 minutes ago /bin/sh -c #(nop) ADD file:9ac19caf793524884… 128B
08928447fafa 6 minutes ago /bin/sh -c #(nop) ADD file:c7624c195e7c047ee… 67.4MB
c057ab4a7d18 6 minutes ago /bin/sh -c #(nop) WORKDIR /app 0B
63fb6c823893 6 minutes ago /bin/sh -c #(nop) ENV LC_ALL=en_US.utf8 0B
9304102c278e 6 minutes ago /bin/sh -c #(nop) ENV TZ=Asia/Shanghai 0B
1869b73a8999 2 years ago /bin/sh -c set -ex; cd /tmp; curl -fsSLO… 302MB
2 years ago /bin/sh -c #(nop) ENV JAVA_VERSION=8 JAVA_U… 0B
2 years ago /bin/sh -c #(nop) ENV TZ=Asia/Shanghai LANG… 0B
2 years ago /bin/sh -c set -ex; yum update -y; yum i… 116MB
2 years ago /bin/sh -c #(nop) LABEL maintainer=currycan… 0B
2 years ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
2 years ago /bin/sh -c #(nop) LABEL org.label-schema.sc… 0B
2 years ago /bin/sh -c #(nop) ADD file:b3ebbe8bd304723d4… 204MB
#镜像也推送了两层一样大小
[root@localhost old-nochown]# docker push registry.xxx.com/layer/build:old
The push refers to repository [registry.xxx.com/layer/build]
bb35157fcb0b: Pushing [=============> ] 18.35MB/67.45MB
3c0641004863: Pushed
6ed0abaff975: Pushing [=====================> ] 28.38MB/67.45MB
aa0794231b2c: Pushed
c33745b35097: Layer already exists
2105884a1756: Layer already exists
174f56854903: Layer already exists
原因知道了。优化思路是先创建目录并递归授权。同时构建出来的jar的权限可以通过COPY 指定1001属主属组。
FROM registry.xxx.com/base/oracle-jdk:8u201
ENV TZ=Asia/Shanghai
ENV LC_ALL en_US.utf8
RUN mkdir -pv /app/{bin,lib,logs} && ln -s /app/logs /app/log && chown 1001:1001 -R /app
WORKDIR /app
COPY --chown=1001:1001 ./xxx-1.0.0.jar /app/lib/xxx-1.0.0.jar
COPY --chown=1001:1001 ./entrypoint.sh /app/bin/entrypoint.sh
ENTRYPOINT exec bash /app/bin/entrypoint.sh
优化后的镜像只有COPY这一层的jar包了。
docker image history registry.xxx.com/layer/build:old-mkdir
IMAGE CREATED CREATED BY SIZE COMMENT
b9fc3a8a97f2 29 hours ago /bin/sh -c #(nop) ENTRYPOINT ["/bin/sh" "-c… 0B
c823448f9582 29 hours ago /bin/sh -c #(nop) COPY --chown=1001:1001file… 128B
9a997f3962bf 29 hours ago /bin/sh -c #(nop) COPY --chown=1001:1001file… 67.4MB
f272c4e64f87 29 hours ago /bin/sh -c #(nop) WORKDIR /app 0B
1a526d3df7c2 29 hours ago /bin/sh -c mkdir -pv /app/{bin,lib,logs} && … 9B
63fb6c823893 30 hours ago /bin/sh -c #(nop) ENV LC_ALL=en_US.utf8 0B
9304102c278e 30 hours ago /bin/sh -c #(nop) ENV TZ=Asia/Shanghai 0B
1869b73a8999 2 years ago /bin/sh -c set -ex; cd /tmp; curl -fsSLO… 302MB
2 years ago /bin/sh -c #(nop) ENV JAVA_VERSION=8 JAVA_U… 0B
2 years ago /bin/sh -c #(nop) ENV TZ=Asia/Shanghai LANG… 0B
2 years ago /bin/sh -c set -ex; yum update -y; yum i… 116MB
2 years ago /bin/sh -c #(nop) LABEL maintainer=currycan… 0B
2 years ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B