docker swarm 介绍
为什么不建议在生产环境中使用docker-Compose
- 多机器如何管理?
- 如果跨机器做scale横向扩展?
- 容器失败退出时如何新建容器确保服务正常运行?
- 如何确保零宕机时间?
- 如何管理密码,Key等敏感数据?
- 其它
容器编排 swarm
Swarm的基本架构
docker swarm vs kubernetes
k8s在容器编排领域处于绝对领先的地位
2021年redhat调查https://www.redhat.com/en/resources/kubernetes-adoption-security-market-trends-2021-overview
为什么还要学些了解docker swarm呢?
Swarm 单节点快速上手
初始化
docker info
这个命令可以查看我们的docker engine有没有激活swarm模式, 默认是没有的,我们会看到
Swarm: inactive
激活swarm,有两个方法:
- 初始化一个swarm集群,自己成为manager
- 加入一个已经存在的swarm集群
PS C:\Users\Peng Xiao\code-demo> docker swarm init Swarm initialized: current node (vjtstrkxntsacyjtvl18hcbe4) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-33ci17l1n34fh6v4r1qq8qmocjo347saeuer2xrxflrn25jgjx-7vphgu8a0gsa4anof6ffrgwqb 192.168.65.3:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. PS C:\Users\Peng Xiao\code-demo> docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION vjtstrkxntsacyjtvl18hcbe4 * docker-desktop Ready Active Leader 20.10.7 PS C:\Users\Peng Xiao\code-demo>
docker swarm init 背后发生了什么
主要是PKI和安全相关的自动化
- 创建swarm集群的根证书
- manager节点的证书
- 其它节点加入集群需要的tokens
创建Raft数据库用于存储证书,配置,密码等数据
RAFT相关资料
- http://thesecretlivesofdata.com/raft/
- https://raft.github.io/
- https://docs.docker.com/engine/swarm/raft/
看动画学会 Raft 算法
https://mp.weixin.qq.com/s/p8qBcIhM04REuQ-uG4gnbw
Swarm 三节点集群搭建
创建3节点swarm cluster的方法
- https://labs.play-with-docker.com/ play with docker 网站, 优点是快速方便,缺点是环境不持久,4个小时后环境会被重置
- 在本地通过虚拟化软件搭建Linux虚拟机,优点是稳定,方便,缺点是占用系统资源,需要电脑内存最好8G及其以上
- 在云上使用云主机, 亚马逊,Google,微软Azure,阿里云,腾讯云等,缺点是需要消耗金钱(但是有些云服务,有免费试用)
多节点的环境涉及到机器之间的通信需求,所以防火墙和网络安全策略组是大家一定要考虑的问题,特别是在云上使用云主机的情况,下面这些端口记得打开 防火墙
以及 设置安全策略组
- TCP port
2376
- TCP port
2377
- TCP and UDP port
7946
- UDP port
4789
为了简化,以上所有端口都允许节点之间自由访问就行。
Warning
请大家注意,请大家使用自己熟悉的方式去创建这样的三节点集群,如果熟悉vagrant和virtualbox,那可以使用我们课程的里方法,如果不熟悉想学习,请参考B站和Youtube视频,我们在课程里不会去讲解 什么是vagrant/virtualbox以及怎么去使用它。
Vagrant + Virtualbox
下载安装 VirtualBox
https://www.virtualbox.org/
下载安装 Vagarnt
https://www.vagrantup.com/
Vagrant入门系列视频
- Youtube https://www.youtube.com/playlist?list=PLfQqWeOCIH4B6YAEXMr6cx4AfnKNBLbZO
- B站 https://space.bilibili.com/364122352/channel/detail?cid=174004
本节Vagrant搭建的文件下载
- :download:`Centos 版 vagrant相关文件 <vagrant-setup.zip>`
- :download:`Ubuntu 版 vagrant相关文件 <vagrant-setup-ubuntu.zip>`
Vagrant的基本操作请参考我们的上面的B站或者Youtube视频
虚拟机的启动:vagrant up
虚拟机的停止:vagrant halt
虚拟机的删除:vagrant destroy
Swarm 的 overlay 网络详解
对于理解swarm的网络来讲,个人认为最重要的两个点:
- 第一是外部如何访问部署运行在swarm集群内的服务,可以称之为
入方向
流量,在swarm里我们通过ingress
来解决 - 第二是部署在swarm集群里的服务,如何对外进行访问,这部分又分为两块:
- 第一,
东西向流量
,也就是不同swarm节点上的容器之间如何通信,swarm通过overlay
网络来解决; - 第二,
南北向流量
,也就是swarm集群里的容器如何对外访问,比如互联网,这个是Linux bridge + iptables NAT
来解决的
- 第一,
创建 overlay 网络
vagrant@swarm-manager:~$ docker network create -d overlay mynet
这个网络会同步到所有的swarm节点上
创建服务
创建一个服务连接到这个 overlay网络, name 是 test , replicas 是 2
vagrant@swarm-manager:~$ docker service create --network mynet --name test --replicas 2 busybox ping 8.8.8.8 vagrant@swarm-manager:~$ docker service ps test ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS yf5uqm1kzx6d test.1 busybox:latest swarm-worker1 Running Running 18 seconds ago 3tmp4cdqfs8a test.2 busybox:latest swarm-worker2 Running Running 18 seconds ago
可以看到这两个容器分别被创建在worker1和worker2两个节点上
网络查看
到worker1和worker2上分别查看容器的网络连接情况
vagrant@swarm-worker1:~$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES cac4be28ced7 busybox:latest "ping 8.8.8.8" 2 days ago Up 2 days test.1.yf5uqm1kzx6dbt7n26e4akhsu vagrant@swarm-worker1:~$ docker container exec -it cac sh / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 24: eth0@if25: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue link/ether 02:42:0a:00:01:08 brd ff:ff:ff:ff:ff:ff inet 10.0.1.8/24 brd 10.0.1.255 scope global eth0 valid_lft forever preferred_lft forever 26: eth1@if27: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 02:42:ac:12:00:03 brd ff:ff:ff:ff:ff:ff inet 172.18.0.3/16 brd 172.18.255.255 scope global eth1 valid_lft forever preferred_lft forever / #
这个容器有两个接口 eth0和eth1, 其中eth0是连到了mynet这个网络,eth1是连到docker_gwbridge这个网络
vagrant@swarm-worker1:~$ docker network ls NETWORK ID NAME DRIVER SCOPE a631a4e0b63c bridge bridge local 56945463a582 docker_gwbridge bridge local 9bdfcae84f94 host host local 14fy2l7a4mci ingress overlay swarm lpirdge00y3j mynet overlay swarm c1837f1284f8 none null local
在这个容器里是可以直接ping通worker2上容器的IP 10.0.1.9的
Swarm 的 ingress网络
docker swarm的ingress网络又叫 Ingress Routing Mesh
主要是为了实现把service的服务端口对外发布出去,让其能够被外部网络访问到。
ingress routing mesh是docker swarm网络里最复杂的一部分内容,包括多方面的内容:
- iptables的 Destination NAT流量转发
- Linux bridge, network namespace
- 使用IPVS技术做负载均衡
- 包括容器间的通信(overlay)和入方向流量的端口转发
service创建
创建一个service,指定网络是overlay的mynet, 通过-p把端口映射出来
我们使用的镜像 containous/whoami
是一个简单的web服务,能返回服务器的hostname,和基本的网络信息,比如IP地址
vagrant@swarm-manager:~$ docker service create --name web --network mynet -p 8080:80 --replicas 2 containous/whoami a9cn3p0ovg5jcz30rzz89lyfz overall progress: 2 out of 2 tasks 1/2: running [==================================================>] 2/2: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS a9cn3p0ovg5j web replicated 2/2 containous/whoami:latest *:8080->80/tcp vagrant@swarm-manager:~$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS udlzvsraha1x web.1 containous/whoami:latest swarm-worker1 Running Running 16 seconds ago mms2c65e5ygt web.2 containous/whoami:latest swarm-manager Running Running 16 seconds ago vagrant@swarm-manager:~$
service的访问
8080这个端口到底映射到哪里了?尝试三个swarm节点的IP加端口8080
可以看到三个节点IP都可以访问,并且回应的容器是不同的(hostname),也就是有负载均衡的效果
vagrant@swarm-manager:~$ curl 192.168.200.10:8080 Hostname: fdf7c1354507 IP: 127.0.0.1 IP: 10.0.0.7 IP: 172.18.0.3 IP: 10.0.1.14 RemoteAddr: 10.0.0.2:36828 GET / HTTP/1.1 Host: 192.168.200.10:8080 User-Agent: curl/7.68.0 Accept: */* vagrant@swarm-manager:~$ curl 192.168.200.11:8080 Hostname: fdf7c1354507 IP: 127.0.0.1 IP: 10.0.0.7 IP: 172.18.0.3 IP: 10.0.1.14 RemoteAddr: 10.0.0.3:54212 GET / HTTP/1.1 Host: 192.168.200.11:8080 User-Agent: curl/7.68.0 Accept: */* vagrant@swarm-manager:~$ curl 192.168.200.12:8080 Hostname: c83ee052787a IP: 127.0.0.1 IP: 10.0.0.6 IP: 172.18.0.3 IP: 10.0.1.13 RemoteAddr: 10.0.0.4:49820 GET / HTTP/1.1 Host: 192.168.200.12:8080 User-Agent: curl/7.68.0 Accept: */*
ingress 数据包的走向
以manager节点为例,数据到底是如何达到service的container的
vagrant@swarm-manager:~$ sudo iptables -nvL -t nat Chain PREROUTING (policy ACCEPT 388 packets, 35780 bytes) pkts bytes target prot opt in out source destination 296 17960 DOCKER-INGRESS all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL 21365 1282K DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT 388 packets, 35780 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 340 packets, 20930 bytes) pkts bytes target prot opt in out source destination 8 590 DOCKER-INGRESS all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL 1 60 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT 340 packets, 20930 bytes) pkts bytes target prot opt in out source destination 2 120 MASQUERADE all -- * docker_gwbridge 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match src-type LOCAL 3 252 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0 0 0 MASQUERADE all -- * !docker_gwbridge 172.18.0.0/16 0.0.0.0/0 Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0 0 0 RETURN all -- docker_gwbridge * 0.0.0.0/0 0.0.0.0/0 Chain DOCKER-INGRESS (2 references) pkts bytes target prot opt in out source destination 2 120 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.18.0.2:8080 302 18430 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
通过iptables,可以看到一条DNAT的规则,所有访问本地8080端口的流量都被转发到 172.18.0.2:8080
那这个172.18.0.2 是什么?
首先 172.18.0.0/16 这个网段是 docker_gwbridge
的,所以这个地址肯定是连在了 docker_gwbridge
上。
docker network inspect docker_gwbridge
可以看到这个网络连接了一个叫 ingress-sbox
的容器。它的地址就是 172.18.0.2/16
这个 ingress-sbox
其实并不是一个容器,而是一个网络的命名空间 network namespace, 我们可以通过下面的方式进入到这个命名空间
vagrant@swarm-manager:~$ docker run -it --rm -v /var/run/docker/netns:/netns --privileged=true nicolaka/netshoot nsenter --net=/netns/ingress_sbox sh ~ # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 8: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:00:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.0.0.2/24 brd 10.0.0.255 scope global eth0 valid_lft forever preferred_lft forever inet 10.0.0.5/32 scope global eth0 valid_lft forever preferred_lft forever 10: eth1@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 1 inet 172.18.0.2/16 brd 172.18.255.255 scope global eth1 valid_lft forever preferred_lft forever
通过查看地址,发现这个命名空间连接了两个网络,一个eth1是连接了 docker_gwbridge
,另外一个eth0连接了 ingress
这个网络。
~ # ip route default via 172.18.0.1 dev eth1 10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.2 172.18.0.0/16 dev eth1 proto kernel scope link src 172.18.0.2 ~ # iptables -nvL -t mangle Chain PREROUTING (policy ACCEPT 22 packets, 2084 bytes) pkts bytes target prot opt in out source destination 12 806 MARK tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 MARK set 0x100 Chain INPUT (policy ACCEPT 14 packets, 1038 bytes) pkts bytes target prot opt in out source destination 0 0 MARK all -- * * 0.0.0.0/0 10.0.0.5 MARK set 0x100 Chain FORWARD (policy ACCEPT 8 packets, 1046 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 14 packets, 940 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 22 packets, 1986 bytes) pkts bytes target prot opt in out source destination ~ # ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 256 rr -> 10.0.0.6:0 Masq 1 0 0 -> 10.0.0.7:0 Masq 1 0 0 ~ #
通过ipvs做了负载均衡
关于这里的负载均衡
- 这是一个stateless load balancing
- 这是三层的负载均衡,不是四层的 LB is at OSI Layer 3 (TCP), not Layer 4 (DNS)
- 以上两个限制可以通过Nginx或者HAProxy LB proxy解决 (https://docs.docker.com/engine/swarm/ingress/)
内部负载均衡和 VIP
创建一个mynet的overlay网络,创建一个service
vagrant@swarm-manager:~$ docker network ls NETWORK ID NAME DRIVER SCOPE afc8f54c1d07 bridge bridge local 128fd1cb0fae docker_gwbridge bridge local 0ea68b0d28b9 host host local 14fy2l7a4mci ingress overlay swarm lpirdge00y3j mynet overlay swarm a8edf1804fb6 none null local vagrant@swarm-manager:~$ docker service create --name web --network mynet --replicas 2 containous/whoami jozc1x1c1zpyjl9b5j5abzm0g overall progress: 2 out of 2 tasks 1/2: running [==================================================>] 2/2: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS jozc1x1c1zpy web replicated 2/2 containous/whoami:latest vagrant@swarm-manager:~$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS pwi87g86kbxd web.1 containous/whoami:latest swarm-worker1 Running Running 47 seconds ago xbri2akxy2e8 web.2 containous/whoami:latest swarm-worker2 Running Running 44 seconds ago vagrant@swarm-manager:~$
创建一个client
vagrant@swarm-manager:~$ docker service create --name client --network mynet xiaopeng163/net-box:latest ping 8.8.8.8 skbcdfvgidwafbm4nciq82env overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS skbcdfvgidwa client replicated 1/1 xiaopeng163/net-box:latest jozc1x1c1zpy web replicated 2/2 containous/whoami:latest vagrant@swarm-manager:~$ docker service ps client ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS sg9b3dqrgru4 client.1 xiaopeng163/net-box:latest swarm-manager Running Running 28 seconds ago vagrant@swarm-manager:~$
尝试进入client这个容器,去ping web这个service name, 获取到的IP 10.0.1.30,称之为VIP(虚拟IP)
vagrant@swarm-manager:~$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 36dce35d56e8 xiaopeng163/net-box:latest "ping 8.8.8.8" 19 minutes ago Up 19 minutes client.1.sg9b3dqrgru4f14k2tpxzg2ei vagrant@swarm-manager:~$ docker container exec -it 36dc sh /omd # curl web Hostname: 6039865a1e5d IP: 127.0.0.1 IP: 10.0.1.32 IP: 172.18.0.3 RemoteAddr: 10.0.1.37:40972 GET / HTTP/1.1 Host: web User-Agent: curl/7.69.1 Accept: */* /omd # curl web Hostname: c3b3e99b9bb1 IP: 127.0.0.1 IP: 10.0.1.31 IP: 172.18.0.3 RemoteAddr: 10.0.1.37:40974 GET / HTTP/1.1 Host: web User-Agent: curl/7.69.1 Accept: */* /omd # curl web Hostname: 6039865a1e5d IP: 127.0.0.1 IP: 10.0.1.32 IP: 172.18.0.3 RemoteAddr: 10.0.1.37:40976 GET / HTTP/1.1 Host: web User-Agent: curl/7.69.1 Accept: */* /omd # /omd # ping web -c 2 PING web (10.0.1.30): 56 data bytes 64 bytes from 10.0.1.30: seq=0 ttl=64 time=0.044 ms 64 bytes from 10.0.1.30: seq=1 ttl=64 time=0.071 ms --- web ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.044/0.057/0.071 ms /omd #
这个虚拟IP在一个特殊的网络命令空间里,这个空间连接在我们的mynet这个overlay的网络上
通过 docker network inspect mynet 可以看到这个命名空间,叫lb-mynet
"Containers": { "36dce35d56e87d43d08c5b9a94678fe789659cb3b1a5c9ddccd7de4b26e8d588": { "Name": "client.1.sg9b3dqrgru4f14k2tpxzg2ei", "EndpointID": "e8972d0091afaaa091886799aca164b742ca93408377d9ee599bdf91188416c1", "MacAddress": "02:42:0a:00:01:24", "IPv4Address": "10.0.1.36/24", "IPv6Address": "" }, "lb-mynet": { "Name": "mynet-endpoint", "EndpointID": "e299d083b25a1942f6e0f7989436c3c3e8d79c7395a80dd50b7709825022bfac", "MacAddress": "02:42:0a:00:01:25", "IPv4Address": "10.0.1.37/24", "IPv6Address": "" }
通过下面的命令,找到这个命名空间的名字
vagrant@swarm-manager:~$ sudo ls /var/run/docker/netns/ 1-14fy2l7a4m 1-lpirdge00y dfb766d83076 ingress_sbox lb_lpirdge00 vagrant@swarm-manager:~$
名字叫 lb_lpirdge00
通过nsenter进入到这个命名空间的sh里, 可以看到刚才的VIP地址10.0.1.30
vagrant@swarm-manager:~$ sudo nsenter --net=/var/run/docker/netns/lb_lpirdge00 sh # # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 50: eth0@if51: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:00:01:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.0.1.37/24 brd 10.0.1.255 scope global eth0 valid_lft forever preferred_lft forever inet 10.0.1.30/32 scope global eth0 valid_lft forever preferred_lft forever inet 10.0.1.35/32 scope global eth0 valid_lft forever preferred_lft forever #
和ingress网络一样,可以查看iptables,ipvs的负载均衡, 基本就可以理解负载均衡是怎么一回事了。 Mark=0x106, 也就是262(十进制),会轮询把请求发给10.0.1.31 和 10.0.1.32
# iptables -nvL -t mangle Chain PREROUTING (policy ACCEPT 128 packets, 11198 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 92 packets, 6743 bytes) pkts bytes target prot opt in out source destination 72 4995 MARK all -- * * 0.0.0.0/0 10.0.1.30 MARK set 0x106 0 0 MARK all -- * * 0.0.0.0/0 10.0.1.35 MARK set 0x107 Chain FORWARD (policy ACCEPT 36 packets, 4455 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 101 packets, 7535 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 128 packets, 11198 bytes) pkts bytes target prot opt in out source destination # ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 262 rr -> 10.0.1.31:0 Masq 1 0 0 -> 10.0.1.32:0 Masq 1 0 0 FWM 263 rr -> 10.0.1.36:0 Masq 1 0 0 #
这个流量会走我们的mynet这个overlay网络。
部署多 service 应用
如何像docker-compose一样部署多服务应用,这一节我们先看手动的方式
本节课所用的源码文件 https://github.com/xiaopeng163/flask-redis
创建一个mynet的overlay网络
vagrant@swarm-manager:~$ docker network ls NETWORK ID NAME DRIVER SCOPE afc8f54c1d07 bridge bridge local 128fd1cb0fae docker_gwbridge bridge local 0ea68b0d28b9 host host local 14fy2l7a4mci ingress overlay swarm lpirdge00y3j mynet overlay swarm a8edf1804fb6 none null local vagrant@swarm-manager:~$
创建一个redis的service
vagrant@swarm-manager:~$ docker service create --network mynet --name redis redis:latest redis-server --requirepass ABC123 qh3nfeth3wc7uoz9ozvzta5ea overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker servce ls docker: 'servce' is not a docker command. See 'docker --help' vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS qh3nfeth3wc7 redis replicated 1/1 redis:latest vagrant@swarm-manager:~$ docker service ps redis ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 111cpkjn4a0k redis.1 redis:latest swarm-worker2 Running Running 19 seconds ago vagrant@swarm-manager:~$
创建一个flask的service
vagrant@swarm-manager:~$ docker service create --network mynet --name flask --env REDIS_HOST=redis --env REDIS_PASS=ABC123 -p 8080 :5000 xiaopeng163/flask-redis:latest y7garhvlxah592j5lmqv8a3xj overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS y7garhvlxah5 flask replicated 1/1 xiaopeng163/flask-redis:latest *:8080->5000/tcp qh3nfeth3wc7 redis replicated 1/1 redis:latest vagrant@swarm-manager:~$ docker service ps flask ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS quptcq7vb48w flask.1 xiaopeng163/flask-redis:latest swarm-worker1 Running Running 15 seconds ago vagrant@swarm-manager:~$ curl 127.0.0.1:8080 Hello Container World! I have been seen 1 times and my hostname is d4de54036614. vagrant@swarm-manager:~$ curl 127.0.0.1:8080 Hello Container World! I have been seen 2 times and my hostname is d4de54036614. vagrant@swarm-manager:~$ curl 127.0.0.1:8080 Hello Container World! I have been seen 3 times and my hostname is d4de54036614. vagrant@swarm-manager:~$ curl 127.0.0.1:8080 Hello Container World! I have been seen 4 times and my hostname is d4de54036614. vagrant@swarm-manager:~$
swarm stack 部署多 service 应用
先在swarm manager节点上安装一下 docker-compose
vagrant@swarm-manager:~$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose vagrant@swarm-manager:~$ sudo chmod +x /usr/local/bin/docker-compose
clone我们的代码仓库
vagrant@swarm-manager:~$ git clone https://github.com/xiaopeng163/flask-redis Cloning into 'flask-redis'... remote: Enumerating objects: 22, done. remote: Counting objects: 100% (22/22), done. remote: Compressing objects: 100% (19/19), done. remote: Total 22 (delta 9), reused 7 (delta 2), pack-reused 0 Unpacking objects: 100% (22/22), 8.60 KiB | 1.07 MiB/s, done. vagrant@swarm-manager:~$ cd flask-redis vagrant@swarm-manager:~/flask-redis$ ls Dockerfile LICENSE README.md app.py docker-compose.yml vagrant@swarm-manager:~/flask-redis$
环境清理
vagrant@swarm-manager:~/flask-redis$ docker system prune -a -f
镜像构建和提交, 如果你想做这一步,可以把docker-compose.yml里的 xiaopeng163/flask-redis
改成你的dockerhub id
vagrant@swarm-manager:~/flask-redis$ docker-compose build vagrant@swarm-manager:~/flask-redis$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE xiaopeng163/flask-redis latest 5efb4fcbcfc3 6 seconds ago 126MB python 3.9.5-slim c71955050276 3 weeks ago 115MB
提交镜像到dockerhub
vagrant@swarm-manager:~/flask-redis$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: xiaopeng163 Password: WARNING! Your password will be stored unencrypted in /home/vagrant/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded vagrant@swarm-manager:~/flask-redis$ docker-compose push WARNING: The REDIS_PASSWORD variable is not set. Defaulting to a blank string. Pushing flask (xiaopeng163/flask-redis:latest)... The push refers to repository [docker.io/xiaopeng163/flask-redis] f447d33c161b: Pushed f7395da2fd9c: Pushed 5b156295b5a3: Layer already exists 115e0863702d: Layer already exists e10857b94a57: Layer already exists 8d418cbfaf25: Layer already exists 764055ebc9a7: Layer already exists latest: digest: sha256:c909100fda2f4160b593b4e0fb692b89046cebb909ae90546627deca9827b676 size: 1788 vagrant@swarm-manager:~/flask-redis$
通过stack启动服务
vagrant@swarm-manager:~/flask-redis$ env REDIS_PASSWORD=ABC123 docker stack deploy --compose-file docker-compose.yml flask-demo Ignoring unsupported options: build Creating network flask-demo_default Creating service flask-demo_flask Creating service flask-demo_redis-server vagrant@swarm-manager:~/flask-redis$ vagrant@swarm-manager:~/flask-redis$ docker stack ls NAME SERVICES ORCHESTRATOR flask-demo 2 Swarm vagrant@swarm-manager:~/flask-redis$ docker stack ps flask-demo ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS lzm6i9inoa8e flask-demo_flask.1 xiaopeng163/flask-redis:latest swarm-manager Running Running 23 seconds ago ejojb0o5lbu0 flask-demo_redis-server.1 redis:latest swarm-worker2 Running Running 21 seconds ago vagrant@swarm-manager:~/flask-redis$ docker stack services flask-demo ID NAME MODE REPLICAS IMAGE PORTS mpx75z1rrlwn flask-demo_flask replicated 1/1 xiaopeng163/flask-redis:latest *:8080->5000/tcp z85n16zsldr1 flask-demo_redis-server replicated 1/1 redis:latest vagrant@swarm-manager:~/flask-redis$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS mpx75z1rrlwn flask-demo_flask replicated 1/1 xiaopeng163/flask-redis:latest *:8080->5000/tcp z85n16zsldr1 flask-demo_redis-server replicated 1/1 redis:latest vagrant@swarm-manager:~/flask-redis$ curl 127.0.0.1:8080 Hello Container World! I have been seen 1 times and my hostname is 21d63a8bfb57. vagrant@swarm-manager:~/flask-redis$ curl 127.0.0.1:8080 Hello Container World! I have been seen 2 times and my hostname is 21d63a8bfb57. vagrant@swarm-manager:~/flask-redis$ curl 127.0.0.1:8080 Hello Container World! I have been seen 3 times and my hostname is 21d63a8bfb57. vagrant@swarm-manager:~/flask-redis$
在 swarm 中使用 secret
文档 https://docs.docker.com/engine/swarm/secrets/
创建secret
有两种方式
从标准的收入读取
vagrant@swarm-manager:~$ echo abc123 | docker secret create mysql_pass - 4nkx3vpdd41tbvl9qs24j7m6w vagrant@swarm-manager:~$ docker secret ls ID NAME DRIVER CREATED UPDATED 4nkx3vpdd41tbvl9qs24j7m6w mysql_pass 8 seconds ago 8 seconds ago vagrant@swarm-manager:~$ docker secret inspect mysql_pass [ { "ID": "4nkx3vpdd41tbvl9qs24j7m6w", "Version": { "Index": 4562 }, "CreatedAt": "2021-07-25T22:36:51.544523646Z", "UpdatedAt": "2021-07-25T22:36:51.544523646Z", "Spec": { "Name": "mysql_pass", "Labels": {} } } ] vagrant@swarm-manager:~$ docker secret rm mysql_pass mysql_pass vagrant@swarm-manager:~$
从文件读取
vagrant@swarm-manager:~$ ls mysql_pass.txt vagrant@swarm-manager:~$ more mysql_pass.txt abc123 vagrant@swarm-manager:~$ docker secret create mysql_pass mysql_pass.txt elsodoordd7zzpgsdlwgynq3f vagrant@swarm-manager:~$ docker secret inspect mysql_pass [ { "ID": "elsodoordd7zzpgsdlwgynq3f", "Version": { "Index": 4564 }, "CreatedAt": "2021-07-25T22:38:14.143954043Z", "UpdatedAt": "2021-07-25T22:38:14.143954043Z", "Spec": { "Name": "mysql_pass", "Labels": {} } } ] vagrant@swarm-manager:~$
secret 的使用
参考 https://hub.docker.com/_/mysql
vagrant@swarm-manager:~$ docker service create --name mysql-demo --secret mysql_pass --env MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_pass mysql:5.7 wb4z2ximgqaefephu9f4109c7 overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged vagrant@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS wb4z2ximgqae mysql-demo replicated 1/1 mysql:5.7 vagrant@swarm-manager:~$ docker service ps mysql-demo ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 909429p4uovy mysql-demo.1 mysql:5.7 swarm-worker2 Running Running 32 seconds ago vagrant@swarm-manager:~$
swarm 使用 local volume
本节源码,两个文件
docker-compose.yml
version: "3.8" services: db: image: mysql:5.7 environment: - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_pass secrets: - mysql_pass volumes: - data:/var/lib/mysql volumes: data: secrets: mysql_pass: file: mysql_pass.txt
mysql_pass.txt
vagrant@swarm-manager:~$ more mysql_pass.txt abc123 vagrant@swarm-manager:~$
swarm 部署投票 app
源码 https://github.com/dockersamples/example-voting-app
- 无标签