分布式键值高可用存储集群架构搭建（ETCD)

   ETCD 是一个高可用的分布式键值数据库，可用于服务发现。ETCD 采用 raft 一致性算法，基于 Go 语言实现，随着CoreOS和Kubernetes等项目在开源社区日益火热，它们项目中都用到的etcd组件作为一个高可用强一致性的服务发现存储仓库，渐渐为开发人员所关注。在云计算时代，如何让服务快速透明地接入到计算集群中，如何让共享配置信息快速被集群中的所有机器发现，更为重要的是，如何构建这样一套高可用、安全、易于部署以及响应快速的服务集群，已经成为了迫切需要解决的问题。etcd为解决这类问题带来了福音，本文将从etcd的应用场景开始，深入解读etcd的实现方式，以供开发者们更为充分地享用etcd所带来的便利。

特点
简单：安装配置使用简单，提供 HTTP API
安全：支持 SSL 证书
可靠：采用 raft 算法，实现分布式系统数据的可用性和一致性
下载安装包：
[root@master ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz

复制安装包到各接点：
[root@master ~]# ansible k8s -m copy -a'src=/root/etcd-v3.3.2-linux-amd64.tar.gz dest=/root/'

部署环境三台机子：IP--主机名--集群结点名
192.168.27.211 client1 (etcd1)
192.168.27.212 client2 (etcd2)
192.168.27.213 client3 (etcd3)

1. 解压版本包
   tar -zxvf etcd-v3.3.2-linux-amd64.tar.gz 或 [root@master ~]# ansible k8s -m shell -a'tar -zxf etcd-v3.3.2-linux-amd64.tar.gz'
   
   
2. 将解压文件重命名并移到/etc/目录下
   mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2 或
   [root@master ~]# ansible k8s -m shell -a'mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2'
   
3. 将etcd和etcdctl加入系统变量PATH中(可以省略)
   [root@master ~]# ansible k8s -m shell -a'echo export PATH=/etc/etcd-v3.3.2:$PATH >> ~/.bashrc'
   [root@master ~]# ansible k8s -m shell -a'source ~/.bashrc '
   [root@master ~]# ansible k8s -m shell -a'source /root/.bashrc ' #靠普些
   
   
   4.进入/etc/etcd-v3.3.2，并创建配置文件etcd.conf，内容：
   
   
   name: etcd1
   data-dir: /etc/etcd-v3.3.2/data
   listen-client-urls: http://0.0.0.0:2379
   advertise-client-urls: http://192.168.27.211:2379
   listen-peer-urls: http://0.0.0.0:2380
   initial-advertise-peer-urls: http://192.168.27.211:2380
   initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
   initial-cluster-token: etcd-cluster-my
   initial-cluster-state: new
   5.设置etcd版本号，默认为2，我们安装的版本为3.3.2，所以此处设置为3
   
   在 文件~/.bashrc中加入一行改为3
   export ETCDCTL_API=3
   
   
   
4. 设置开机启动，新建文件/usr/lib/systemd/system/etcd.service，设置方便管理配置内容：
   [root@client1 etcd-v3.3.2]# cat /usr/lib/systemd/system/etcd.service
   [Unit]
   Description=Etcd Server
   After=network.target
   After=network-online.target
   Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/etc/etcd-v3.3.2
#User=etcd
ExecStart=/etc/etcd-v3.3.2/etcd --config-file /etc/etcd-v3.3.2/etcd.conf
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

5. 启动etcd，启动第一个member的时候，此member等待其它member初始化完毕，处于watting状态，此时紧接着启动第二个就行了
   systemctl daemon-reload
   systemctl enable etcd
   systemctl start etcd
   systemctl restart etcd

注意：192.168.27.212，192.168.27.213部署如同上面步骤，只需要将步骤4里面接点名字、ip改为自己ip即可。

name: etcd2
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.212:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.212:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new

name: etcd3
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.213:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.213:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new
6、启动成功后查看集群成员及状态:
[root@client3 etcd-v3.3.2]# etcdctl member list

etcdctl endpoint health --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379

通过非ETCD集群成员访问测试：
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
192.168.27.212:2379, 265ebeb3298fa51f, 3.3.2, 25 kB, false, 69, 71
192.168.27.213:2379, 510ff85ed64c7436, 3.3.2, 25 kB, true, 69, 71
[root@master etcd-v3.3.2]# etcdctl put name jerry --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl put topic "this is a etcd client test" --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
topic
this is a etcd client test
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379
topic
this is a etcd client test

Etcd使用：
读写键值：
[root@client1 etcd-v3.3.2]# etcdctl put key1 value1
OK
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1

前缀读（将所有前缀是key的键值对都读出来）

键值删除：
[root@client1 etcd-v3.3.2]# etcdctl del key
0
[root@client1 etcd-v3.3.2]# etcdctl get key
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1
[root@client1 etcd-v3.3.2]# etcdctl del key1
1
[root@client1 etcd-v3.3.2]# etcdctl get key1

观察已无KEY1键及值的存在。

按前缀全部删除

观察已无键值及数据

重新创建三个键值：

其它ETCD集群成员上访问键值测试：

监听某个键，每次键值有变化
[root@client1 etcd-v3.3.2]# etcdctl watch key --prefix
PUT
key1
value
PUT
key2
jerry

设置过期时间TTL


[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287411 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287411
Error: etcdserver: requested lease not found
[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287414 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287414
OK
[root@client2 etcd-v3.3.2]# etcdctl lease keep-alive 251f6cf538287414
lease 251f6cf538287414 expired or revoked.
[root@client2 etcd-v3.3.2]# etcdctl get key5
[root@client2 etcd-v3.3.2]# etcdctl get key5