您好,登录后才能下订单哦!
时下大数据时代,海量数据与吞吐量的数据库应用对单机的性能造成了较大的压力,将会发生CPU耗尽,存储压力大,可用资源耗尽等问题。
便出现了新的技术,分片技术。它是MongoDB用来将大型集合分割到不同服务器上所采用的的方法,它几乎是能够自动完成所有事情,只要告诉MongoDB要分配的数据,它就能够自动维护数据到不同服务器之间均衡存储。
Config server: 存储集群所有节点,分片数据路由信息,默认需要配置3个config server节点;
Mongos: 提供对外应用的访问,所有操作均通过mongos执行,一般有多个mongos节点,数据迁移和数据自动平衡;
Mongod: 存储应用数据记录,一般有多个Mongod节点,达到数据的分片目的。
1. 三台服务器配置如下:
IP地址 | 路由服务器 | 配置服务器 | shard1 | shard2 | shard3 |
---|---|---|---|---|---|
192.168.96.10 | 20000 | 21000 | 27001 | 27002 | 27003 |
192.168.96.11 | 20000 | 21000 | 27001 | 27002 | 27003 |
192.168.96.12 | 20000 | 21000 | 27001 | 27002 | 27003 |
2.操作系统:CentOS 7
3.关闭防火墙和Selinux功能
4.软件包:mongodb-linux-x86_64-rhel70-3.6.6.tgz 密码:4h77
tar -zxvf mongodb-linux-x86_64-rhel70-3.6.6.tgz -C /usr/local
cd /usr/local
mv mongodb-linux-x86_64-rhel70-3.6.6 mongodb
mkdir -p /data/mongodb/conf
mkdir -p /data/mongodb/mongos/log
mkdir -p /data/mongodb/config/data
mkdir -p /data/mongodb/config/log
#循环创建分片服务器目录
for i in 1 2 3
do
mkdir -p /data/mongodb/shard$i/data
mkdir -p /data/mongodb/shard$i/log
done
useradd -M -s /sbin/nologin mongo
chown -R mongo.mongo /usr/local/mongodb
chown -R mongo.mongo /data/mongodb
echo "PATH=/usr/local/mongodb/bin:$PATH" >> /etc/profile
source /etc/profile
vim ~/.bash_profile
ulimit -n 25000 //可以打开的最大文件数量
ulimit -u 25000 //用户最大可用的进程数
sysctl -w vm.zone_reclaim_mode=0 //当内存不足时,从其他节点分配内存
source ~/.bash_profile
vim /etc/rc.d/rc.local
......
//对大内存页面优化
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
chmod +x /etc/rc.d/rc.local
特别提醒:请检查三台服务器配置是否都已经配置并启动好
vim /data/mongodb/conf/config.conf
#配置文件内容
pidfilepath = /data/mongodb/config/log/config-srv.pid
dbpath = /data/mongodb/config/data
logpath = /data/mongodb/config/log/config-srv.log
logappend = true
bind_ip = 0.0.0.0
port = 21000
fork = true
#declare this is a config db of a cluster;
configsvr = true
#复制集名称
replSet=configs
#设置最大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-config.service
[Unit]
Description=mongodb-config
After=network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/config.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/config.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-config
systemctl start mongo-config
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 21000
//配置
> config={"_id":"configs",members:[{"_id":0,"host":"192.168.96.10:21000"},{"_id":1,"host":"192.168.96.11:21000"},{"_id":2,"host":"192.168.96.12:21000"}]}
//初始化
> rs.initiate(config)
//查看状态
> rs.status()
vim /data/mongodb/conf/shard1.conf
pidfilepath = /data/mongodb/shard1/log/shard1.pid
dbpath = /data/mongodb/shard1/data
logpath = /data/mongodb/shard1/log/shard1.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27001
fork = true
#复制集名称
replSet=shard1
#declare this is a shard db of a cluster;
shardsvr = true
#设置最大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard1.service
[Unit]
Description=mongodb-shard1
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard1.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard1.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-shard1
systemctl start mongo-shard1
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 27001
> use admin
> config={
"_id":"shard1",members:[{"_id":0,"host":"192.168.96.10:27001",arbiterOnly:true},{"_id":1,"host":"192.168.96.11:27001"},{"_id":2,"host":"192.168.96.12:27001"}]}
> rs.initiate(config)
> rs.status()
vim /data/mongodb/conf/shard2.conf
pidfilepath = /data/mongodb/shard2/log/shard2.pid
dbpath = /data/mongodb/shard2/data
logpath = /data/mongodb/shard2/log/shard2.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27002
fork = true
#复制集名称
replSet=shard2
#declare this is a shard db of a cluster;
shardsvr = true
#设置最大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard2.service
[Unit]
Description=mongodb-shard2
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard2.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard2.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-shard2
systemctl start mongo-shard2
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 27002
> use admin
> config={"_id":"shard2",members:[{ "_id":0,"host":"192.168.96.10:27002"}, {"_id":1,"host":"192.168.96.11:27002",arbiterOnly:true},{"_id":2,"host":"192.168.96.12:27002"}]}
> rs.initiate(config);
> rs.status()
vim /data/mongodb/conf/shard3.conf
pidfilepath = /data/mongodb/shard3/log/shard3.pid
dbpath = /data/mongodb/shard3/data
logpath = /data/mongodb/shard3/log/shard3.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27003
fork = true
#复制集名称
replSet=shard3
#declare this is a shard db of a cluster;
shardsvr = true
#设置最大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard3.service
```[Unit]
Description=mongodb-shard3
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard3.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard3.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
#### 3.创建启动脚本
systemctl daemon-reload
systemctl enable mongo-shard3
systemctl start mongo-shard3
**特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤**
#### 4.初始化复制集(登录任意一台服务器进行初始化复制集)
> mongo --port 27003
use admin
config={"_id":"shard3",members:[{ "_id":0,"host":"192.168.96.10:27003"}, {"_id":1,"host":"192.168.96.11:27003"},{"_id":2,"host":"192.168.96.12:27003",arbiterOnly:true}]}
rs.initiate(config)
rs.status()
vim /data/mongodb/conf/mongos.conf
pidfilepath = /data/mongodb/mongos/log/mongos.pid
logpath = /data/mongodb/mongos/log/mongos.log
logappend = true
bind_ip = 0.0.0.0
port = 20000
fork = true
#监听的配置服务器的地址:端口,(重要、重要、重要)
configdb = configs/192.168.96.10:21000,192.168.96.11:21000,192.168.96.12:21000
#设置最大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongos.service
[Unit]
Description=Mongo Router Service
After=mongo-config.service
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongos --config /data/mongodb/conf/mongos.conf
Restart=on-failure
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongos
systemctl start mongos
特别提醒:三台服务器都配置完毕后,再进行以下步骤
mongo --port 20000
> use admin
> sh.addShard("shard1/192.168.96.10:27001,192.168.96.11:27001,192.168.96.12:27001")
sh.addShard("shard2/192.168.96.10:27002,192.168.96.11:27002,192.168.96.12:27002")
sh.addShard("shard3/192.168.96.10:27003,192.168.96.11:27003,192.168.96.12:27003")
> sh.status()
> use school
> for(i=1;i<=1000;i++){db.user.insert({"id":i,"name":"jack"+i})}
//开启school数据库分片功能
> db.runCommand({enablesharding:"school"}};
//指定school数据库需要分片的集合和片键
> db.runCommand({shardcollection:"school.user",key:{id:1}})
//查看状态
> db.user.stats()
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。