hdfs使用手册balancer（数据平衡）命令详解

1、数据均衡服务（Rebalancing Server）首先要求 NameNode 生成 DataNode 数据分布分析报告,获取每个DataNode磁盘使用情况 2、Rebalancing Server汇总需要移动的数据分布情况，计算具体数据块迁移路线图。数据块迁移路线图，确保网络内最短路径 3、开始数据块迁移任务，Proxy Source Data Node复制一块需要移动数据块 4、将复制的数据块复制到目标DataNode上 5、删除原始数据块 6、目标DataNode向Proxy Source Data Node确认该数据块迁移完成 7、Proxy Source Data Node向Rebalancing Server确认本次数据块迁移完成。然后继续执行这个过程，直至集群达到数据均衡标准

[root@hadoop-master ~]# su - hdfs 2.查看当前的数据分布情况 [hdfs@hadoop-master ~]$ hdfs dfsadmin -report > /tmp/bq [hdfs@hadoop-master ~]$ cat /tmp/bq Configured Capacity: 273287419086 (254.52 GB) Present Capacity: 209643254756 (195.25 GB) DFS Remaining: 199579415524 (185.87 GB) DFS Used: 10063839232 (9.37 GB) DFS Used%: 4.80% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 配置容量：273287419086（254.52 GB）现有容量：209643254756（195.25 GB） DFS剩余：199579415524（185.87 GB）使用的DFS：10063839232（9.37 GB）使用DFS％：4.80％在复制块下：0 具有损坏副本的块：0 缺少块：0 缺少块（复制因子1）：0 ------------------------------------------------- Live datanodes (3): 实时数据节点（3）： Name: 192.168.0.117:50010 (hadoop-node01) Hostname: hadoop-node01 Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354603520 (3.12 GB) Non DFS Used: 12246245786 (11.41 GB) DFS Remaining: 69809631564 (65.02 GB) DFS Used%: 3.68% DFS Remaining%: 76.63% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:45:24 CST 2019 名称：192.168.0.117：50010（hadoop-node01）主机名：hadoop-node01 机架：/默认退役状态：正常配置容量：91095806362（84.84 GB）使用的DFS：3354603520（3.12 GB）非DFS使用：12246245786（11.41 GB）剩余DFS：69809631564（65.02 GB）使用DFS％：3.68％ DFS剩余％：76.63％配置的缓存容量：4294967296（4 GB）使用的缓存：0（0 B）剩余高速缓存：4294967296（4 GB）使用的缓存％：0.00％剩余高速缓存％：100.00％ Xceivers：10 最后联系人：Mon Jan 21 10:45:24 CST 2019 Name: 192.168.0.118:50010 (hadoop-master) Hostname: hadoop-master Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354632192 (3.12 GB) Non DFS Used: 29517959578 (27.49 GB) DFS Remaining: 52537889100 (48.93 GB) DFS Used%: 3.68% DFS Remaining%: 57.67% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:45:24 CST 2019 Name: 192.168.0.121:50010 (hadoop-node02) Hostname: hadoop-node02 Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354603520 (3.12 GB) Non DFS Used: 4823982490 (4.49 GB) DFS Remaining: 77231894860 (71.93 GB) DFS Used%: 3.68% DFS Remaining%: 84.78% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:45:24 CST 2019 3.使用命令平衡数据 [hdfs@hadoop-master ~]$ hdfs balancer 19/01/21 10:49:19 INFO balancer.Balancer: namenodes = [hdfs://vg-cdh-test] 19/01/21 10:49:19 INFO balancer.Balancer: parameters = Balancer.Parameters [BalancingPolicy.Node, threshold = 10.0, max idle iteration = 5, number of nodes to be excluded = 0, number of nodes to be included = 0, run during upgrade = false] Time Stamp Iteration# Bytes Already Moved Bytes Left To Move Bytes Being Moved 19/01/21 10:49:21 INFO net.NetworkTopology: Adding a new node: /default/192.168.0.117:50010 19/01/21 10:49:21 INFO net.NetworkTopology: Adding a new node: /default/192.168.0.118:50010 19/01/21 10:49:21 INFO net.NetworkTopology: Adding a new node: /default/192.168.0.121:50010 19/01/21 10:49:21 INFO balancer.Balancer: 0 over-utilized: [] 19/01/21 10:49:21 INFO balancer.Balancer: 0 underutilized: [] The cluster is balanced. Exiting... 2019-1-21 10:49:21 0 0 B 0 B -1 B 2019-1-21 10:49:21 Balancing took 2.738 seconds 4.查看数据平衡后的数据分布情况 [hdfs@hadoop-master ~]$ hdfs dfsadmin -report > /tmp/bh [hdfs@hadoop-master ~]$ cat /tmp/bh Configured Capacity: 273287419086 (254.52 GB) Present Capacity: 209660106924 (195.26 GB) DFS Remaining: 199596266468 (185.89 GB) DFS Used: 10063840456 (9.37 GB) DFS Used%: 4.80% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 ------------------------------------------------- Live datanodes (3): Name: 192.168.0.117:50010 (hadoop-node01) Hostname: hadoop-node01 Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354603928 (3.12 GB) Non DFS Used: 12246663170 (11.41 GB) DFS Remaining: 69809213772 (65.01 GB) DFS Used%: 3.68% DFS Remaining%: 76.63% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:50:10 CST 2019 Name: 192.168.0.118:50010 (hadoop-master) Hostname: hadoop-master Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354632600 (3.12 GB) Non DFS Used: 29501419522 (27.48 GB) DFS Remaining: 52554428748 (48.95 GB) DFS Used%: 3.68% DFS Remaining%: 57.69% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:50:10 CST 2019 Name: 192.168.0.121:50010 (hadoop-node02) Hostname: hadoop-node02 Rack: /default Decommission Status : Normal Configured Capacity: 91095806362 (84.84 GB) DFS Used: 3354603928 (3.12 GB) Non DFS Used: 4823252994 (4.49 GB) DFS Remaining: 77232623948 (71.93 GB) DFS Used%: 3.68% DFS Remaining%: 84.78% Configured Cache Capacity: 4294967296 (4 GB) Cache Used: 0 (0 B) Cache Remaining: 4294967296 (4 GB) Cache Used%: 0.00% Cache Remaining%: 100.00% Xceivers: 10 Last contact: Mon Jan 21 10:50:10 CST 2019 ———————————————————————————————————————————————————————————————————————————————— 5.对比数据平衡前后的报告信息 [hdfs@hadoop-master ~]$ diff /tmp/bq /tmp/bh 2,4c2,4 < Present Capacity: 209643254756 (195.25 GB) < DFS Remaining: 199579415524 (185.87 GB) < DFS Used: 10063839232 (9.37 GB) --- > Present Capacity: 209660106924 (195.26 GB) > DFS Remaining: 199596266468 (185.89 GB) > DFS Used: 10063840456 (9.37 GB) 19,21c19,21 < DFS Used: 3354603520 (3.12 GB) < Non DFS Used: 12246245786 (11.41 GB) < DFS Remaining: 69809631564 (65.02 GB) --- > DFS Used: 3354603928 (3.12 GB) > Non DFS Used: 12246663170 (11.41 GB) > DFS Remaining: 69809213772 (65.01 GB) 30c30 < Last contact: Mon Jan 21 10:45:24 CST 2019 --- > Last contact: Mon Jan 21 10:50:10 CST 2019 38,40c38,40 < DFS Used: 3354632192 (3.12 GB) < Non DFS Used: 29517959578 (27.49 GB) < DFS Remaining: 52537889100 (48.93 GB) --- > DFS Used: 3354632600 (3.12 GB) > Non DFS Used: 29501419522 (27.48 GB) > DFS Remaining: 52554428748 (48.95 GB) 42c42 < DFS Remaining%: 57.67% --- > DFS Remaining%: 57.69% 49c49 < Last contact: Mon Jan 21 10:45:24 CST 2019 --- > Last contact: Mon Jan 21 10:50:10 CST 2019 57,59c57,59 < DFS Used: 3354603520 (3.12 GB) < Non DFS Used: 4823982490 (4.49 GB) < DFS Remaining: 77231894860 (71.93 GB) --- > DFS Used: 3354603928 (3.12 GB) > Non DFS Used: 4823252994 (4.49 GB) > DFS Remaining: 77232623948 (71.93 GB) 68c68 < Last contact: Mon Jan 21 10:45:24 CST 2019 --- > Last contact: Mon Jan 21 10:50:10 CST 2019

在生产上实际的操作如下： hdfs dfsadmin -fs hdfs://uhadoop-mzwc2w-master2:8020 -setBalancerBandwidth 3145728000 [hadoop@uhadoop-mzwc2w-master1 ~]$ hdfs dfsadmin -fs hdfs://uhadoop-mzwc2w-master2:8020 -setBalancerBandwidth 3145728000 Balancer bandwidth is set to 3145728000 [hadoop@uhadoop-mzwc2w-master1 ~]$ hdfs dfsadmin -fs hdfs://uhadoop-mzwc2w-master1:8020 -setBalancerBandwidth 3145728000 Balancer bandwidth is set to 3145728000 //一定要2个节点namenode节点 master1 master2 都要执行在生产上2019/7/22 星期一我在master1 和master2 都执行balancer [hadoop@uhadoop-mzwc2w-master1 majihui0718]$ nohup hdfs balancer > balancer.log & //生产上这样处理 [hadoop@uhadoop-mzwc2w-master2 majihui0722]$ pwd /home/hadoop/majihui0722 [hadoop@uhadoop-mzwc2w-master2 majihui0722]$ ll total 596 -rw-r--r-- 1 hadoop wheel 609794 Jul 22 14:11 balancer.log 我们是1000M的网，给300M的带宽用于datanode的数据balance

相关阅读