CentOS中HBase如何进行资源调度 - 问答

在CentOS系统中，HBase的资源调度主要依赖于YARN（Yet Another Resource Negotiator）来实现。以下是HBase在CentOS中进行资源调度的基本步骤：

1. 安装和配置YARN

首先，确保你的CentOS系统上已经安装了Hadoop和YARN。如果没有安装，可以参考Hadoop官方文档进行安装和配置。

安装Hadoop

wget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz
tar -xzvf hadoop-3.3.1.tar.gz -C /usr/local/

配置Hadoop

编辑/usr/local/hadoop/etc/hadoop/core-site.xml：

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

编辑/usr/local/hadoop/etc/hadoop/hdfs-site.xml：

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

编辑/usr/local/hadoop/etc/hadoop/yarn-site.xml：

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>4096</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.cpu-vcores</name>
        <value>4</value>
    </property>
</configuration>

2. 启动Hadoop集群

启动HDFS和YARN：

start-dfs.sh
start-yarn.sh

3. 配置HBase使用YARN

编辑HBase的配置文件/usr/local/hbase/conf/hbase-site.xml，添加以下配置：

<configuration>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://localhost:9000/hbase</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>localhost</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/usr/local/hbase/zookeeper</value>
    </property>
    <property>
        <name>hbase.master.info.port</name>
        <value>16000</value>
    </property>
    <property>
        <name>hbase.regionserver.info.port</name>
        <value>16020</value>
    </property>
    <property>
        <name>hbase.regionserver.handler.count</name>
        <value>100</value>
    </property>
    <property>
        <name>hbase.regionserver.global.memstore.size</name>
        <value>0.4</value>
    </property>
    <property>
        <name>hbase.regionserver.wal.codec</name>
        <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.blocksize</name>
        <value>64M</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.writer.impl</name>
        <value>org.apache.hadoop.hbase.regionserver.wal.HLogWriterFactory</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.count</name>
        <value>1</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.impl</name>
        <value>org.apache.hadoop.hbase.regionserver.wal.Syncer</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint</name>
        <value>100</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.interval</name>
        <value>300000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.size</name>
        <value>104857600</value>
    </property>
    <property>
        <name>hbase.regionserver.hlog.syncer.checkpoint.timeout</name>
        <value>6000

0 赞

0 踩