Filebeat实现日志压缩存储的常用方式
一、传输层压缩
output.logstash:
hosts: ["logstash.example.com:5044"]
compress: true
output.elasticsearch:
hosts: ["es.example.com:9200"]
compression_level: 5 # 可选,取值 1-9,默认 0(不压缩)
二、存储层压缩
PUT _template/filebeat_compress
{
"index_patterns": ["filebeat-*"],
"settings": {
"index.codec": "best_compression",
"index.number_of_shards": 1,
"index.number_of_replicas": 1
}
}
三、源日志文件压缩归档
/var/log/myapp.log {
daily
rotate 7
compress
missingok
notifempty
create 640 root root
}
四、实践建议