CentOS7下ELK7环境搭建入门篇

DevOps 2020年06月02日

本篇笔记记录了在Linux的CentOS7发行版下,安装Elasticsearch7、Logstash7、Kibana7的全过程,并安装了Nginx,一篇文章入门ELK7环境搭建和对Nginx访问日志的收集和展示

工作准备

虚拟机
192.168.75.238 - 安装Elasticsearch,Kibana
192.168.75.239 - 安装Logstash,Nginx
系统信息
CentOS Linux release 7.8.2003 (Core)
firewalld - 已关闭
selinux - 已关闭

下载RPM安装包

192.168.75.238上执行

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.7.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.7.0-x86_64.rpm

192.168.75.239上执行

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.7.0.rpm
wget http://nginx.org/packages/rhel/7/x86_64/RPMS/nginx-1.8.1-1.el7.ngx.x86_64.rpm

安装Elasticsearch(192.168.75.238)

安装公共签名密钥

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

安装Elasticsearch(已包含匹配的jdk,所以不需要单独安装jdk)

rpm --install elasticsearch-7.7.0-x86_64.rpm

设置开机自启动

systemctl daemon-reload
systemctl enable elasticsearch

启动Elasticsearch

systemctl start elasticsearch

systemd查看Elasticsearch运行状态

systemctl status elasticsearch


curl查看Elasticsearch运行状态

修改配置文件

vim /etc/elasticsearch/elasticsearch.yml

如下

#节点名称
node.name: node-1
#监听IP
network.host: 0.0.0.0
#初始化设置
cluster.initial_master_nodes: ["node-1"]

重启Elasticsearch

systemctl restart elasticsearch

浏览器访问Elasticsearch

安装Kibana(192.168.75.238)

rpm --install kibana-7.7.0-x86_64.rpm

设置开机自启动

systemctl daemon-reload
systemctl enable kibana

修改配置文件

vim /etc/kibana/kibana.yml

配置如下

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.75.238:9200"]
i18n.locale: "zh-CN"

启动Kibana

systemctl start kibana

查看运行状态

systemctl status kibana


浏览器访问Kibana

添加样例数据,我选择的是“Sample web logs”
然后展开左侧菜单并点击“Discover”

Kibana和Elasticsearch已经连通状态

安装Nginx(192.168.75.239)(为测试Logstash收集日志,本步骤非必须)

wget http://nginx.org/packages/rhel/7/x86_64/RPMS/nginx-1.8.1-1.el7.ngx.x86_64.rpm
rpm --install nginx-1.8.1-1.el7.ngx.x86_64.rpm

设置开机自启动

systemctl daemon-reload
systemctl enable nginx

启动Nginx

systemctl start nginx

浏览器访问Nginx

Nginx日志默认位置

/var/log/nginx/access.log

查看Nginx访问日志

安装Logstash(192.168.75.239)

安装Java
Logstash的RPM包不含Java,和Elasticsearch又是不同的虚拟机,安装Java吧,官方要求Java 8 or Java 11

yum search openjdk


我的源里有,如果没有的话需要去官方下载,或设置新的安装源

yum install java-11-openjdk

安装公共签名密钥

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

安装Logstash

rpm -ivh logstash-7.7.0.rpm

设置Logstash开机自启动

systemctl daemon-reload
systemctl enable logstash

启动Logstash

systemctl start logstash

查看运行状态

systemctl status logstash

Logstash收集Nginx访问日志并输出到Elasticsearch

设置日志文件权限

chmod -R 755 /var/log/nginx/access.log

查看Elasticsearch现有索引

创建Logstash配置文件

vim /etc/logstash/conf.d/nginx_log.conf

输入以下配置

input {
    file {
        path => ["/var/log/nginx/access.log"]
        start_position => "beginning"
    }
 }

filter {
        grok {
                match => { "message" => "%{COMBINEDAPACHELOG}" }
        }
}

output {
        elasticsearch {
                hosts => ["192.168.75.238:9200"]
                index => "nginx-access"
        }
}

重启Logstash

systemctl restart logstash

查看运行状态

systemctl status logstash


我们看到了Logstash的启动过程,运行了Pipelines插件,并向默认的sincedb_path目录写入了监听信息等
浏览器再次访问Nginx
再次查看Elasticsearch现有索引

nginx-access - 便是我们输出到Elasticsearch的索引
回到Kibana,展开左侧菜单,点击“management”
Kibana区块点击“索引模式”,点击“创建索引模式”按钮
输入“nginx-access”,点击“下一步”

选择时间字段为“@timestamp”,点击“创建索引模式”按钮

展开左侧菜单,点击“Discover”,选择我们刚才创建的索引

Logstash同步过来的日志

完毕!