How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS/RHEL 7

If You Appreciate What We Do Here On TecMint, You Should Consider:

  1. Stay Connected to: Twitter | Facebook | Google Plus
  2. Subscribe to our email updates: Sign Up Now
  3. Use our Linode referral link if you plan to buy VPS (it starts at only $10/month).
  4. Support us via PayPal donate - Make a Donation
  5. Support us by purchasing our premium books in PDF format.
  6. Support us by taking our online Linux courses

We are thankful for your never ending support.

Gabriel Cánepa

Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

Receive Your Free Complimentary eBook NOW! -

Download Free Linux eBooks

Advanced Bash-Scripting Guide
Linux Bible
A Newbie's Getting Started Guide to Linux
Introduction to Linux - A Hands on Guide

You may also like...

13 Responses

  1. jwood says:

    I think this is my issue on the clients regarding Filebeat: “/usr/bin/filebeat[6387]: transport.go:125: SSL client failed to connect with: dial tcp x.x.x.x:5044: getsockopt: connection refused”.

  2. JWOOD says:

    I’m now having an issue with Filebeat. I can only see stats from my ElkServer. I don’t have anything from the Client Servers. when I run:

    curl -XGET ‘http://localhost:9200/filebeat-*/_search?pretty’

    My clients are on Centos 6 32-bit, so I had to use the following to install:

    curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.3.0-i686.rpm
    rpm -Uvh filebeat-1.3.0-i686.rpm

    My file: /etc/filebeat/filebeat.yml is identical to the Elk Server that works.

    Am I missing something? ie. ports, config somewhere…

    This is my output:

    curl -XGET ‘http://localhost:9200/filebeat-*/_search?pretty’
    {
    “took” : 8,
    “timed_out” : false,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “hits” : {
    “total” : 4096,
    “max_score” : 1.0,
    “hits” : [ {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo60”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:26:30 VM-LXPDElkStack unix_chkpwd[26690]: password check failed for user (root)”,
    “offset” : 0,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo66”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:38:41 VM-LXPDElkStack sshd[26896]: Accepted password for root from 172.22.2.90 port 50049 ssh2”,
    “offset” : 616,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo67”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:38:41 VM-LXPDElkStack sshd[26896]: pam_unix(sshd:session): session opened for user root by (uid=0)”,
    “offset” : 721,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo6-“,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:49:11 VM-LXPDElkStack polkitd[840]: Registered Authentication Agent for unix-process:27095:32300900 (system bus name :1.1289 [/usr/bin/pkttyagent –notify-fd 5 –fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_CA.UTF-8)”,
    “offset” : 1044,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo6_”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:49:11 VM-LXPDElkStack polkitd[840]: Unregistered Authentication Agent for unix-process:27095:32300900 (system bus name :1.1289, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_CA.UTF-8) (disconnected from bus)”,
    “offset” : 1308,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo7D”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 09:55:39 VM-LXPDElkStack polkitd[840]: Registered Authentication Agent for unix-process:27199:32339699 (system bus name :1.1294 [/usr/bin/pkttyagent –notify-fd 5 –fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_CA.UTF-8)”,
    “offset” : 1863,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo7K”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 10:03:16 VM-LXPDElkStack sshd[27344]: Accepted password for root from 172.22.2.90 port 50171 ssh2”,
    “offset” : 3189,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo7M”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 10:03:16 VM-LXPDElkStack sshd[27344]: pam_unix(sshd:session): session closed for user root”,
    “offset” : 3403,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo7a”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 10:33:33 VM-LXPDElkStack sshd[27973]: Accepted password for root from 172.22.2.90 port 51083 ssh2”,
    “offset” : 6348,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    }, {
    “_index” : “filebeat-2016.09.09”,
    “_type” : “syslog”,
    “_id” : “AVcPPp7AigWXUuN_Qo7b”,
    “_score” : 1.0,
    “_source” : {
    “@timestamp” : “2016-09-09T13:59:04.400Z”,
    “beat” : {
    “hostname” : “VM-LXPDElkStack”,
    “name” : “VM-LXPDElkStack”
    },
    “count” : 1,
    “fields” : null,
    “input_type” : “log”,
    “message” : “Sep 6 10:33:33 VM-LXPDElkStack sshd[27973]: pam_unix(sshd:session): session opened for user root by (uid=0)”,
    “offset” : 6453,
    “source” : “/var/log/secure”,
    “type” : “syslog”
    }
    } ]
    }
    }

    • Gabriel A. Cánepa says:

      You pasted the output of curl, but I can’t see your configuration files. Did you intend to share those as well?

  3. RedL says:

    thank you for nice tutorial bro :)
    i wanna ask you, can we send logs to logstash dynamically?
    because in my case, my filebeat not pushing the logs to logstash dynamically.
    so, i have to manually restart filebeat each and everytime so as to send the
    logs from filebeat to logstash.
    So please let me know about this.
    Thankyou

    • Gabriel A. Cánepa says:

      @RedL,
      If you followed the steps outlined in this article, the logs should be pushed to the server automatically without intervention on your side. Check your setup and make sure it matches each step provided here.

  4. JWood says:

    I’m getting the following error while installing filebeat:

    yum install filebeat
    Loaded plugins: downloadonly, fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirrors.cmich.edu
    * epel: mirror.steadfast.net
    * extras: centos.mirror.lstn.net
    * rpmfusion-free-updates: lug.mtu.edu
    * rpmfusion-free-updates-testing: lug.mtu.edu
    * rpmfusion-nonfree-updates: lug.mtu.edu
    * rpmfusion-nonfree-updates-testing: lug.mtu.edu
    * updates: mirror.cisp.com
    http://mirrors.cmich.edu/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://holmes.umflint.edu/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://mirrors.seas.harvard.edu/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://ftp.linux.ncsu.edu/pub/CentOS/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://ftpmirror.your.org/pub/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://centos.corenetworks.net/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://mirrors.arsc.edu/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://mirrors.gigenet.com/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    http://mirror.kentdigital.net/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 – “Couldn’t resolve host ‘mirror.kentdigital.net'”
    Trying other mirror.
    http://mirrors.einstein.yu.edu/centos/6.5/os/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 7 – “couldn’t connect to host”
    Trying other mirror.
    https://packages.elastic.co/beats/yum/el/i386/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 – “The requested URL returned error: 404 Not Found”
    Trying other mirror.
    Error: Cannot retrieve repository metadata (repomd.xml) for repository: filebeat. Please verify its path and try again

  5. Aamir says:

    I am getting yum failure while installing elasticsearch. Below is my elasticsearch.repo file and error log

    -> elasticsearch.repo
    elasticsearch-2.4]
    name=Elasticsearch repository for 2.4 packages
    baseurl=http://packages.elastic.co/elasticsearch/2.4/centos
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1

    — Yum error logs —

    http://packages.elastic.co/elasticsearch/2.4/centos/repodata/repomd.xml: [Errno 14] HTTP Error 404 – Not Found

    • Gabriel A. Cánepa says:

      You are obviously using a different repository configuration than the one we’re using in this article. Take a look at step 2 near the top of this post and you will see why it’s not working :).

Got something to say? Join the discussion.

Your email address will not be published. Required fields are marked *

Join Over 150K+ Linux Users
  1. 100,756
  2. 5,113
  3. 36,418

Enter your email to get latest Linux Howto's