Introduction
Code on Github: Elasticsearch and Beats
This tutorial is a continuation of Heartbeat with Elasticsearch Part 1 Install and Secure.
In this tutorial, we will set up Alerts in Kibana, and use Logstash to email you alerts.
Requirements
Resources from Heartbeat with Elasticsearch Part 1 Install and Secure.
Steps
Step 1 - Install Metricbeat and Filebeat on the same IP [02:25]
You can follow along with the current video to do the installation, or if you want to install Metricbeat and Filebeat individually you can refer to the videos on Metricbeat Installation - Part 1 and Filebeat Installation - Part 1 to install them.
Step 2 - Configure Kibana [09:15]
Go to Kibana in the Stack Management > Rules and Connectors. If you see Create your first rule, then skip this step. If you see Additional setup required, then continue with this step.
Kibana Alert Setup
Go to your Kibana server shell window and type:
/usr/share/kibana/bin/kibana-keystore add xpack.encryptedSavedObjects.encryptionKey
When prompted, paste in a secret that is at least 32 characters long.
Then restart kibana:
systemctl restart kibana
Step 3 - Setup Rule [13:00]
Visit Kibana in your browser again and we can either visit Stack Management > Rules and Connectors
Kibana Alert First Rule
Or Observability > Uptime > Monitors.
Kibana Observability Uptime Monitor Alert Rule
Now you should be allowed to create your first rule.
We will create a rule to monitor for any downtime.
Click on Create rule and then fill out the initial fields:
Name: monitor uptime
Check every: 2 minute
Notify: On check intervals
Creating rule type
Then finalize the reminaing few points as show in this image:
Creating rule query
Press save.
Step 4 - Prepare Email Alert Message - Create Server Log Connector [16:28]
We want to output the alerts to a different location so that Logstash can email alerts on our behalf.
Click on Connectors and choose Server logs.
Kibana Alert Server Log
Click Create a connector and type anything meaningful for Connector name in the pop up.
Set the Level to anything that reflects what you deem the alert to be. You can use it as a helpful way to tell Logstash how to filter and transform messages.
If desired, edit the Message. The Message field is what will be delivered via email.
Press save.
After a few minutes, go to your Kibana server and confirm that messages appear in /var/log/kibana.log.
Step 5 - Send Emails with Logstash [20:10]
Go to your Kibana server and install Logstash with this command:
apt-get install -y logstash
Download this logstash file: Logstash file to email alerts
You can place this file anywhere you want. For now we will assume that you placed it in /root/logstash.conf.
In the output stage, fill out the appropriate values for the email connection.
To run logstash and start sending emails as they happen, type this command:
/usr/share/logstash/bin/logstash -f /root/logstash.conf