#!/bin/bash # Configure Logstash for Odin # echo -e "\e[93m" log() { echo -e "\t\e[96m[*]${1}\e[93m" } export IP=$(ip route | awk '/src/{print $9}') echo "logstash" >/etc/hostname echo -e "${IP}\tlogstash" >> /etc/hosts deluser -q --remove-home ubuntu apt-get update || exit 1 apt-get --purge remove snapd lxd -y apt-get upgrade -y apt-get install -y htop wget default-jre prometheus-node-exporter wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add - echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" >/etc/apt/sources.list.d/elastic-5.x.list apt-get update apt-get install logstash -y apt-get clean cat >/etc/logstash/conf.d/odin.conf < "kafka:9092" topics => ["conn", "dhcp", "dns", "files", "http", "intel", "notice", "software", "ssh", "ssl", "weird", "x509", "stats", "smtp", "fsf"] consumer_threads => 4 decorate_events => true codec => "json" type => "odin" } } filter{ mutate { remove_field => ["[kafka][key]", "[beat]"] } if [id.orig_h] { geoip { source => "id.orig_h" target => "geoip" } if ![geoip.ip] { if [id.resp_h] { geoip { source => "id.resp_h" target => "geoip" } } } } mutate { remove_tag => ["_geoip_lookup_failure"] } } output{ if [type] == "odin" { elasticsearch { hosts => ["elasticsearch:9200"] index => "odin-%{+YYYY.MM.dd}" template => "/etc/logstash/odin_mapping.json" template_name => "odin-*" template_overwrite => true } } if "fsf" in [tags] { elasticsearch { hosts => ["elasticsearch:9200"] index => "fsf-%{+YYYY.MM.dd}" template => "/etc/logstash/fsf_mapping.json" template_name => "fsf-*" template_overwrite => true } } } EOF cat >> /etc/logstash/odinmap.gz.b64 < /etc/logstash/odin_mapping.json.gz gunzip /etc/logstash/odin_mapping.json.gz cat > /etc/logstash/fsfmap.gz.b64 < /etc/logstash/fsf_mapping.json.gz gunzip /etc/logstash/fsf_mapping.json.gz rm /etc/logstash/odinmap.gz.b64 rm /etc/logstash/fsfmap.gz.b64 apt-get install -y prometheus-node-exporter systemctl enable logstash systemctl start logstash echo -e "\e[0m"