Elastic Stack
Elastic Stack is an open source information and event management solution that focuses on centralized collection, processing, analysis and storage.
The focus is not only on classic log management, but also in the areas of IT security (audit, SIEM, and threat intelligence) and anomaly detection.
Elasticsearch
The Open Source Enterprise search server
Elasticsearch is a distributed search and analytics server, which represents the core of the Elastic Stack. The communication between Elasticsearch and the service consumer is almost exclusively based on JSON via REST interface. This has the advantage that even smaller read and write operations can be tested and developed using CURL, without having to resort to a heavy-weight API.
Scaling
Especially Elasticsearch stands out because it brings along everything that is needed for scaling and distribution of the installation. If an increase is necessary due to increasing load, then Elasticsearch only needs to be notified of a new server. It then takes care of the redistribution of data and inquiries independently.
Performance
All incoming data is immediately indexed by Elasticsearch. Whether log information, numeric or geographical data. Since everything is indicated, boredom never arises. This means that all data can be retrieved and processed at tremendously high speed. And of course, the whole comes with a frantic fast full text search in the luggage.
Logstash
Flexible Log- and Eventmanagement
Logstash is an open source log management solution that specializes in channeling, filtering and distribution of log and event information. It supports a large number of input and output formats and thus integrates into almost any IT environment. In short, Logstash is the open source solution for managing and analyzing log information and the tool of choice to address the increasing volume of information in an auditable manner. Logstash has a variety of input, filtering and output plug-ins. Thus all events and log messages available in your network can be received, processed and forwarded.
Integration
Logstash supports a variety of inputs and outputs to process and forward the various information from a variety of sources. In addition to standards such as syslog, pipe and SNMP traps, various message brokers are also supported. The integration of application logs is thus possible without further detours and requires no installation of additional third-party software.
input { stdin { } }
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
Kibana
Your look in the Elastic Stack
Kibana visualizes and analyzes the data stored in Elasticsearch. Sophisticated filtering option allows the construction of dashboards for all data stored in Elasticsearch. Working with Kibana is the real reward for the effort invested in collecting the log and event information in advance. The access to all stored information is very fast and no deeper knowledge of a query language like SQL is necessary. Of course understanding the underlying intersections of the queried data is an advantage in order to get to the goal quickly, but even without it, visual viewing of the data is simply fun.
Geoinformation can easily be used in Kibana to create a local data reference. With the help of the Geoip filter plug-in, log data can be enriched with corresponding geoinformation based on the existing IP address.
Elasticsearch is more than a pure index database and has a powerful time series engine, the so-called Timeseries. This allows performance information from log data to be stored and analyzed for longer periods of time.
Beats
Collect, analyze and send
Beats is the platform for building lightweight data collectors for a wide variety of data types.
There are different Beats with a multitude of possibilities for reading in data, from enriching or processing data in advance by processors, through a multitude of modules with prefabricated processing chains for direct submission in Elasticsearch, to ready-made dashboards for a wide variety of products. In addition, there is the possibility of not only sending data to Elasticsearch and Logstash, because the absolute plus is the possibility of enriching the data with additional information in the form of document fields and tags while reading it in before sending it.
Beats not only convince with their integrations, but also with their very small footprint on the system.
Like every member of the Elastic Stack toolbox, Beats are a standard tool for processing information and events of all kinds, codecs and sources. So it is almost impossible to get around tBeats when collecting information with the Elastic Stack or related tools.
Filebeat
Filebeat is the tool for collecting log information from a variety of sources in a wide variety of formats. The range of inputs goes from plain file to syslog. This also predestines Filebeat as a site collector in order to then deliver it to a central Elasticsearch cluster. Yes, you read that right – Elasticsearch! Because with its large number of modules for a large number of products in a modern infrastructure, Filebeat provides already processed data or processing pipelines for Elasticsearch, as well as index patterns and Kibana dashboards – and of course everything is ECS-compliant. These mechanisms guarantee perfect functioning with, for example, the new Elastic Security Integration for a comprehensive IT security evaluation.
Winlogbeat
For Windows Eventlogs the Winlogbeats represent a special variant of the Filebeats, which was developed especially for the collection of Eventlogs. The result is a lightweight solution for centrally storing Windows log data in Elasticsearch with the usual filtering and enrichment capabilities with Logstash.
Enterprise
Even more features
Get self-managed Elastic Subscriptions so that you can use your on-premise deployments optimally and with all ELK-Stack features. Just ask and we will create an individual offer for your environment.
Starterpaket
The simple beginning of something big
With our starter packs we want to simplify the entry into the log management with the Elastic Stack (Elasticsearch, Logstash and Kibana) and offer a cost-effective way to get to know the open source system without first having to go into large financial inputs, such as commercial products often the case. our experienced consultants will be there for 4 or 7 days, set up the system directly on site and provide the basics for further operation. The package is settled at a fixed price and there are no additional costs.
Elastic Stack Starterpaket Standard
- Elastic stack training (4 days for 3 participants – additional participants possible for an extra charge – 6 max.) *
- Joint workshop on log and event management
- Introduction to the components Logstash, Elasticsearch, Kibana and Redis Installation and basic configuration on customer hardware
- Exemplary integration of customer logs and evaluation with the help of Kibana
Elastic Stack Starterpaket Premium
- Elastic Stack Schulung (4 Tage für 3 Teilnehmer – weitere Teilnehmer gegen Aufpreis möglich – 6 max.) *
- Gemeinsamer Workshop zum Thema Log- und Eventmanagement
- Einführung in die Komponenten Logstash, Elasticsearch, Kibana und Redis
- Installation und Grundkonfiguration auf Kundenhardware
- Exemplarische Integration von Kundenlogs und Auswertung mit Hilfe von Kibana
* The training will cover the scope of the Elastic Stack training and will be conducted on-site (including rental notebooks, training material and attendee certificates)
News
Posts from our Blog
Mermaid zum Visualisieren von Graphen
Gerade im Umfeld von Logstash-Pipelines steht man oft vor dem Problem, wie die einzelnen Code Teile zusammenhängen. Dafür hat sich für mich Mermaid bewährt. Was mir an Mermaid besonders gefällt ist, dass man mit einer relativ einfachen Syntax Graphen definieren kann,...
NETWAYS Webinar Plan 2022 – Phase 1
Es ist wieder soweit: In 2022 starten wir wieder voll durch mit neuen Webinaren rund um unsere angebotenen Dienstleistungen von Icinga, über Graylog bis hin zu Elastic. Den Fokus legen wir dabei auch wie üblich auf einen einfachen Einstieg und im späteren Verlauf auf...
Webinars
Our Webinars for Elastic
Subscribe
Service from the market leader
We are happy to bring you our experience from more than 500 projects. Whether professional services, training or support for your environment.