Elastic - NETWAYS GmbH

Elasticsearch

The Open Source Enterprise search server

Elasticsearch is a distributed search and analytics server, which represents the core of the Elastic Stack. The communication between Elasticsearch and the service consumer is almost exclusively based on JSON via REST interface. This has the advantage that even smaller read and write operations can be tested and developed using CURL, without having to resort to a heavy-weight API.



Scaling

Especially Elasticsearch stands out because it brings along everything that is needed for scaling and distribution of the installation. If an increase is necessary due to increasing load, then Elasticsearch only needs to be notified of a new server. It then takes care of the redistribution of data and inquiries independently.



Performance

All incoming data is immediately indexed by Elasticsearch. Whether log information, numeric or geographical data. Since everything is indicated, boredom never arises. This means that all data can be retrieved and processed at tremendously high speed. And of course, the whole comes with a frantic fast full text search in the luggage.

Logstash

Flexible Log- and Eventmanagement

Logstash is an open source log management solution that specializes in channeling, filtering and distribution of log and event information. It supports a large number of input and output formats and thus integrates into almost any IT environment. In short, Logstash is the open source solution for managing and analyzing log information and the tool of choice to address the increasing volume of information in an auditable manner. Logstash has a variety of input, filtering and output plug-ins. Thus all events and log messages available in your network can be received, processed and forwarded.



Integration

Logstash supports a variety of inputs and outputs to process and forward the various information from a variety of sources. In addition to standards such as syslog, pipe and SNMP traps, various message brokers are also supported. The integration of application logs is thus possible without further detours and requires no installation of additional third-party software.

input { stdin { } }

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

Kibana

Your look in the Elastic Stack

Kibana visualizes and analyzes the data stored in Elasticsearch. Sophisticated filtering option allows the construction of dashboards for all data stored in Elasticsearch. Working with Kibana is the real reward for the effort invested in collecting the log and event information in advance. The access to all stored information is very fast and no deeper knowledge of a query language like SQL is necessary. Of course understanding the underlying intersections of the queried data is an advantage in order to get to the goal quickly, but even without it, visual viewing of the data is simply fun.

Geoinformation can easily be used in Kibana to create a local data reference. With the help of the Geoip filter plug-in, log data can be enriched with corresponding geoinformation based on the existing IP address.

Elasticsearch is more than a pure index database and has a powerful time series engine, the so-called Timeseries. This allows performance information from log data to be stored and analyzed for longer periods of time.

Beats

Collect, analyze and send

Beats is the platform for building lightweight data collectors for a wide variety of data types.

There are different Beats with a multitude of possibilities for reading in data, from enriching or processing data in advance by processors, through a multitude of modules with prefabricated processing chains for direct submission in Elasticsearch, to ready-made dashboards for a wide variety of products. In addition, there is the possibility of not only sending data to Elasticsearch and Logstash, because the absolute plus is the possibility of enriching the data with additional information in the form of document fields and tags while reading it in before sending it.

Beats not only convince with their integrations, but also with their very small footprint on the system.

Like every member of the Elastic Stack toolbox, Beats are a standard tool for processing information and events of all kinds, codecs and sources. So it is almost impossible to get around tBeats when collecting information with the Elastic Stack or related tools.



Filebeat

Filebeat is the tool for collecting log information from a variety of sources in a wide variety of formats. The range of inputs goes from plain file to syslog. This also predestines Filebeat as a site collector in order to then deliver it to a central Elasticsearch cluster. Yes, you read that right – Elasticsearch! Because with its large number of modules for a large number of products in a modern infrastructure, Filebeat provides already processed data or processing pipelines for Elasticsearch, as well as index patterns and Kibana dashboards – and of course everything is ECS-compliant. These mechanisms guarantee perfect functioning with, for example, the new Elastic Security Integration for a comprehensive IT security evaluation.



Winlogbeat

For Windows Eventlogs the Winlogbeats represent a special variant of the Filebeats, which was developed especially for the collection of Eventlogs. The result is a lightweight solution for centrally storing Windows log data in Elasticsearch with the usual filtering and enrichment capabilities with Logstash.

Starterpaket

The simple beginning of something big

With our starter packs we want to simplify the entry into the log management with the Elastic Stack (Elasticsearch, Logstash and Kibana) and offer a cost-effective way to get to know the open source system without first having to go into large financial inputs, such as commercial products often the case. our experienced consultants will be there for 4 or 7 days, set up the system directly on site and provide the basics for further operation. The package is settled at a fixed price and there are no additional costs.

Elastic Stack Starterpaket Standard

We recommend that you visit our Elastic Stack Training before booking the Elastic Stack Starter Pack Standard. This training is already included in the Elastic Stack Starter Package Premium.

Joint workshop on log and event management
Introduction to the components Logstash, Elasticsearch, Kibana and Beats
Installation and basic configuration on customer hardware
Exemplary integration of customer logs and evaluation with the help of Kibana

Elastic Stack Starterpaket Premium

Elastic stack training (4 days for 3 participants – additional participants possible for an extra charge – 6 max.) *
Joint workshop on log and event management
Introduction to the components Logstash, Elasticsearch, Kibana and Redis Installation and basic configuration on customer hardware
Exemplary integration of customer logs and evaluation with the help of Kibana

* The training will cover the scope of the Elastic Stack training and will be conducted on-site (including rental notebooks, training material and attendee certificates)

News

Posts from our Blog

Warum OpenSearch? Was ist OpenSearch? OpenSearch VS. Elastic

by Daniel Neuberger | Jun 10, 2021 | Elastic Stack, Graylog, NETWAYS

Es fing bereits 2015 an zu brodeln, als AWS seinen neuen SaaS-Dienst „AWS Elasticsearch Services“ veröffentlichte und später dann 2019 das Produkt OpenDistro erschien. Bei zuletzt genanntem Produkt erschließt sich mir bis heute nicht, warum ich dieses der X-Pack Basic...

Elastic Stack Trainings – Last Minute Call

by Natalie Regn | Jan 22, 2021 | Elastic Stack, Trainings

Möchtest Du mehr rund um Elasticsearch, Logstash, Kibana & Beats erfahren? Schau Dir doch mal unsere Elastic Stack Schulungen an. Du hast bereits am 02. Februar 2021 die Chance, in die Elastic Stack Welt einzutauchen. Selbstverständlich hast Du, vor allem in...