12:56 PM
Installing and Configuring Elasticsearch,
Logstash and Kibana ELK Stack on Windows
In this post we will
install Elasticsearch, Logstash
and Kibana and configure the sample log file.
Quick description
on each item.
Elasticsearch – A search Engine,Elasticsearch is a
distributed, RESTful search and analytics engine capable of solving a growing
number of use cases. As the heart of the Elastic Stack, it centrally stores
your data so you can discover the expected and uncover the unexpected.
Logstash - Logstash is an open source, server-side data processing pipeline
that ingests data from a multitude of sources simultaneously, transforms it,
and then sends it to your favorite “stash.” (Ours here is Elasticsearch, naturally.)
Kibana - Kibana lets you visualize your Elasticsearch data and navigate
the Elastic Stack
Installing Elasticsearch
Download url :
https://www.elastic.co/downloads/elasticsearch
Used in this post : https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.0.zip
Extract the folder into D:\ELK\elasticsearch-5.6.0. Open windows cmd and goto D:\ELK\elasticsearch-5.6.0\bin
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
D:\ELK\elasticsearch-5.6.0\bin>elasticsearch-service.bat
Usage: elasticsearch-service.bat
install|remove|start|stop|manager [SERVICE_ID]
D:\ELK\elasticsearch-5.6.0\bin>elasticsearch-service.bat
install
Installing service
:
"elasticsearch-service-x64"
Using JAVA_HOME (64-bit):
"C:\Program Files\Java\jdk1.8.0_131"
The service 'elasticsearch-service-x64' has been installed.
Once installation is done we can create a service by doing
the below.
D:\ELK\elasticsearch-5.6.0\bin>elasticsearch-service.bat
Usage: elasticsearch-service.bat
install|remove|start|stop|manager [SERVICE_ID]
D:\ELK\elasticsearch-5.6.0\bin>elasticsearch-service.bat
manager
Successfully started service manager for
'elasticsearch-service-x64'.
D:\ELK\elasticsearch-5.6.0\bin>
{
"name" :
"wNh1HLd",
"cluster_name"
: "elasticsearch",
"cluster_uuid"
: "EZwefHS3QnOAAZ1_Ju7yyQ",
"version" : {
"number" :
"5.6.0",
"build_hash"
: "781a835",
"build_date"
: "2017-09-07T03:09:58.087Z",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" :
"You Know, for Search"
}
Installing Logstash
Download url : https://www.elastic.co/downloads/logstash
Used in this post : https://artifacts.elastic.co/downloads/logstash/logstash-5.6.2.zip
Logstash can be
installed using a service manager called Non-Sucking Service Manager -NSSM.
I
have downloaded this package and extracted into same folder as below.
D:\ELK\nssm-2.24
Browse
to and select the logstash.bat file. The “Startup” directory field below
is completed automatically. In the “Arguments” field, enter as below
Installing Kibana
Download url : https://www.elastic.co/downloads/kibana
Used in this post : https://artifacts.elastic.co/downloads/kibana/kibana-5.6.2-windows-x86.zip
Kibana is
also installed using NSSM using the same steps as below.
Once all installation is done, verify the services are running
in Task Manager.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
D:\ELK\logstash-5.6.0\bin>logstash -e 'input { stdin { } }
output { stdout {} }'
Sending Logstash's logs to D:/ELK/logstash-5.6.0/logs which is
now configured via log4j2.properties
[2017-09-14T23:28:20,785][INFO ][logstash.modules.scaffold]
Initializing module {:module_name=>"fb_apache",
:directory=>"D:/ELK/logstash-5.6.0/modules/fb_apache/configuration"}
[2017-09-14T23:28:20,814][INFO ][logstash.modules.scaffold]
Initializing module {:module_name=>"netflow",
:directory=>"D:/ELK/logstash-5.6.0/modules/netflow/configuration"}
[2017-09-14T23:28:20,823][INFO
][logstash.setting.writabledirectory] Creating directory
{:setting=>"path.queue",
:path=>"D:/ELK/logstash-5.6.0/data/queue"}
[2017-09-14T23:28:20,831][INFO
][logstash.setting.writabledirectory] Creating directory
{:setting=>"path.dead_letter_queue",
:path=>"D:/ELK/logstash-5.6.0/data/dead_letter_queue"}
[2017-09-14T23:28:20,851][INFO ][logstash.agent ] No persistent UUID file found.
Generating new UUID
{:uuid=>"9477f21e-0582-49c3-a50c-ff181535f93f",
:path=>"D:/ELK/logstash-5.6.0/data/uuid"}
[2017-09-14T23:28:21,046][INFO ][logstash.pipeline ] Starting pipeline
{"id"=>"main", "pipeline.workers"=>4,
"pipeline.batch.size"=>125, "pipeline.batch.delay"=>5,
"pipeline.max_inflight"=>500}
[2017-09-14T23:28:21,089][INFO ][logstash.pipeline ] Pipeline main started
The stdin plugin is now waiting for input:
[2017-09-14T23:28:21,186][INFO ][logstash.agent ] Successfully started Logstash API
endpoint {:port=>9600}
hello world
2017-09-14T17:58:52.706Z DLXL51LLPC0019 hello world
[2017-09-14T23:29:13,215][WARN ][logstash.runner ] SIGINT received. Shutting down the
agent.
[2017-09-14T23:29:13,229][WARN ][logstash.agent ] stopping pipeline
{:id=>"main"}
Terminate batch job (Y/N)?
^C
D:\ELK\logstash-5.6.0\bin>
You can refer this to load sample data into kibana.
https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html
You can verify all indices from here http://localhost:9200/_cat/indices
All data from http://localhost:9200/_all
once you loaded the data, you can see the indexed data as below
That's All ! You can play around it
Reference:
https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html
https://www.elastic.co/guide/en/kibana/current/tutorial-discovering.html
No comments
Post a Comment