Category Archives: Programming

Query Elasticsearch Cluster in Filter Section when using Logstash

Query Elasticsearch Cluster in Filter Section when using Logstash

Query Elasticsearch Cluster in Filter Section when using Logstash

Query Elasticsearch Cluster in Filter Section when using Logstash

     This article details about on how to query Elasticsearch from Logstash when processing logs or when using any other inputs. input { file { } #Any other inputs } filter { grok { pattern => “%{NUMBER:attribute1} %{GREEDYDATA:attribute2}” } elasticsearch { hosts => [“HOSTNAME:9200/index_name/type_name”] query => ‘field_name.sub_field:%{attribute1}’ fields => [“attribute3_elasticsearch”, “attribute3_new”,”attribute4_elasticsearch”, “attribute4_new”] #We can… Read More »

Custom Elasticsearch Index Name/Type Name based on Events in Logstash

Custom Elasticsearch Index NameType Name based on Events in Logstash

Custom Elasticsearch Index Name/Type Name based on Events in Logstash

Custom Elasticsearch Index NameType Name based on Events in Logstash

     The article details on how to use custom Index/Type name when using Logstash for proessing logs and Elasticsearch as Output. input { file { } } filter { grok { pattern => “%{NUMBER:attribute1} %{GREEDYDATA:attribute2}” } ruby { code => ‘ event[“custom_index_name”] = event[“attribute1”] event[“custom_type_name”] = event[“attribute2”] ‘ #You can also set an field… Read More »

Create Custom Filter/Plugin to Emit New Events Manually in Logstash

Create Custom FilterPlugin to Emit New Events Manually in Logstash

Create Custom Filter/Plugin to Emit New Events Manually in Logstash

Create Custom FilterPlugin to Emit New Events Manually in Logstash

     This article details about on how to create new events using custom filters in Logstash. There is already a SPLIT filter available in Logstash which can be used to split a single event into multiple events based on a delimiter. But that will not suit us in all the cases. So below is… Read More »