This article details about on how to create new events using custom filters in Logstash. There is already a SPLIT filter available in Logstash which can be used to split a single event into multiple events based on a delimiter. But that will not suit us in all the cases. So below is a sample Logstash configuration and a custom filter to create custom filter to emit events manually in Logstash. 

Logstash Configuration File

input {   
     	file {
     	}
}

filter {
     	grok {
          		pattern => "%{NUMBER:attribute1} %{GREEDYDATA:attribute2}" 		
     	}
     customfilter {}
}

output {
     	stdout { 
		          codec => rubydebug 
     	}
}

     In the filter section of the above configuration, we have GROK filter that reads and parses the input event into two fields namely attribute1 and attribute2. Then the control is moved to customfilter.

Ruby Custom Filter

require "logstash/filters/base"
require "logstash/namespace"
class LogStash::Filters::CustomFilter < LogStash::Filters::Base
config_name "customfilter"
milestone 1
public
def register
end

public
def filter(event)

     	#Read Events
     	attribute1_temp=event["attribute1"]
	     attribute2_temp=event["attribute2"]

     	#Your Business Logic

     	#Create New Event
     	custom_event=LogStash::Event.new()
     	custom_event["attribute1_modified"]=attribute1_temp
     	custom_event["attribute2_modified"]=attribute2_temp

     	#Emit New Event
     	yield custom_event

     	#Cancel the Main Event
     	event.cancel

end
end

     In custom filters, the parsed fields can be read using the syntax event[“attribute_name”]. Then we implement our own business logic and create new events based on the modified input fields. Thus we create a new event and assign custom fields as required and emit the event using the command yield event_name. Note that we can either emit the existing event(event in all the case) or emit the custom created event(custom_event in our case). Also make sure to cancel the existing event if you are emitting the custom_event. The main event which I refer is the event created by the Logstash. We can also use yield command in a loop to create any number of events as per the business requirement.

The customfilter is stored in path LOGSTASH_HOME/lib/logstash/filters/customfilter.rb.

Related Links :