We’re using Logstash to collect some of our own logs here at Sematext, because it can easily forward events to Logsene via the elasticsearch_http output. And we recently upgraded to the latest version 1.2.2.
Since we’re obsessed with performance monitoring, the first question that came up was: did the upgrade make any difference in terms of load? So we did a test to find out.
Context
Logstash runs on a JVM, so we’re already monitoring it with SPM for Java Apps. So we just put it through a steady, moderate logs of about 70 events per second for a while, running both 1.2.1 and 1.2.2, on the same machine.
The configuration remained the same on both machines:
- a file input that was tailing a single file using the multiline codec
- a couple of grok filters and a geoIP filter, that we’ll talk about in later posts
- resulting events are fed to Logsene through the elasticsearch_http output, because Logsene exposes the Elasticsearch API
Results
The biggest difference we’ve seen was in memory usage. The new version uses about 30% less memory:
Next, there was a significant difference in the amount of garbage collection going on. Again, some 30% difference in favor of 1.2.2:
We also had slightly less CPU usage. The difference wasn’t significant in our case because we didn’t have a lot of traffic: Logstash is installed on every host and there’s no “central” Logstash to process lots of data. I’m sure that if we had a more complex configuration and/or more traffic, we would have seen much less CPU.
Conclusions
Logstash is getting lighter and lighter, which seems to address its only criticism. You definitely get less memory usage and less garbage collection, so we definitely recommend upgrading to 1.2.2. And if you want to monitor its usage, you can always use our SPM. Happy stashing!
