One thing that you notice instantly when you ever try to figure out what goes on in a SharePoint instance is the unbelievable amount of log messages that are written. Even when no user is on your site, SharePoint itself has so many things going on that a constant stream of messages is created.
Logstash doesn’t have to be that complicated. Last week’s example with log files from IIS looked so scary because the fields can vary from one IIS to the other. But when you want to use logstash to parse a well-known file format then all can be much simpler. Today I will show you the configuration to parse log files from the Apache web server.
The IIS log files collect all the actions that occur on the web server. As explained last week you can use Log Parser to filter the events. However, when you use Kibana for all your applications then you would prefer to have the IIS log events there as well. Today I will explain how you can use logstash to read the log files from IIS and store the events in Elasticsearch.
Every request made to a web application running on IIS is logged. Not only will the page that was requested be registered, but a lot of additional information like the user agent, the client IP address and if the request resulted in an error.
Last week I was at the Soft-Shake conference in Geneva to speak about Xanpan and Metrics Based Decisions. This year’s conference was a little bit smaller than the last one, but it could still offer 8 parallel tracks on both days. Being back was nice experience and made all a bit simpler.
In the last few weeks I posted a series of articles on log messages. Starting at the missed opportunities, continued with the great power of structured logging with Serilog and different sinks to the helpful dashboard of Kibana. With that entire infrastructure in place we are now able to turn log messages into knowledge. And even better, this doesn’t have to be one-time action.
Kibana is the powerful search interface of Elasticsearch. Depending on how many log messages you need to store you can run both tools locally or in the cloud. Today you will learn how you can install Kibana and what possibilities it has to monitor your applications.
When multiple systems continuously produce log messages they accumulate quickly. In combination with a longer retention time we easily talk about millions of log messages that need to be stored and searched. A single System like Seq will reach its limits and we need a bigger, more scalable solution.
When you play with different configurations for Serilog or try to add another sink you may run into problems. If you are out of luck your application crashes without the slightest notice - Serilog silently eats it's own errors. In such cases you have to dig a little deeper into Serilog to find out what’s going on.
Seq and Serilog can be easily combined and when we keep in mind some of the important aspects of writing log messages, we can fully leverage the benefits of both tools. Today I will show you some features I find particularly helpful when working with Seq.