Last week I was at the Soft-Shake conference in Geneva to speak about Xanpan and Metrics Based Decisions. This year’s conference was a little bit smaller than the last one, but it could still offer 8 parallel tracks on both days. Being back was nice experience and made all a bit simpler.
In the last few weeks I posted a series of articles on log messages. Starting at the missed opportunities, continued with the great power of structured logging with Serilog and different sinks to the helpful dashboard of Kibana. With that entire infrastructure in place we are now able to turn log messages into knowledge. And even better, this doesn’t have to be one-time action.
Kibana is the powerful search interface of Elasticsearch. Depending on how many log messages you need to store you can run both tools locally or in the cloud. Today you will learn how you can install Kibana and what possibilities it has to monitor your applications.
When multiple systems continuously produce log messages they accumulate quickly. In combination with a longer retention time we easily talk about millions of log messages that need to be stored and searched. A single System like Seq will reach its limits and we need a bigger, more scalable solution.
When you play with different configurations for Serilog or try to add another sink you may run into problems. If you are out of luck your application crashes without the slightest notice - Serilog silently eats it's own errors. In such cases you have to dig a little deeper into Serilog to find out what’s going on.
Seq and Serilog can be easily combined and when we keep in mind some of the important aspects of writing log messages, we can fully leverage the benefits of both tools. Today I will show you some features I find particularly helpful when working with Seq.
Serilog is a great tool to write structured log messages. To fully profit from the structured part you need to write your log messages in a certain way. Otherwise you lose most of the great features and could use any other traditional logging framework. Luckily there are only a few simple things you should keep in mind and I will show you today what those are.
As I explained in the last posts, log messages are not just strings but have an inner meaning. When we use Serilog to write structured log messages we make them explicit to query them later. The tool we want to use to search for a specific orderId should give us this capability without the need to write all the code from scratch. We therefore need a tool that understands log messages and, if possible, even Serilog. One such tool is Seq that I explain in this blog post.
As I explained in the last post, creating JSON documents with Serilog is easy. But to make something useful with the log messages we must be able to query them. When I think about storing JSON documents then RavenDB is the first system that comes to my mind.
As a .Net developer you can choose from a variety of logging frameworks. All are more or less built to mitigate the shortcomings of the built in tracing framework. While System.Diagnostics.Trace has some helpful concepts, it misses many basic features most other logging frameworks can offer out of the box.