The Elastic stack (also referred to as ELK) can bring a lot of value to your production services. But it is not that much of value if you don’t use structured logs in your services.
In one of my latest posts, I wrote about what ELK is and why you should care. I also wrote a tutorial about how to integrate ELK with your Go app.
In this article, I will walk you through how to integrate structured logging in your Go services. We will use a sample HTTP service with a few basic endpoints and we’ll use the zap library to emit logs on error/success, which would also include some domain-specific info.
In my last post, I shared how much value the ELK stack could bring for your application in terms of the monitoring capabilities it gives you.
In this post, I will walk you through how to integrate your Go application with ELK, what are the different parts of ELK, how they work and how to create a basic configuration for them.
Let’s jump straight in, shall we?
When you start developing your application, you typically instrument it with some logging to be able to debug problems later.
Some skip it in the development phase, but once the application hits production, then it is crucial to have some logging.
After all, once users complain that something isn’t working, how would you be able to find the root-cause?
And although logging proves to be useful, many companies don’t really capitalise on its potential as they’re still clinging to the classic way of writing freestyle logs and grep-ing them on their prod machines afterwards.
However, there is so much more potential that logging holds for monitoring our production systems. In this article, I will show you how to get the maximum value from your logs using the ELK stack.
In many companies nowadays, microservices is the de facto way of handling service architecture.
Some do it out of necessity as their application has reached a scale where the monolith is a bottleneck. Others, simply like being onboard the hype train.
Whatever the scenario, the decision is often backed by the classical case for adopting microservices, which every junior dev studies extensively before their system design interview.
What gets often neglected, however, is the problems which come with such an approach.
Each of these problems usually demands a sophisticated solution, which raises system complexity.
One such problem is how to reuse the shared infrastructure components in your microservices. Each of your services will probably have a distinct business logic, but it will also come with a big baggage of infrastructure code.
These components usually don’t change too much between your services – healthchecks, monitoring configs, logging, standard service configurations, etc.
Fortunately, there is a very elegant solution for this problem for your Go services, which utilises the Fx Framework. It helps you by automatically managing your dependencies, but it can do much more than that as I’ll show you in the upcoming sections.
In this article, I will show you how to effectively extract your components into reusable & independent modules which can easily be shared across your Go services.