Service Metrics Project, A Step-by-Step Journey to Analytics Development

This project is aimed to collect metrics about backend systems running remotely and display them. The goal is to first be able to deploy the project using docker on any device. After this is successful, create a managed service on Amazon Web Services to be able to allow others to easily start collecting data about their service without having to run it themselves.

Project Description

The scope of this project is very small. The current target is to be able to ping services within a given time interval and save the data in a time series database. One feature is to be able to manage multiple IP addresses given to the system without interrupting the monitoring that is already running.

As data is being collected, the second goal is to have a Docker container with a React website running to serve the metrics and whether the given IP is operational, what percentage of time it has been online, and simple metrics.

Decisions to start off with

As the job market is currently terrible, I’m trying to use this project to build up a portfolio and also gain more skills. To satisfy this, I’m using Go as it has been very popularized recently. The language itself, having very fast compilation times, native support for concurrency, being statically typed, and being cloud compatible, is very attractive to me and for this project.

I used to work on all my projects with Nodejs or Python. Although I like both of them, Nodejs projects, when built, are massive due to all the packages being installed. This is a massive con in cloud services as it takes much longer to get them running. On the other hand, Python is much better at this. Loading up a service or a Lambda function is much faster. The tradeoff is not being able to use static typing (I used Typescript in Nodejs projects, and I know it does not help during runtime, but I still prefer having types). Golang seems to be in the middle ground of both.

The second decision I had to make was going for a monolith or microservice architecture. Even though there aren’t other features that I intend to build, if I later on, decide to expand the project and release it as a service running in the cloud, it will help to have independent services. This way, I can deploy all separately and manage them depending on user interactions by creating another service to spin up these services as docker containers or remove them. This way, I will also manage the number of services running depending on the user’s needs.

Conclusion

This is a very brief introduction to the project. I will be posting updates and the progress made on this blog. Later, I will release the code on GitHub for people to try it out. The service is already partially built in the refactoring process. Updates will arrive soon.