Skip to main content

Logging and Tracing

Purpose

The purpose of this document is to describe how microservices should log. As a note, this excludes infrastructure containers since they all have their own log formats. The goal of having a standard logging format for containers is to allow for easier debugging.

Log Levels

The following standard log levels will be used. A description of each level will also be provided

FATAL Very sever errors that will lead to the application crashing. 
ERROR Events that could be fatal but also allow the application to continue running.
WARN Events that could lead to potentially harmful situations.
INFO Informational messages that highlight the current state of the application
DEBUG Events that will be useful for a developer when debugging
TRACE Events that provides finer detail than DEBUG. This could be a dump of a data structure, progress of a task ect.

Standard Logging Format

The standard log format for all services will be a JSON object:

{
	"timestamp": "2022-09-22T10:33:20.123456",
	"level": "FATAL|ERROR|WARN|INFO|DEBUG|TRACE",
	"service": "my-microservice",
	"session_id": "7cddeb94-4d9b-4231-bb5a-2c9ce889926c",
	"client": {
		"name"user_agent": "client-name",User "version":agent "v1.0.0"
	}here",
	"method": {
		"name": "someMethod",
		"args": ["some", "args"]
	},
	"message": "some super cool message",
  	"stacktrace":"the stacktrace here if there is an error"
}

PII (Personally Identifiable Information)

Personally Identifiable Information shall be removed from all logs. Some PII includes:

  • Username
  • Passwords
  • API Tokens
  • API Keys
  • Names
  • Age
  • Addresses
  • Phones
  • SSN
  • Bank account numbers
  • Credit card numbers
  • Biometric data
  • Geographic data
  • Date of birth
  • ect

If PII should be put in a log message it should be changed to have only 5 * characters. For instance, the full name "Kyle Heestand" would become "*****". This is to not only protect against the data in the database, but also to protect against people guessing who/what the data is based on the size of it. As an exampleexample, take a standard login message:

{
	"timestamp": "2022-09-22T10:33:20.123456",
	"level": "INFO",
	"service": "LoginService",
	"session_id": "7cddeb94-4d9b-4231-bb5a-2c9ce889926c",
	"client": {
		"name"user_agent": "GenericUser Webclient",agent "version": "v1.0.0"
	}here",
	"method": {
		"name": "login",
		"args": ["kheestand", "some-password"]
	},
	"message": "Kyle Heestand has logged in"
}

After removing the PII, we will get something like this:

{
	"timestamp": "2022-09-22T10:33:20.123456",
	"level": "INFO",
	"service": "LoginService",
	"session_id": "7cddeb94-4d9b-4231-bb5a-2c9ce889926c",
	"client": {
		"name"user_agent": "GenericUser Webclient",agent "version": "v1.0.0"
	}here",
	"method": {
		"name": "login",
		"args": ["*****", "*****"]
	},
	"message": "***** has logged in"
}

Obviously, you would more than likely have a different message, but we can see that the user's username, password and full name has been removed to preserve their privacy. 

Back to top