Microservices as a term has become increasingly popular in recent years, and is used to describe a variant of the SOA architectural style that creates an application as a collection of loosely coupled services. Every microservice has its own purpose, and will implement different interfaces or communication protocols. Therefore, any problems within a microservice will require a different, ad-hoc solution. In my previous teams, building microservices has provided us with a set of industry standard practices and techniques to help us ensure successful, seamless transitions when it comes to a big refactor or incremental changes. Trying to provide instructions on how to tackle every single change is challenging, so we have decided to turn our knowledge into a useful guide: our 10 commandments of changing a microservice.
Without further ado, here are our 10 commandments — follow them closely!
- You shall not take the name of tests in vain:
Before starting a big change, failing to do so without good test coverage (from a unit and/or integration perspective) is highly risky. Are you sure your changes won’t break anything if you can’t define with tests what ‘anything’ means? You should have your expectations defined and automatically controlled in advance. The importance of good testing beforehand cannot be understated!
2. You shall not tackle the whole change in a single step:
A famous saying applies here: Q. How do you eat an elephant? A. One bite at a time. Attempting to make a massive change at once is risky, messy and prone to error. Don’t let deadlines or pressure cloud your thinking. Patience and discipline are key here: plan your strategy as a set of micro changes instead. Every small commit will be an extra step towards success. One bite at a time.
3. Honour the main branch:
Make several changes, make them small and commit them into main. That way you will think about making your changes harmless and ready to go live as soon as possible. It’s better to go live with unfinished, safe code than to branch out with a time bomb. Also, committing in main will make your teammates aware of what you’re doing.
4. You shall not avoid live promotions:
Following the previous commandment will allow you to commit smaller, safer changes and put them live with minimal risk. It’s better to perform staggered releases, since this will add value quicker and minimise the risk that comes with a big reveal. Live promotions allow you to slowly make changes, and can give you tangible improvements as you go.
5. You shall use scaffolding:
Scaffolding is the process of adding temporary code to keep the application working while you are changing something else. You wouldn’t try and fix your roof without scaffolding. The same is for microservices — for example, in the case of a new API you’d create new routes to support the new API whilst serving requests to the old one, so you don’t break any clients. In other words, make your microservice backwards compatible with the ongoing change.
6. You shall use feature toggles:
If you can control the behaviour of your microservice by configuration, then you will add a lot of flexibility to your change process. With feature toggles, you could enable your changes for every environments, for example (so you could test your changes before going live). You may also configure your microservice to support switching between versions by http, so an external entity could change your code version based on certain criteria. Another possibility is to make your microservice a bit smarter and allow your feature toggle to be enabled/disabled based on health check metrics: this would allow you to roll back to the older version automatically if metrics reported lots of errors with the new one. Finally, with a more sophisticated feature toggle you could even specify different levels of granularity (for example, you could enable the new version only for certain users in a given environment, or from an individual country using a certain device). Nonetheless, feature toggles have a dark side: if you have too many, or if they are too fine grain it will be hard to test, as the code would behave differently according to a number of combinations. Try keeping a balance.
7. You shall increase your test coverage:
If you have followed the previous commandments, then you will have ended up with backwards compatible versions of the code — which means you will have an older version and a newer one. You should have started with tests for the old behaviour, but now you should also add test coverage for the new one. This is what will remain after you complete your changes, so you should ensure it will be safe.
8. Remember to clean up the no longer needed code:
Not cleaning unused or redundant code is one of the reasons to consider a refactor, so save some time now while the context is fresh in your mind. Kill the old code and all the scaffolding after you ensure using the new changes is safe. Don’t leave garbage in your services, it’s confusing and forces people to spend more time reading through the code. Getting into the habit of ensuring your code is clean will pay dividends in the long run.
9. You shall change your clients before doing a clean-up:
As obvious as it is, your service will be used by many different clients. If for example you’ve created a breaking change requiring the use of a new API or a new routing key, you will need to make all clients start using the new version before removing the old one, or they will start failing and report errors. This commandment can seem simple, but to avoid problems with clients, make sure that you do things in the right order.
10. You shall ensure nobody is using the old code.
Since microservices are agnostic as to who is using their API, there’s no obvious way to tell whether removing the old version of the code is totally safe or not. However, you could put some monitoring in place, use some logging or send metrics over a reasonable time period to watch the use levels of these old functions. That way, you can ensure you can kill the old code if no clients are making use of the old version. Microservices, and their changes, can be difficult obstacles to tackle. Creating an ad-hoc solution can be challenging and it’s important to follow a set of guidelines and principles whilst making changes to ensure the result is as desired, for both yourself and the client.