Keeping Your Sub-Production Instances Healthy

Robert Webb - Senior ITSM Consultant

One of the problems you may be faced with when working with multiple ServiceNow administrators/developers across multiple instances is keeping your sub-production instances representative of your production instance. A good indication that you are doing a good job of this is when you rarely have any update set conflicts, missing related records, approval activities with no group, or some other issue relating to instance differences once you promote your changes to the production instance.

First let’s talk about a datacenter term called configuration drift.

On Configuration Drift

Configuration drift usually refers to primary/secondary systems in a data center and how changes over time cause their configuration to drift apart, which impacts backup/recovery operations. I think you can apply this same concept to ServiceNow development but with a slight twist. Instead of impacting backup or recovery operations, it will impact your ability to safely promote changes to your production instance.

Well then…what do I mean by a “healthy sub-production instance”?

A Healthy Sub-Production Instance

Simply put, a healthy sub-production instance is a non-production instance that closely resembles your production instance with as little configuration drift as possible.

When looking to evaluate how healthy a sub-production instance is, you should be asking questions like these (ideally you should be able to answer “yes” to each one of these):

Is the instance stable and performing as expected?
Have you recently cloned from the production instance?
Is user access and the roles they have been provisioned appropriate for this instance?
Are only plugins that are currently in use or soon to be used in production activated?
Are only current and relevant configuration or development changes present?
Is the foundational data representative of production (examples can include users, groups, locations, companies, schedules, configuration items, etc.)?
Is the transactional data representative of production (examples can include incidents, changes, cases, requests, etc.)?
Is the instance configuration representative of production (examples include flows, workflows, integrations, notifications, system properties, etc.)?

What You Can Do to Keep Them Healthy

Only make changes in your development instance

This one is easy in theory but can be hard for people in practice. Do not make changes in your production instance as you almost never have a good reason to do so. I would extend this guideline to your test instance as well if you have both a development instance and a test instance. If you make changes directly into production, you will get conflicts when promoting update sets for anything that you have also changed in development. In my experience this has been the number one cause of things working when tested in the development instance, working when tested in the test instance, and then not working correctly when you promote the changes to the production instance.

Clone production down to your sub-production instances often

Cloning your production instances over your sub-production instances often is a good way to ensure that the instances are very similar to each other and therefore the risk of something unexpected happening after promoting changes to the production instance is diminished. This one takes a bit of planning though as cloning can be disruptive to the development process if you do not plan for it. If you use a methodology like scrum you can clone after every sprint or every couple of sprints. Even if you do not use something like scrum with defined timeboxed development windows, you can still achieve similar results by timing your clones to coincide with when code promotions to promotion typically occur. Regardless of your development methodology, the goal is to get your MVP or finished product out of your sub-production instances and into your production instance so there is minimal risk of work loss when you clone. While this may seem aggressive, it will also yield the best possible results and synergizes with the next point very well.

Do not leave abandoned changes or code in the developer instance

In my experience, abandoned changes in a sub-production instance serve to cause two primary issues. The first is that they could complicate current development activities because they contribute to configuration drift in the sub-production instance. The second, is that they can unintentionally end up in production by being included with another change or by becoming an unintentional dependent of other work being done. If you have ever wondered “how did this get into production?” I can almost assure you that this was how. If you are using update sets and find that the changes you have been making are no longer required, you can backout that update set which will revert any tracked customization changes you may have made. However, if backing out the update set is not possible, then you should try to manually convert them back or notify the other developers to be mindful of these changes. This along with cloning is a good way to ensure that someone does not include your abandoned changes as part of their work.

Use a sandbox or personal developer instance to test out new applications, new plugins, or prototype/proof of concept new features

This one is simple, do not use your development instance or test instance as a sandbox. Your development instance is where work that will end up in the production instance goes. If you start playing in your development instance, those changes will inevitably break something someone is working on at best or miraculously end up in the production instance at worst. ServiceNow provides free personal developer instances to anyone who wants one. If you or someone on your team has one, use it to play around and test things. If not, you can quickly sign up and requisition your own personal developer instance in a matter of minutes. And for the lucky few, perhaps your organization has even purchased a special instance to use specifically for this purpose.

In closing, if you spend a little time up front making sure that your sub-production instances stay healthy you can save a little time down the road by minimizing issues you may have when promoting changes to your production instance.

Official Documentation and Links:

To find about GSI’s ServiceNow Services, call us at 855-GSI-4ERP or click on CONTACT US to send us a request for more information.

ServiceNow, ServiceNow Blog