By: Jim Russ, Enterprise Solution Architect at Nth Generation
I have been working in IT in one form or another for over 40 years. I have had many interesting experiences – from standing in a data center during an earthquake and getting out just before the entire data center was flattened, to helping a company recover from a major flood, to several other man-made or natural disasters.
The most important lesson that I have learned in my years in IT is that protecting the company data needs to be the number one priority. Data is the life blood of an organization. When you lose data or access to data, you can lose money, and in some cases, people may lose their lives.
With that knowledge, I have made it a matter of importance in my life to study the best ways to provide business continuance, High Availability (the ability of a system to operate continuously without failing for a designated period of time), and Disaster Recovery in a worst-case scenario. Because of my experiences in these areas, I have been asked to speak at several technology conferences over the years. Typically, the topic has been “Business Continuance and Disaster Recovery.”
I like to start these discussions by asking the audience, “How many of you have a Disaster Recovery Plan?” Early on, surprisingly, I would get very few hands raised. Then my follow up question would be, “How many of you backup your data?” Most of the audience would raise their hands. Then I would ask, “Why are you backing up your data if you do not have a plan?”
Why do you need a Disaster Recovery Plan? Plain and simple, with the current condition of the world today, it is a matter of when, not if, your business will experience some kind of natural disaster or cybersecurity breach. And if you do not have a plan to prepare for such threats, your business may be at risk.
So, what is an IT Disaster Recovery Plan? For business, an IT DR plan is an action plan that lists the steps to recover data assets in the event of any data outage scenario. If a worst-case scenario plays out and you are not able to operate your business data systems from the normal location or have access to your internal network, an effective DR plan will allow you to continue to do business from some other location.
That is really the focus of this article. Protecting data is extremely important. But what happens when you cannot access your data? Imagine working for days or weeks on an important proposal. The night before the proposal is due, you get a call that you need to be home. You save your work to the network share, log off your computer, and head home thinking that you will review and make final edits in the morning before submitting the document. You arrive early the next morning, turn on your computer, and get a message saying “Your credentials have expired – contact IT.” But you are IT! As you look through your systems, you start receiving messages from the rest of the company. “Where is my data? Systems won’t let me in!” Unfortunately, you were just hit with a virus. Your data is no longer there.
I won’t elaborate here regarding the detailed forensics for this scenario, but this example shows why a DR plan was initially created. If you were doing local High Availability clusters, all that data is most likely compromised, and even remote replication may not be reliable. There needs to be a separation of the data from the production version – a periodic point in time copy, snapshot, or air gap from the primary data.
Many times, recovery can be as simple as mounting a snapshot and drag and drop. But in more critical infections, all systems are inaccessible. That is when we need to refer to our backups for recovery.
Backing up the data is important if the data is secure and pertinent to run your business. The DR plan should include the prioritization of the mission critical applications for your business, as well as a list of those support applications that are required to get the business in operation. Most importantly, when doing backups, the 3 – 2 – 1 – 1 - 0 rule should be followed:
3 copies of the data
2 different types of media
1 copy offsite
1 copy “Air-Gapped” or “Immutable” or “Offline”
0 errors on backup recovery verification
Once again, backing up data is immensely important. My signature line states, “Blessed are those who backup daily, for they shall be restored.” But that is not all there is to it. I have worked with many customers over the years who had a disaster play out and they tried to restore their data and the backups were not reliable. Test your backups.
I have seen many times when the backup server was also compromised. That is where having an air-gapped or immutable solution in place is critical; you can rebuild the backup server and then begin the restoration process from your offsite media.
Depending on your plan and your available resources, recovery from a disaster is doable. Will it be easy? Not likely. But if you plan your work for DR, you will have/use your recovery plan if the time comes.
If you need more assistance building out a recovery/DR plan and learning how to protect yourself from ransomware or other attacks, contact Nth Generation here. We are happy to help.
Discover how Nth can safely emulate a ransomware attack in a replica of your server, and provide a custom report. Learn more about Nth’s Ransomware Readiness Assessment (RRA) SM here.
Comments