3 steps to extend your archival options to Microsoft Azure Blob storage

Source: Veeam

Long-term archival policies remain a consistent part of many enterprise infrastructures today. The need to maintain the 3-2-1 rule, ensure corporate standards, or provide regulatory compliance, keep liability of archival options out of the question. Convenient use of virtual tape libraries (VTL) enables enterprise businesses to extend their tape-based backups to virtual disks, or simply switch to newer operational methods without a long preparation as the staff is familiar with the terms already. While Veeam provides native tape support for backing up to virtual tape libraries, it’s now extending it with an option to leverage StarWind VTL for Microsoft Azure Blob Storage, enabling all users looking for cheap and reliable cloud storage to easily and securely store backups/files there.

With this integration, Veeam and StarWind customers can tier their backup data on site, maintaining one to two weeks of data on-premises, while moving longer-term archives directly to a more cost-effective Microsoft Azure Blob Storage. In this blog, I’ll be covering how to tackle the latter. For a more detailed look, you can view the webinar.

Before we rush into details, I’d like to mention that it will take very little effort from existing Veeam customers to accomplish the process. However, due to many available deployment scenarios (starting from Veeam Backup & Replication (VBR) and StarWind software installed on the same server on-premise and ending with VBR and StarWind being independently deployed on Azure VMs), please take a moment to think about traffic flow and the best configuration for your system before you proceed.

As for the configuration itself, I’d prefer to split it onto three logical steps:

1. Azure preparation

Go to Azure portal, find storage accounts, and add another one (or repurpose an existing blob storage). Make sure to provide all the required information and select blob storage within the account kind option. Then, proceed to a newly created storage account and copy the storage account name and a key (settings –> access keys) as well as create a container which is going to be used for storing the data (blob service –> containers –> new). You’ll need this data later, when configuring a cloud replication in StarWind VTL.


Figure 1. Azure blob storage details

2. Configuring StarWind VTL

The purpose of such an action is to emulate a tape library setup on a desired server, so Veeam is going to send data to that library where it will be processed and ready for a cloud archival. A classic Disk to Disk to Cloud (D2D2C) scheme in action.

Get the latest StarWind VTL package (version 8.0.0.12248 or newer) and install it on any appropriate physical, virtual, cloud server or even Veeam server itself. During installation, make sure to select the “VTL and Cloud Replication” option so StarWind automatically deploys the corresponding components. Specify a convenient path for the storage pool or leave it by default at disk C. Then, operating from the StarWind management console, connect to a desired server (use localhost or 127.0.0.1 when setting up an all-in-one scenario) and add a virtual tape device (drive) with as many virtual tapes you’d like. StarWind VTL emulates the actual HPE MSL8096 Tape Library, so all the principles of working with such a library will be applicable here. Note: You might need to install the latest drivers pack so that the server recognizes the said tape library properly. Once that is done, this server can be pointed to a VTL using the standard Windows iSCSI tools (control panel –> administrative tools –> iSCSI initiator). Go to Discovery –> Discovery Portal to initialize VTL and then connect to it from the Targets tab.


Figure 2. Connecting to discovered VTL target

Now you should enable a cloud archival via the cloud replication functionality: Simply select Microsoft Azure Cloud Storage on the first step, then specify the required Azure details from step #1, and finish the process by providing the desired retention settings.


Figure 3. StarWind. Setting up the retention settings

3. Veeam Backup & Replication

From a Veeam Backup & Replication perspective, you’ll need to add a server from above as a tape server to the VBR console. For that, an IP/DNS address and appropriate credentials will be required. During the procedure, Veeam will install a Transport and Tape Proxy services to the server and perform a tape libraries inventory if the option is specified. Once the tapes are detected and put into a Free Media Pool, it’s a good idea to create a dedicated Media Pool with some tapes, which will be used on another step.

Now, Veeam is connected to VTL and can push the data there. Create a Backup to Tape or File to Tape Job, specify the backup scope (you’ll need some pre-created backups for the first option), and point the Job to a previously created Media Pool. Depending on the backup/file size, you’ll get the data effectively delivered to the VTL server.


Figure 4. Backup to Tape Job in action


Figure 5. StarWind management console. Tape in Slot 1 has gotten a backup.

Now you can switch to the VTL server and remove the tape from the slot if it wasn’t automatically exported upon the Veeam Job completion. Since in my case the cloud replication was scheduled to start immediately, I can already see the motion in progress.


Figure 6. Uploading to Azure Blob Storage in action

After a successful upload, the Cloud tab will get a blue check and I should be able to verify that by navigating to my Azure Blob Storage and seeing the actual files uploaded to this container.


Figure 7. “ColdContainer” with uploaded data

I can go ahead and manually change the access tier for any of those files right from the Azure portal.


Figure 8. Azure Access Tier change

As an alternative, I could tweak the StarWind settings or even use PowerShell to manipulate the access tier in an automated way.

Restore VMs from VTLs in Azure

On the restore side, StarWind customers can initiate restores from Azure through their Veeam Backup & Replication console to recover the necessary files or VMs. Better yet, why not recover in Azure? Available in the Azure Marketplace, the virtual StarWind appliance, as well as Veeam Backup & Replication, can be installed in an Azure instance, and recoveries can be done from the archive storage directly into a new Azure virtual machine, accelerating your restore times and providing application portability across your backup infrastructure.

In addition, you can provide access to these newly restored VMs in Azure with Veeam PN (Powered Network), establishing a secure connection back to your HQ data center or wherever you need to provide access to these workloads.

Conclusion

Organizations are still using tape for a variety of reasons, but many want to take advantage of the cloud for their backups in order to maintain business continuity and unlock Availability. With Veeam Backup & Replication customers can leverage a seamless integration with StarWind to get their backups off site and into Microsoft Azure Blob Storage. To watch a full demo on the solution, you can watch this webinar.

The post 3 steps to extend your archival options to Microsoft Azure Blob storage appeared first on Veeam Software Official Blog.


3 steps to extend your archival options to Microsoft Azure Blob storage

57 recovery scenarios from Veeam Backup & Replication 9.5

Source: Veeam

It’s been a while since we published the last technical poster about Veeam technologies but please don’t be sad as I’m going to fix that right away!

Let me introduce you to a radical update to Veeam recovery scenarios technical poster. The last one was created almost two years ago and contained 47 restore scenarios of corresponding Veeam Backup & Replication v8. There have been two major product versions since that time and things have definitely changed. People constantly bugged me asking when we’ll have an update and I’m happy to report that the poster has been completely revised and now you can download its updated version, highlighting 57 recovery scenarios possible with Veeam Backup & Replication 9.5.

57 recovery scenarios from Veeam

From first glance you’ll find it looking very much like the previous one but as always the devil is in the details. Key differences with restore options are related to Oracle DB scenarios and revised Veeam Explorers block, focusing now on recovery ways instead of possible recoverable objects. In addition to the above, you’ll notice new information about SQL DB schema restore, Self-service portal and Direct Restore to Microsoft Azure.

Besides the top blocks, I had to create eight new featured bottom blocks to show new product technologies highlighting “new to 9.5” and what I previously didn’t have a place for. For example, this time I’ve dedicated a special place for U-Air restore, which actually appeared way before all Veeam Explorers and used to be the only way for restore of individual items of virtualized applications. Speaking of Veeam Explorers, the corresponding section is now focused on specific features new to 9.5 as the technology itself is pretty famous and it is much more important to specify what it is that we have within the latest release. PowerShell cmdlets, divided by hypervisor, should be very helpful when you need to do something quickly and there is no time to look up the documentation. Direct restore to Microsoft Azure of a Windows based machine should provide some insight on how complicated the operation is underneath. There will be much more to come!

“How to use” instruction doesn’t differ this time: Print it and hang on your office wall or use it when preparing slides for a technical presentation. For your convenience, the poster is available in A0, A1 and A2 formats.

Stay tuned and don’t forget to let me know if you have an idea what technical poster you’d like to get next.

The post 57 recovery scenarios from Veeam Backup & Replication 9.5 appeared first on Veeam Software Official Blog.

57 recovery scenarios from Veeam Backup & Replication 9.5

How to recover Group Policy Objects

Source: Veeam

Read the full series:

Ch.1 — Backing up Domain Controller
Ch.2 — How to recover a Domain Controller
Ch.3 — Reanimating Active Directory tombstone objects
Ch.4 — Leveraging Active Directory Recycle Bin

 

Continuing my previous posts about Active Directory (AD) management and restore with Veeam products, I would like to talk a bit more about relatively recent enhancements we brought into this process. Today, I’ll focus this article on the newest recovery features of Veeam Explorer for Microsoft Active Directory, released with Veeam Backup & Replication versions v9 and 9.5. Warning: Each new version of Veeam Backup & Replication comes with updated Veeam Explorer for Microsoft Active Directory, so it’s important to be aware of software versioning in order to know the scope of possible operations. Additionally, it’s generally a good idea to keep your Veeam infrastructure (and operating systems of VMs) at supported and recent versions.

As I mentioned before, we introduced Veeam Explorer for Microsoft Active Directory — a very helpful utility when it comes to AD objects recovery — as a part of Veeam Backup & Replication v8. Its initial functionality was intended to solve the most frequent cases administrators have with Active Directory: Granular objects and containers recovery (ok, password recovery also was included, as well as AD data export in LDIFDE format). All of that made a lot of people happy, but, as always, they wanted more. The community gave us great feedback, asking for additional features for less frequent cases or specific scenarios. We found out that, besides the most frequent operations like adding and removing users/computers to the domain, sometimes they had to deal with more advanced restore operations related to Group Policy Objects (GPO), DNS-integrated records and so on. That said, we worked hard and added some new functionality to provide administrators with such options.

Starting from Veeam Backup & Replication v9, you can restore Group Policy Objects, and the process is very easy.

Note: Group Policy is a Windows Server feature (since Windows Server 2000) that allows an administrator to centrally manage the working environment of users and computers, allowing common policies to be configured from one place and then distributed at ease, while also controlling what users/computers can or cannot do.

In order to restore GPO, you have to make sure you are running the appropriate Veeam Backup & Replication version and that you have already taken a valid backup file of your Domain Controller (DC). The actual recovery procedure is very similar to one I described before:

  • Administrator starts application-item restore for Microsoft Active Directory from the main ribbon or via the backups hive
  • Then, the administrator selects an appropriate backup point with a known valid state
  • Veeam Backup & Replication mounts that restore point to the backup server, extracting the Active Directory database and SYSVOL catalog, and automatically opens them in Veeam Explorer for Microsoft Active Directory
  • If all prerequisites are met, the administrator should be able to find the Group Policy Objects container right below the Users and Computers container
  • Then, the administrator finds a desired GPO manually or by using the search, and performs either the restore or export procedure (figure 1)
Figure 1. Veeam Explorer for Microsoft Active Directory GPO options
Figure 1. Veeam Explorer for Microsoft Active Directory: GPO options

 
Hint: As an option, the administrator can compare GPO attributes with the production state and see what exactly was changed (figure 2).

Figure 2. Veeam Explorer for Microsoft Active Directory Comparing backed up GPO with production
Figure 2. Veeam Explorer for Microsoft Active Directory: Comparing backed up GPO with production

 
Additional improvements to Veeam Explorer for Microsoft Active Directory

Besides that, in the same version 9, Veeam Explorer for Microsoft Active Directory added support for the recovery of:

  • Active Directory-integrated DNS records (DNS integrated into Active Directory and replicated as a part of Domain Services replication)
  • Objects in Active Directory configuration partition (Native AD partitions containing forest-wide information about existing domains and sites available services, which come per forest and are replicated to all Domain Controllers)

This is a huge step forward for experienced administrators who know what they’re doing. There is just one small trick you need to know to find this functionality: Within the restore operation, hit advanced features button in the main ribbon to be able to see integrated DNS and configuration partition containers, which are normally hidden by default (figure 3).

Figure 3. Veeam Explorer for Microsoft Active Directory Advanced features
Figure 3. Veeam Explorer for Microsoft Active Directory: Advanced features

 
With version 9.5, Veeam Explorer for Microsoft Active Directory got something new as well. Since the general release was aligned to the release of Windows Server 2016, we spent a great deal of time making sure you have support for all of the new Active Directory version forests that run in the Windows Server 2016 functional level, as well as other enhancements. Now, using Veeam Backup & Replication 9.5, you can restore the following AD items (in addition to those previously mentioned):

  • Objects from forests running in the 2016 functional level and using Windows Server 2016 Directory Services for Active Directory (including user and computer account password restore)
  • Expiring links (export to LDF file, not available with LDIFDE utility, is included)

That is, obviously, great for new installations that are running all DC on Windows Server 2016 or Azure mixed domains. And the coolest part is that all of the above is working right out of the box, and you don’t even need to do anything extraordinary.

In conclusion, I can assure you that we were listening to your feedback while developing Veeam Explorer for Microsoft Active Directory, as well as our other products. Write comments below or, even better, vote up the most wanted new feature you’re currently missing on Veeam forums, so we’re able to adjust our program development and provide you with new functionality in future software releases.

Have a great time managing your Active Directory better with Veeam!

 

Resources

Veeam Backup Explorers User Guide

Best practices for Active Directory with Veeam

 

 

 

The post How to recover Group Policy Objects appeared first on Veeam Software Official Blog.

How to recover Group Policy Objects

Leveraging Active Directory Recycle Bin: Best practices for AD protection (Part 4)

Source: Veeam

Read the full series:

Ch.1 — Backing up Domain Controller
Ch.2 — How to recover a Domain Controller
Ch.3 — Reanimating Active Directory tombstone objects
Ch.4 — Leveraging Active Directory Recycle Bin

 

This post is part four of a series where I discuss granular recovery of Active Directory objects and different scenarios and tools for such operations.

In the previous article, I described the cases where administrators worked with Domain Controllers running Active Directory off a functional level of Windows Server 2003 and Windows Server 2008. I detailed the steps they had to do in order to reanimate the tombstone objects using LDP and Veeam Explorer for Microsoft Active Directory utilities.

Today, I’m moving on to newer systems with the Active Directory recycle bin feature enabled.

With Windows Server 2008 R2, Microsoft implemented a long-awaited Active Directory recycle bin. This extended the standard life cycle of an Active Directory object and changed the logic of object deletion. With this feature enabled, the object started going to the deleted objects container right after deletion, where it stays for the lifetime of the deleted object (equal to recycled object lifetime by default). Most important, the system is able to preserve all of the object’s link-valued and non-link-valued attributes for the same lifetime period. This means you can easily restore an object with those attributes during this period.

Once the lifetime is over, the system changes the object status to recycled and drops most of its attributes. Additionally, the object becomes logically equal to what used to be tombstone in Windows Server 2003 and Windows Server 2008. The only difference is that you can’t restore or reanimate the recycled object now. A garbage collector removes it automatically after a recycled object lifetime expires (180 days by default).

Active Directory object life cycle with Active Directory recycle bin enabled

Figure 1. Active Directory object life cycle with Active Directory recycle bin enabled

Enabling Active Directory recycle bin

So far, the Active Directory recycle bin is not enabled by default on any Windows Server OS. To utilize this tool, you should prepare your environment, make sure that every DC in your forest is running Windows Server 2008 R2 and newer, and set your forest functional level to Windows 2008 R2 or above.

NOTE: Enabling the Active Directory recycle bin as well as any other substantial change to Active Directory (or other production systems) may be an intimidating task. You can use Veeam Virtual Lab technology to test schema upgrades or any other potentially impactful change before you do so in production. Additionally, the virtual lab can hold other VMs you have which are critical, so that they can go through the change as well and test multi-tiered applications for compatibility after the changes. The virtual lab can run from backups, replicas or even storage snapshots (depending on configuration). This way, there are no surprises in production.

Before using the Active Directory recycle bin, keep in mind that:

  1. Enabling the Active Directory recycle bin changes all current tombstone objects into recycled objects, so you won’t be able to restore them once enabling is done.
  2. The process of restoring multiple dependent objects can be difficult, because it requires a strict order of restore, starting from the higher-placed objects.
  3. In Windows Server 2008 R2, every operation related to the Active Directory recycled bin should be done via PowerShell cmdlets, no GUI provided. Windows Server 2012 and above introduce Active Directory Administration Center (ADAC), where all recycle bin operations can be performed via GUI.
  4. The recycle bin doesn’t have anything in common with Active Directory backup, and it won’t help to restore a whole DC if it is damaged.

Enabling Active Directory recycle bin in Windows Server 2012 via ADAC

Figure 2. Enabling Active Directory recycle bin in Windows Server 2012 via ADAC

The Pros & Cons of Active Directory recycle bin

When you enable the Active Directory recycle bin, you will notice a new Deleted Objects container visible via the Active Directory Administration Center. By browsing this container, you can see all deleted but not recycled objects, check their properties and restore them to a default or custom place.

Navigating through the Deleted Objects container in the Active Directory recycle bin

Figure 3. Navigating through the Deleted Objects container in the Active Directory recycle bin

Even though it looks much easier to perform granular recovery operations using this functionality than using LDP utility or performing authoritative restore of a domain controller, it also has some considerations. Below, I outlined the pros and cons of using Active Directory with the recycle bin feature enabled.

Pros:

  • A universal method for Windows Server 2008 R2 functionality level (or newer) domains
  • Great lifetime period (180 days by default is a sufficient time for majority cases)
  • Object attributes are preserved for a lifetime period
  • Recovery doesn’t require a reboot of a DC
  • GUI for Windows Server 2012 and newer

Cons:

  • Doesn’t work for Windows Server 2008 functionality level (or older) domains
  • Doesn’t work for changed objects (object can be restored if deleted, not changed)
  • Recovery is limited by a lifetime period value
  • Doesn’t protect against issues with DC itself (never be good as a backup copy)
  • No automation for hierarchy recovery

The second drawback is the most disturbing. What if the object wasn’t deleted, but occasionally changed, and the mistake is noticed after a while? Unfortunately, recycle bin won’t help here, and you have an issue to solve.

Solving recycle bin limitations with Veeam

Considering cons of recycle bins, they might not be a deal-breaker for some of you. However, those who want an ultimate solution should look somewhere else. Here comes Veeam, offering the same Veeam Explorer for Active Directory, which was previously discussed. This tool simply eliminates limitations of Active Directory recycle bin. With this utility, your Active Directory objects are protected as long as you keep backups around. It works for domain controllers with forest functional level of Windows Server 2003 and newer. Most importantly, it is a part of Veeam Backup & Replication, and is included in the Free Edition.

With a combination of Veeam Backup & Replication and Veeam Explorer for Active Directory, you can restore the entire DC at once and recover individual Active Directory objects: OUs, computer and users accounts with their passwords, GPOs, DNS records and more. Besides that, it’s easy to launch the Explorer and compare objects in a backup copy with live objects in production to notice the difference and changed objects attributes.

The example below shows a situation in which an administrator noticed a change in attributes of a user account and was asked to recover the user account to a previous condition.

Recovering changed Active Directory objects

Figure 4: Recovering changed Active Directory objects

Either way, thinking about possible Active Directory disasters in advance and testing different tools to prevent and fix these disasters will help you sleep soundly at night.

Hope this series triggered your mind and made you recheck your Active Directory protection strategy. Feel free to reach out to me in the comments to discuss more.

Helpful resources

The post Leveraging Active Directory Recycle Bin: Best practices for AD protection (Part 4) appeared first on Veeam Software Official Blog.

Leveraging Active Directory Recycle Bin: Best practices for AD protection (Part 4)

Reanimating Active Directory tombstone objects: Best practices for AD protection (Part 3)

Source: Veeam

This is the 3rd article from my Active Directory protection series. In the previous one, I covered the domain controller recovery process. However, I believe that is a less common operation when compared to other Active Directory requests system administrators constantly receive. Among those requests, I believe the most frequent is Active Directory objects change.

That’s why today, I’d like to talk about Active Directory objects recovery and Active Directory tombstone objects reanimation when using old systems with the forest functional level of Windows Server 2008 and older. Fortunately, it’s a rare case now, but I wouldn’t be surprised to find examples of these systems still being around. Newer systems and features like Active Directory recycle bin are going to be covered in the next article in the series.

Ok, so why is it important to remember old cases? Because modern logic and features don’t work there. Prior to Windows Server 2008 R2, an Active Directory object lifecycle looked like this:

AD object lifecycle
Figure 1. AD object life cycle

Once the Active Directory object is deleted, it is not hard deleted from a system. As you may know, Active Directory makes the object hidden by changing its attribute isDeleted to TRUE value. Then, it drops most of the objects’ attributes, renames the object, and moves it to a special container (CN=Deleted Objects). From now on, the object has a tombstone status, and standard Active Directory utilities don’t see its presence. Then, the object is conserved within this special state for a lifetime period (60 days for Windows Server 2000/2003 and 180 days for Windows 2003 SP1/2008). This is to ensure that the information about removal was successfully replicated across the system. Once the tombstone lifetime period is over, a special process called garbage collector physically removes the object from the database.

Here comes the question. If the tombstone object was not physically deleted within a certain amount of time, would it be possible to recover (reanimate) it? The short answer is yes. Even though the tombstone mechanism was never intended to be a temporary recycle bin, and objects weren’t ever supposed to be reanimated, the possibility exists and I’m going to show how you can do that.

LDP (LDP.exe) is an old and robust program that works with Active Directory, designed by Active Directory developers. It might look a bit ordinary, but it is very powerful and can give you full control under Active Directory objects. As a downside, you should invest quite some time into learning the program functionality, and it’s not quite obvious and modern.

To recover a deleted (tombstone) object using LDP, you should:

  • Run this program (Start – Run – ldp)
  • Connect it to a domain controller (Connection – Connect..)
  • Use the appropriate (domain or enterprise administrator) credentials to authenticate. (Connection – Bind..)
  • Search for a sought-for object (Browse – Search) within the Deleted Objects container. You will have to apply searching options and filters the smart way (check the figure 2 below). When navigating via Controls dialog, make sure you select the “return deleted objects” option, then press check in to add the object identifier for this option to the Active Control list. Then, save your settings and run the query to find tombstone
  • Reanimate the tombstone object by using (Browse – Modify) a wizard to find the object via its distinguishedName (DN) parameter and remove its isDeleted value with object renaming, so the object is recovered back and you can see it from Active Directory Users and Computers snap-in.

The picture below shows a typical search I performed to find tombstone objects in my test domain:

Searching with LDP
Figure 2. Searching tombstone objects with LDP utility.

As this article isn’t intended to be an ultimate guide for LDP utility, here’s an LDP guide, which will help you master your skills.

In addition to above, keep in mind that even when you reanimate tombstone objects this way, some object attributes (group membership for example) are dropped within an initial deletion, so they aren’t going to be recovered with reanimation and it might bring you a headache.

As an alternative to the approach above, it’s possible to utilize Veeam solutions and Veeam Explorer for Active Directory in particular. This utility allows you to perform exactly the same reanimation, just doing it faster and much easier.

Yes, in order to use Veeam Explorer for Active Directory you should have a backup copy of your DC where deletion happened. Yes, your domain controller should be virtualized, thus eligible to be backed up by Veeam Backup & Replication in the first place. That’s why it wouldn’t suit every scenario, since some preliminary steps are required. However, if you’re a lucky administrator of a virtual domain controller with a forest functional level of Windows Server 2003 or Windows Server 2008, read carefully. The information below will be helpful for you.

  1. Make sure you have a backup of your Domain Controller, processed with the application-aware processing option enabled (I explained the importance of it in the first article)
    VBR - Editing Backup Job
    Figure 3. Veeam Backup & Replication, editing Backup Job
  2. When the deletion happened and you would like to reanimate the object, navigate to the DC backup and select “Microsoft Active Directory objects…” to start the actual recovery and launch Veeam Explorer for Active Directory.
    Launching Veeam Explorer for Active Directory
    Figure 4. Launching Veeam Explorer for Active Directory 
  3. Browse to the required container and enable “compare all objects” and “show changed objects only” to make a pre-selection and force Veeam Explorer to compare the backup data with the real DC condition and display only changed items. See object status and don’t miss the “tombstone” one.
    AD object lifecycle
    Figure 5. Veeam Explorer for Active Directory, comparing objects 
  4. Restore the required object(s) back to the production or export them as an .lde file.
    AD object lifecycle
    Figure 6. Veeam Explorer for Active Directory, granular restore options 

Considering the steps above, you can see that Veeam Explorer for Microsoft Active Directory provides a relatively simple method of recovering tombstone Active Directory objects. If you by any chance run such an environment, don’t miss this product.

In addition to that, don’t forget that Veeam Explorer for Microsoft Active Directory is a very tiny part of overall Veeam Backup & Replication functionality. When selecting this product, you gain much more than Active Directory object recovery.

In the next article I’ll compare Active Directory recycle bin with other methods of restoring AD objects.

Reanimating Active Directory tombstone objects: Best practices for AD protection (Part 3)

How to recover a Domain Controller: Best practices for AD protection (part 2)

Source: Veeam

This is the second article from my series on Active Directory (AD) protection with Veeam. In the previous post, I reviewed physical and virtual Domain Controller (DC) backup procedures. Today, I will discuss recovery procedures.

Disclaimer: This post is not intended to be a comprehensive AD Domain Services recovery guide. Instead, it will give you important information to consideration when recovering AD or a particular DC, as well as explain how Veeam can accomplish this process. If you’re interested in granular recovery of a deleted AD object, navigate to the next article.

Knowing your infrastructure 100% is a great help for AD recovery planning. Do you have a single-DC or a multi-DC environment? Read/Write Domain Controller (RWDC) or a Read-Only Domain Controller (RODC)? Have you lost just one DC or has an entire AD infrastructure been damaged or corrupted? If you have multiple DCs, do you still use File Replication Service (FRS) or have you migrated to a Distributed File System Replication (DFSR) service for syncing changes between the multiple DCs? Those are a few questions you should be able to answers if you want performing a successful recovery.

Note: FRS is a service for distributing shared files and Group Policy Objects (GPO) in Windows Server 2000 and Windows Server 2003. It was replaced by the DFSR in later Windows Server OS (operating system) versions. Since Windows Server 2008, DFSR has been a default option for SYSVOL replication. If the first domain controller of the domain was promoted to Windows Server 2008 functional level or higher, then you’re using DFSR. Refer to this article to determine whether FRS or DFSR is used in your domain. Here are the benefits of using DFSR over FRS.

Whenever you’re about to restore a DC, first determine whether a non-authoritative restore is enough, or if should you go further and perform an authoritative restore. The difference between those two restore types is that within a non-authoritative restore, the DC understands that it was out for a while, so it lets other in site DCs update its own database with the latest changes that occurred when it was down. With an authoritative restore, the DC claims itself as the only one with correct information and a valid database, and it authoritatively updates other DCs with its own data.

In most scenarios, a non-authoritative restore is what you need because it’s usually a multi-DC environment. In addition, restoring a DC in authoritative mode can be harmful and cause further damage. Due to this, the logic of Veeam Backup & Replication was developed accordingly, and by default, it performs automated, non-authoritative DC restore, assuming that it was not the only DC in place. For an authoritative restore with Veeam, see below for some additional steps, which are required.

NOTE: Another important practice is to leave a failed DC out of scope and seize its roles, as well as perform metadata cleanup if it is not likely to be coming back. This way, you allow other DC(s) to take over and you don’t need to fix a broken DC.

Let’s go back to the backup files I created when I wrote the previous article. Restoring a DC from Veeam Backup & Replication backup is quite easy. You simply:

  • Select a Restore wizard in GUI
  • Find a desired DC
  • Choose the Restore Entire VM option from the recovery menu
  • Then, select the recovery point
  • Choose if the restore should happen to the original location or a new one
  • Complete the procedure

The cool thing here is that, due to the application-aware image processing we used within a VM backup, you don’t have to do anything else at the moment. Veeam recognizes the DC role of this VM and gently restores it using special logic:

  • Recover VM files and hard disks
  • Boot it into a Directory Services Restore Mode (DSRM) mode
  • Apply the settings
  • Reboot it into a normal mode

The DC will be aware of the restored from the backup state and start acting accordingly, invalidating the existing database and allowing replication partners to update it with the most recent information.

Domain Controller (DC) restore from backup
Figure 1. Veeam Backup & Replication: Entire VM recovery

Here you can read about Bare-metal restore of a backup using Veeam Endpoint Backup. You will need a Veeam recovery media prepared beforehand and the access to the backup file itself (USB disk or a network share). Keep in mind that the special logic of Veeam Backup & Replication will not be applied here. After a restore with Veeam Endpoint Backup, your DC will boot into a recovery mode and you will need to decide whether you’d like to reconfigure registry keys or reboot into a normal mode right away. This KB article will be helpful here.

Boot Domain Controller (DC) into a recovery mode
Figure 2. Veeam Endpoint Backup: bare-metal recovery

As a reminder, you most likely you don’t need this type of restore. But let’s dig inside anyway so you understand the reasoning.

This operation might be done when you’re trying to restore a valid copy of DC in a multi-DC environment, while the entire AD is corrupted at some point (ex. ransomware, virus, etc.). You would, therefore, want to force DCs to accept changes from a restored DC.

NOTE: Follow procedures that are similar to what Veeam SureBackup job performs when restoring a DC in an isolated environment.

To restore a specific deleted object or a subtree (ex. Organization Unit) in authoritative mode and force this DC to replicate it to other DCs:

  1. Select full VM recovery with Veeam and let the program performing a standard, non-authoritative DC restore automatically (described above).
  2. When the DC reboots the second time, open the booting wizard (press F8), select Directory Services Restore Mode (DSRM) mode and then sign in to a system using DSRM credentials (the credientials you provided when you promoted this computer to a DC).
  3. Open a command line and run ntdsutil
  4. Use the following commands: activate instance ntds; then authoritative restore; then restore object “distinguishedName” or restore subtree “distinguishedName”
    Example: restore subtree “OU=Branch,DC=dc,DC=lab, DC=local.
  5. Confirm the authoritative restore and reboot server upon completion.

The procedure of authoritative SYSVOL restore (DFSR service used) goes this way:

  1. Non-authoritative restore of a DC (Example: entire VM restore in Veeam Backup & Replication).
  2. When booted the second time, navigate to HKLMSystemCurrentControlSetServicesDFSR registry hive, create a key Restore and create SYSVOL string with the value authoritative.
    This value is read by the DFSR service. If this value is not set, the SYSVOL restore is performed non-authoritatively by default.
  3. Navigate to HKLMSystemCurrentControlSetControlBackupRestore, create a key SystemStateRestore and create a LastRestoreId string with any GUID value. (Example: 10000000-0000-0000-0000-000000000000).
  4. Restart DFSR service.
Authoritative SYSVOL restore
Figure 3. Authoritative SYSVOL restore (DFSR service)

Procedure of authoritative SYSVOL restore (old FRS service used):

  1. Non-authoritative restore of a DC (Example: entire VM restore in Veeam Backup & Replication).
  2. When booted the second time, navigate to HKLMSystemCurrentControlSetServicesNtFrsParametersBackup/RestoreProcess at Startup registry hive and change the value of the key Burflag to 000000D4 (hex) or 212 (dec).
    This effectively forces the Domain Controllers still using the old FRS technology to start the replication in an authoritative mode. More details about FRS recovery.
  3. Restart the NTFRS service.

While I able to go through a specific DC recovery in this article, the most frequent use cases with AD require you to recover the accidently deleted AD object, this wouldn’t be the best way to restore the whole DC for this purpose. In my next series, I’m moving to granular AD object recovery using native Microsoft tools and Veeam Explorer for Active Directory.

Best practices for authoritative restore (TechNet)
Recovering your Active Directory forest
Registry keys and values for backup and restore
Restoring the SYSVOL (non-)authoritatively when using NTFRS or DFSR

How to recover a Domain Controller: Best practices for AD protection (part 2)

Backing up Domain Controller — Best practices for AD protection

Source: Veeam

Microsoft Active Directory is a standard in corporate environments where authentication and central user-management are required. It’s almost impossible to imagine how system administrators would be able to do their jobs effectively if this technology didn’t exist. Not only is Active Directory a great power, but it’s also a great responsibility — and it requires spending a lot of time with it in order to maximize its capabilities.

The purpose of this series is intended to aid you with the successful backup and recovery of Active Directory Domain Services with Veeam, giving you all the keys to painless AD protection. Before reading this, you might want to take a look at the Active Directory design and implementation series we posted a while ago.

The actual series is going to discuss how Veeam can protect AD data — preserve Domain Controllers (DCs) or individual AD objects and recover either of them when required.

Today, I’m going to talk about the backup options Veeam offers for both physical and virtualized DCs, and backup considerations to keep in mind while you do that.

Backup DC considerations

As Active Directory Domain Services designed with a sort of redundancy, so the common backup rules and tactics can be mitigated and adapted to this level. It wouldn’t be right to apply the same backup policy you have for SQL or Exchange server here. Below are some considerations I believe might be helpful for creating your own AD policies:

  • Learn what domain controllers hold Flexible Single Master Operations (FSMO) roles in your environment. Hint: a simple command to check this via command line: >netdom query fsmo

When performing a full domain recovery, you might want to start from the DC with most FSMO roles, usually one with PDC emulator role. Otherwise, you will have to transfer roles manually after the restore with ntdsutil seize command. Be aware of that, when planning backup and prioritize DCs accordingly. Refer to Active Directory basics white paper to learn more about FSMO roles.

  • If you have multiple DCs for the site and you’re looking for individual objects protection, there’s no need to backup all DCs, as for item-level recovery, one copy of AD database (ntds.dit) would be sufficient
  • There are things that can always mitigate the risk of accidental/intentional deletion/change of AD objects. Consider administration operations’ delegation, setting up the restricted access to elevated groups and maintaining a “lag” site
  • It’s usually recommended to perform backup of one DC per time, not to interfere with DFS Replication — even if the modern backup applications (ex. Veeam Backup & Replication v7 with patch 3 and onwards) know how to deal with this
  • If you have a VMware virtual environment and it is not possible to connect to your DC over the network, as for example, it can be in DMZ. In this case Veeam will fail over to the VIX and should be able to process your DC.

Backup of a virtual Domain Controller

Microsoft’s Active Directory Services organize and keep information about individual objects within the forest and store it to a relational database (ntds.dit), hosted by a domain controller. Backup of a Domain Controller has previously been a tiresome process, involving backing up the server’s system state. It’s a well-known fact, that Active Directory services don’t consume a lot of resources of the system, so Domain Controllers are appearing to be the first servers that are always virtualized in the environment. If you happen to share the old belief of “physical DCs only”, please refer to this post.

Once virtualized, they are pretty easy to be managed by a domain/system administrator and can be easily backed up with Veeam Backup & Replication. As for details, you should have Veeam Backup & Replication installed and configured. The system requirements (of version 9.0) are as following:

Virtual platform: VMware vSphere 4.1 and newer; Microsoft Hyper-V 2008 R2 SP1 and newer

Veeam server: Windows Server 2008 SP2 and newer; Windows 7 SP1 and newer, 64-bit OS

Domain controller virtual machine (VM): Windows Server 2003 SP1 and newer, the minimum supported forest functional level of Windows 2003

Permissions: Administrative rights for target Active Directory. Account of an enterprise administrator or domain administrator.

This article doesn’t intend to cover a process of Veeam Backup & Replication installation and configuration, as it’s already been defined a few times. But, if you need help with that, please refer to the following video recorded by a Veeam system engineer.

I’m going to assume that you have everything running fine. Now you’d like to configure a backup task for your virtual DC. The process of configuration is rather simple (see figure 1 below):

1. Launch a Backup Job creation wizard

2. Add a desired DC to the task

3. Specify the retention policy for the backup chain

4. Make sure you enable application-aware image processing (AAIP) to ensure transactional consistency of backup files, including the Active Directory database, its supportive files and SYSVOL catalog

Note: AAIP is a Veeam technology that allows software to backup VMs in an application-aware way. That means a multi-step process of detecting applications of a guest OS system, quiescing them using corresponding VSS writers, applying specific application settings and truncating transaction logs if the backup task is successful. Please refer to the AAIP documentation for details.

Not enabling AAIP will not trigger Domain Controller guest OS to realize it was backed up and protected. So, a while later, you might notice an internal warning in server logs — event 2089, stating that there was no backup for “backup latency interval” days.

Edit Backup Job: Guest processing
Figure 1. Edit Backup Job: Guest processing 

 

5. Schedule a task or manually run it

6. Ensure the task successfully ran with no errors or warnings

Performing incremental backup of a DC
Figure 2. Performing incremental backup of a DC 

 

7. Find the newly created backup file at the backup repository — that’s it!

Additionally, you can store a backup in the cloud with Veeam Cloud Connect (VCC), copy it to another datastore or tape using Veeam Backup Copy jobs and much more. The most important thing is that backup is now safe and can be restored as soon as you need it.

How to back up a physical Domain Controller

Frankly speaking, I hope that you’ve been reconfiguring AD services in your company and that your DCs have been virtualized for a long time. If not, I hope that you’ve at least been updating your DCs, and that they’re running relatively modern Windows Server OS versions, Windows Server 2008 R2 or newer. (If managing older systems, skip below and go to the third article right away)

So, you have a physical DC — or a set of them — running at Windows Server 2008 R2 or newer, and you want to protect your AD? Meet Veeam Endpoint Backup, the utility aimed to ensure that data on your remaining physical endpoints and servers is safe and secure. Veeam Endpoint Backup catches the desired data of the physical machine and stores it in a backup file. Then, in case of a disaster, you are able to do a bare-metal or volume-level restore — while having full control of recovery procedures. Plus, item-level recovery with Veeam Explorer for Microsoft Active Directory.

In order to back up your physical DC with this tool you should:

  • Download the utility from this page and put it to on your DC
  • Launch the installation wizard, accept the license agreement and install the program

Note: read these instructions for installing in Unattended Mode.

  • Configure a backup task by selecting appropriate backup mode. If you’re configuring file-level backup mode, select Operating system as an object to backup (see Figure 1). This ensures that the program captures all files required for bare-metal restore, Active Directory database and SYSVOL catalog will be also saved. Feel free to refer to a product user guide for details
Selecting objects to backup in Veeam Endpoint Backup
Figure 3. Selecting objects to backup in Veeam Endpoint Backup 

 

Note: If you have Veeam Backup & Replication instance in your infrastructure and you’d like to use a configured Veeam Backup Repository to accept endpoint backups, please reconfigure it right from Veeam Backup & Replication (right click on a desired repository, allow access to the repository and enable backups encryption if needed, see Figure 4).

VBR: Endpoint Backup permissions
Figure 4. Setting Endpoint Backup Permission for backup repository
  • Run the backup, and make sure it’s done with no errors
Veeam Endpoint Backup FREE: Backup job statistics
Figure 5. Veeam Endpoint Backup FREE: Backup job statistics
  • Voila! The backup is done, and your DC is protected from now on. Go to the backup destination and find the backup or the backup chain
Incremental backup chain
Figure 6. Incremental backup chain

 

Note. If you configured a Veeam Backup & Replication repository as a target for DC backup, feel free to find the newly created backup at the backups-disk, placed to Endpoint Backups node.

Veeam Backup & Replication: Backups-disk
Figure 7. Veeam Backup & Replication: Backups-disk 

Conclusion

Is DC backup that simple? Yes and no. Successful backup is great for starters, but that’s not all you need. Like we say at Veeam, “Backup is not worth a penny if you can’t restore from it.”

The following articles in this series are dedicated to different AD recovery scenarios, including the restore of a particular DC, as well as the recovery of individual deleted and changed objects using native Microsoft utilities and Veeam Explorer for Active Directory.

Backing up Domain Controller — Best practices for AD protection

Ultimate FAQ for Scale-out Backup Repository

Source: Veeam

Veeam Availability Suite v9 has been out for more than 2 months now. Many users have already upgraded, and are embracing it as well as asking questions about the new version and its features. Among the new features, there is one we’re especially proud of: Scale-out Backup Repository. This one gets a lot of questions in communities and I’ve decided to address them by issuing the ultimate FAQ. This should answer the most common questions (if not, let me know, I’ll add your question and the answer) and help describe how Veeam’s Scale-out Backup Repository works.

 

Q: What’s Veeam Backup Repository?

A: It’s a place where Veeam Backup & Replication can store backup files. Technically, it’s a folder on a disk/storage, configured from Veeam GUI (or by PowerShell cmdlets). Veeam has full access to this folder, and can read and write files.

 

Q: What types of repositories does Veeam have?

A: As of version 9.0, there are 4 repository types Veeam offers:

  • Windows Server with local or directly attached storage
  • Linux server with local or directly attached storage
  • CIFS (SMB) share
  • Deduplicating storage appliance (EMC Data Domain, ExaGrid, HPE StoreOnce)

Q: Oh, wait! Where is Scale-out Backup Repository? Is it a separate piece of hardware?

A: Veeam’s Scale-out Backup Repository is a software object. It is a logical entity that groups multiple “simple” repositories (see above) into one abstract repository. Thus, Veeam Backup & Replication creates a pool of storage devices, summarizing their capacity and increasing their performance when set properly.

 

Q: Why would I want to have such a thing?

Backup repository administration can be a hard job. It requires a lot of resources from an IT administrator. Repositories have a fixed size, and space may not be consumed properly over time. Not to mention how painful micro-management is, and how difficult it is to plan for future scalability.

Using Scale-out Backup Repository allows you to reduce backup storage management, solving the issue of underutilized backup devices and improving backup performance. As a result, companies can reduce their investments in storages making IT administrators and business owners happy.

 

Q: Ok, how can I visualize this feature graphically?

A:

Veeam before Scale-out Backup Repository
Figure 1. Veeam before Scale-out Backup Repository

 

Veeam with Scale-out Backup Repository
Figure 2. Veeam with Scale-out Backup Repository

 

Q: What kind of tasks does this feature support?

Backup jobs, Backup Copy jobs and VeeamZIP tasks can be pointed to this repository.

Replication (VM metadata), Configuration backup, VM copy and Endpoint backup are not supported as of version 9.0.

 

Q: Gotcha. So how can I create a Scale-out Backup Repository then?

A: In order to be able to set up such repository, you should create two or more simple repositories (actually it could be used even only one repository to start, but it would not make too much sense), then go to Backup Infrastructure and launch a Scale-out Backup Repository creation wizard. Provide the name of the repository, add necessary extents and define the placement policy.

Once you select the extents for this repository, all respective tasks using one of the involved simple repositories will be automatically reconfigured to use this newly created repository.

 

Q: You said something about policies. What are the policies?

A: The way Scale-out Backup Repository distributes backup files across multiple extents is set within the Scale-out Backup Repository policy. There are two policies: Data locality and Performance.

Data locality: All dependent backup files (from the same incremental chain) are placed in the same extent. However, the following incremental chain can be placed to any other extent.

Performance: Veeam Backup & Replication places incremental backup files to a different backup extent from the corresponding full backup file. This provides better transform operation performance as the I/O is distributed across the storage systems when it comes to this operation.

Even though the program attempts to follow these policies, it is flexible. If the needed extent is full or unavailable for example, Veeam Backup & Replication violates the policy and places the new backup file to an available extent. Better to violate a policy and still guarantee a successful backup!

 

Q: How does the extent selection go when a backup task starts?

A: Before a backup task starts, Veeam Backup & Replication applies a special algorithm, aiming to select the most suitable extent at this particular moment of time. Prior to selection, it checks the availability of backup extents, backup placement policy for this task, limitation for a maximum number of tasks that the extents can process, amount of free space at the extents, and presence of files from this backup chain at the extents.

 

Q: What happens if an extent with a backup chain/previous incremental file is down/offline when the task starts?

A: Veeam Backup & Replication checks the presence of the extent and fails the job if the extent is not available.

You can prevent this by enabling the “perform full backup when required extent is offline” option in advanced settings of the corresponding extent. This way the program starts over the backup chain and creates an active full backup file when the necessary extent is not available. Remember you are trading backup completion with additional consumed space with this option.

 

Q: Are there any service actions I can do with extents? What if I need to shut it down and replace the hardware?

There are a few service actions. You can put an extent in maintenance mode at any time. This means, that Veeam won’t start any new task targeted to this extent, and restore operations from files this extent contains won’t be available.

If you want to retire this extent from Scale-out repository, you can perform an automatic backup files evacuation once the extent is in maintenance mode. This operation will migrate backup files off this extent to other extents belonging to the same Scale-out repository according to previously defined policies.

Scale-out Backup Repository. Evacuate backups
Figure 3. Scale-out Backup Repository. Evacuate backups

 

Q: Anything else I should know about Scale-out Backup Repository?

A: Besides the repository itself, we introduced a per-VM backup files option (enabled by default) for each Scale-out Backup Repository. Until version 8, we used to have only one method — grab all VMs from the same job and store them into the same backup file. Now, we are able to create a separate backup chain for each VM belonging to the same backup job. This allows our solution to be more flexible in terms of storage utilization and improve the backup speed when working with storages in parallel.

 

Q: Oh, that’s cool but I’m afraid there is no more deduplication, isn’t it?

Surprisingly, it is not. We continue to perform deduplication and compression per-VM like before. When VM data is processed by Veeam components, it is split to data chunks that we dedupe and compress before placing them to an actual backup file. Plus, you can always use your deduplicating storage as a target for backups. This way Veeam’s technologies will be complemented with storage’s post-line deduplication, giving you good performance results and storage savings at the end.

 

Q: What if I upgrade from v8 to v9, what will happen with existing jobs/chains?

We always follow the rule “no reconfiguration should be done within an upgrade.” It means, that upgrades won’t touch your existing jobs/configuration. Just after the upgrade you will be able to see a newly created Scale-out Repositories node in Backup Infrastructure.

Configuring a Scale-out Backup Repository and pointing it to simple repositories with existing data comes with a warning “jobs and backups using this extent will be automatically updated to point to this repository.” If the extent contains unsupported job data (replication, configuration backup), it should be cleaned from this data and the corresponding jobs should be reconfigured to another simple repository.

If you choose the “per-VM backup files” option, the actual split of backup files happens within the next full backup run.

 

Q: Great. What is the cost of this feature? What type of license do I need?

A: Veeam’s Scale-out Backup Repository is aimed for large companies with lots of repositories to handle. It’s available starting with the Enterprise license. Using this license, you’re able to create one Scale–out Backup Repository with a maximum of three extents. With the Enterprise Plus license, it’s possible to create an unlimited number of repositories, each with unlimited number of extents.

 

Q: We are a Veeam Service Provider, can we use Scale-out Backup Repository for storing customers’ backup?

As of version 9.0, Scale-out Backup Repository is not available for such a scenario, but we’re keeping this in mind and considering to include this functionality to v9 updates. However, you can already use Scale-out Backup Repository for your own internal backups (backups received without Veeam Cloud Connect).

 

If you didn’t find your question above, please post it below and I’ll extend the FAQ.

Helpful Resources

Ultimate FAQ for Scale-out Backup Repository