Restoring Windows 2003 Servers via Image based backups

We have several customers on our Managed Backup product that have Windows 2003 servers that they do not intend to upgrade or replace with newer operating systems.  

We always help our customers, so when this customer had a virtual host fail, we sprang into action.  All guest servers were restored into the recovery environment via StorageCraft quickly – with the exception of the Windows 2003 server.  It was fully patched, of course, but would not boot.  The error looked like so:

STOP: c0000135 {Unable To Locate Component}This application has failed to start because CSRSRV.dll was not found. Re-installing the application may fix this problem.

The error is documented in the Microsoft Knowledgebase here:https://support.storagecraft.com/s/article/c0000135-CSRSRV-dll-was-not-found-when-restoring-Server-2003?language=en_US

To successfully boot the server, you must uninstall KB3045999.  But how do you uninstall a patch when you can’t boot the server?

I had 100% success with the below procedure, which replaces the patched files with the originals and will enable the server to boot..

  1. From any server that can mount Storagecraft images as volumes, do a StorageCraft mount of the Target server’s boot disk image on Z:
  2. Open an Administrative Cmd Shell.  It has to be an administrative shell to perform the copy.  
  3. Move to z:\Windows\$NTUninstallKB3045999\spuninst . This is the uninstall information and original files for the machine you are restoring.
  4. Open spuninst.txt in Notepad.
    1. Use Notepad’s find/replace function to replace all instances of “C:” with “Z:”.
    2. Save as spuninst.cmd.  
    3. Close Notepad.
    4. Run the spuninst.cmd script.  This should replace the patched files with their originals.
    5. Dismount the Z: drive, saving the changes to a new patch file.  DO NOT USE QUICK DISMOUNT.  A quick dismount will not give you the option to save the disk as a patch file.
  5. Move to the Patch file – right click on it and select “Virtual Boot.”
  6. From the Wizard, add any other drives that need to be added to the Virtual Machine.  This step is not necessary when using the regular Storagecraft chain, but is necessary when using the “patch file.”
  7. Before you boot the virtual machine – make sure you heed VirtualBox’s warning about the minimum of 64 Mbytes of Video RAM.  Otherwise, the machine will crash when booting.
  8. Boot the Virtual Machine. The VM will perform a chkdisk on all disks.  Probably good to allow Windows to finish the chkdisk.

The server should now boot normally.

Change Monitoring on the Network…

I have experience with quite a few change monitoring systems on the network.  I am distinguishing change monitoring vs change management in that “management” always implies some measure of “control.”

Some quick Pros and Cons of various Systems:

For Active Directory, I have used:

  • ManageEngine AD Audit+: This is a web based application that can monitor and alert on changes to Active Directory objects, including Group Policy Objects.  It can send regular reports that will list the principle who made the change, the object that was changed, and,both the before and after values. The AD specific application is priced by the number of Domain Controllers in your environment.  This can be cheaper for environments with thousands of objects, but fewdomain controllers.
  • Lepide AD Auditor:  Lepide’s offering is a thick client application that connects to each Domain Controller.  It has all the same features as AD Audit + for monitoring Active Directory objects.  It is priced based on the number of objects in your environment, so may be cheaper for smaller environments with under 100-ish bbjects.
  • Netwrx:  Poor User Interface, scalable, and costs as if it were made of gold.  But an add-on module includes vSphere monitoring, which we found very useful.

Message Analyzer Note

How to get a packet capture on a Windows Machine that does not have Message Analyser, Network Monitor, or Wireshark installed on it:

netsh trace start capture=yes Ethernet.Type=IPv4 IPv4.DestinationAddress=<local ip address>

This will create a trace file nettrace.etl by default.

ETL files can be read with Network Monitor 3.4 (My favorite for its simplicity, and the fact that I have been using it so long), Event Viewer, PerfView, and the most comprehensive application: Windows Performance Analyzer.