Reader’s problems: A corrupt Exchange 2010 install into an Exchange 2003 organisation

Welcome to the first of an occasional series where I’ll walk you through the steps to solve a reader’s problem. I get a lot of mail from people who read my blog asking Exchange and Live@EDU related questions, and some are more involved than others. Earlier this week one reader emailed me with a problem he’d had after the install of his first Exchange 2010 server into his Exchange 2003 environment.

In short, Exchange 2010 installed fine but the media that Exchange was installed from was corrupt; there were a number of .Net DLLs that were 0KB in size and other components that hadn’t registered correctly. Management wasn’t possible using Exchange Management Console or Exchange Management Shell However, the install had completed and the Mailbox and Public folders were online and visible to his Exchange 2003 organisation:

Hi Steve,

I really hope you can help me, I have seen your website and the success others have had from contacting you, so I hope your advice can help in my situation.

I have recently installed Exchange 2010 SP1 into an existing Exchange 2003 environment and read all the documentation to prepare me for the install.  I carried out all the pre tasks and continued with the install.  I will start at the beginning.

Everything worked as planned until the mailbox role was installed, the error I received during I found online to refer certain DLL’s which have a 0kb value.  So, as advice, I replaced this files and all roles with exchange installed successfully.

Next, when I attempted to access both the EMC and EMS, I receive the following error message:
"Connecting to remote server failed with the following error message:  The WinRM client can not process the request .  It cannot determine the content type of the HTTP response from the destination computer".
Next, from sources found on the internet, I checked the modules WMAN and kerbauth to check if they were native or managed and now I am getting this error:
Could not load file or assembly "Microsoft.Exchange.Configuration.RedirectionModule. version=14.0.0.0, culture=neutral…or oneo f its dependenices.  The system cannot find the file specified: c:\program files\microsoft\exchange server\v14\clientaccess\powershell\web.config line =42).

Now i’m worried the installation was in some way corrupt due to the initial problem with the dll files.
I have now tried to uninstall exchange to start again with a new ISO but I am getting errors here as well.  At this point I am ready to pull my hair out, this is the worst install I have ever experienced.
Note: Windows 2008 Standard R2 SP1

I would appreciate any advice you could provide.

Best regards

Buddy

Although it’s possible to fix these components slowly and bring the server into a serviceable state – would you want to run your production environment on that server, not knowing what else might be corrupt or have other issues? No! It’s a freshly installed server, hosting no production mailboxes or any live data; Buddy hasn’t even been able to make a single configuration change yet. I suggested a number of options but the two main options are to perform a recovery mode install (my personal choice) and to uninstall and re-install again (Buddy’s choice).

I’ll run through both those options on simulated “broken” setup that experiences similar errors:

1) Recovery Mode Re-Install

I prefer the recovery mode option because we’re retaining the Active Directory configuration, databases and simply re-installing the server from fresh media. I think it’s the quickest way to get back up and running with the least risk. To perform a recovery mode re-install we do the following:

a) Get access to Exchange Powershell (in this case, using Add-PSSnapin *Exchange*); dismount databases and record information about their on-disk location. Check databases are cleanly shutdown with eseutil -m, then copy the databases to a safe location.

b) Re-build the server after resetting the account in Active Directory.

c) After re-installing pre-reqs, perform a recovery mode installation from new, known good media; restore the databases to their original location then check the Exchange Management Console works. Finally, we mount the databases…

As you can see there is no uninstall and nothing in Active Directory is modified apart from resetting the computer account. We rely on the backup of the database and the existing configuration in Active Directory.

Let’s take a look at some of this in practise. First up, here’s a demo of step A:

Unable to display content. Adobe Flash is required.

 

Next is step B; One thing you should do before re-joining the server with the same name is reset the AD Computer account:

image

After your re-install and re-join (I won’t cover those steps), let’s take a stroll through step C:

Unable to display content. Adobe Flash is required.

Afterwards, give it a reboot, and you should be ready to continue with your Exchange 2010 implementation.

As a footnote, I should add that if you aren’t able to use the Add-PSSnapIn method to gain emergency access to Exchange Powershell cmdlets, an alternative method could be to stop the Microsoft Exchange Information Store service, then perform the eseutil –m command on the Database after manually finding and recording the location of each database. In that scenario I would suggest restoring the databases to their original locations before running Recovery mode setup.

2) Uninstall and re-install

Once you’ve prepared the schema for Exchange 2010 there’s no going back. You’ve got an Exchange 2010-level organisation, unless you roll back your Active Directory. And if you want to just uninstall Exchange 2010 from the corrupt server, it’s not that simple.

If you’ve performed a typical install and try to uninstall, you’ll be presented with an error similar to this, stating that the Mailbox database containing mailboxes, that there is a Public Folder database and that there are routing group connectors that must be removed before uninstall can continue..

image

In a nutshell, we need to do the following:

a) Get access to Exchange Powershell commands (again, using Add-PSSnapin *Exchange*) and moving all Public Folder replicas off the Exchange 2010 server; removing the Discovery Search and Arbitration mailboxes. Theoretically there shouldn’t be any user mailboxes as we’ve never had real access to the Exchange server.

b) Remove the Mailbox database, Public Folder database and routing group connectors, then uninstall Exchange 2010.

c) Clean up the file system (possibly re-install the OS), re-run the AD prep tools then re-install Exchange 2010 from good media.

So let’s check out how we do this. First, step a and b:

Unable to display content. Adobe Flash is required.

And let’s run through the clean-up and re-install from clean media, step c:

Unable to display content. Adobe Flash is required.

In the above demos I’ve shortened some of the steps above (particularly the setup stages) and know that there isn’t anything valuable on this test environment – but before you delete anything, make sure you have a backup of everything even if you don’t think you need it. As this particular scenario relates to a server that had a corrupt installation, no certificates should have been installed, or server-specific configuration should have been performed; no mailboxes should have been moved to it either. If you environment does have any of the above they need to be backed up or moved before either method is attempted.

Finally, how to avoid this happening…

If you are about to install your first Exchange 2010 server into your environment, there are a few things you can do to make sure you don’t run into problems due to bad media:

1) Install Exchange from your media into a Test/Lab environment first. Not only does this give you an opportunity to test the media, but it also allows you walk through the steps you need to install Exchange Server 2010 in an environment that cannot harm your production systems.

2) Check file checksums before installation. Personally, I would recommend installing from the extractable version of Exchange Server 2010 SP1 available from here rather than ISO images from Microsoft Licensing downloads, Technet or MSDN., named Exchange2010-SP1-x64.exe

The MD5 checksum for Exchange2010-SP1-x64.exe is 53e70a12b58bedaa372e4f87f1db7be0

To check your download, grab a copy of Microsoft’s File Checksum Integrity Verifier utility. Extract it to somewhere convenient (such as the same location as the Exchange2010-SP1-x64.exe download) then simply run fciv.exe Exchange2010-SP1-x64.exe:

image

Hope this helps. As usual, comments and suggestions are welcome!

6 thoughts on “Reader’s problems: A corrupt Exchange 2010 install into an Exchange 2003 organisation

  1. I have a kind of similar problem and would appreciate your feedback. I recently added the mailbox role to an exchange server which had already had CAS & HT roles installed and this caused a problem as the server, part of a CAS array, was also a witness server to other mailbox servers in a DAG. At no point was the server participating in the DAG. So to expediate matters I rolled the server back to the previous nights image (they are all virtual servers) and everything is operating correctly.
    The only problem is that I am left with an exchange environment which thinks that the mailbox role is installed on the CAS server but it is not – effectively I have an orphaned role !!
    Is it possible to re-run setup in recovery mode on this server (without resetting the account or rebuilding the server) so that the mailbox role is re-installed. I could then uninstall the mailbox role and that should then bring everything back in to order.

  2. Hi Steve,

    Re: Reader’s problems: A Corrupt Exchange 2010…Step b) Re-build the server after resetting the account in Active Directory.

    After Step B: you say, “After your re-install and re-join (I won’t cover those steps), …”

    Would you mind covering those steps with me, as I am confused about what you mean? Do you mean re-install / rebuild the Server 2008 R2 Server that Exchange Server presently resides on and then rejoin AD on the DC?

    My situation may be a little different but it still seems to fit. I have 1 – DC (Server 2008 R2) and 1- Exchange 2010 Server. The DC’s hard drive fried and the backup was corrupted a week before it happened. I had to build the DC from scratch. I ran setup /PrepareAd again from the Exchange Server but none of the Exchange Services would start. Afterwards, I discovered I had allowed the AD build on the DC to automatically name the NetBios Server name, whereas it was slightly different on the original install, which was a transfer from Exchange 2003. The AD had been bumped up to Server 2008 since then. I had to recreate a new logon account on the Exchange Server after joining this “new” netbios domain (the FQDN was the same) but still no joy. I did what I knew but could not solve it and figured it was an impossible situation. Everything looked good to go on the DC end. So, I found your great instructions on reinstalling in Recovery mode. Seemed like the best solution.

    I have the mailbox and public folder databases backed up with the created .txt files. I did them manually because I could not mount the databases. I was ready to start Step b. when I realized I did not understand what you meant. It would be a great deal more difficult for me to reinstall the OS on top of reinstalling Exchange. The Exchange Server is also a Certificate Server and has Symantec Endpoint and several third party products that take an act of god’s keys to heaven to install. All the information to the keys reside in Exchange which is unavailable.

    Any advice at this point?

    Most humbly and with best regards,

    Lynny Davis Moore

  3. THANK YOU! THANK YOU! THANK YOU!
    You are a LIFESAVER!
    MS really needs to make a script that can simply/fully remove a server that goes down, instead, of having to rebuild one just to get rid of it?! Your examples are simple and straightforward to understand, and most importantly, actually WORK – THANK YOU!

      • Hi Steve, I do prefer the recovery mode option as well, in fact that’s exactly what I did, my exchange 2010 coexisted with a 2003 environment and still does (however user mailboxes are still on 2003). The whole process went very smooth, Mailbox database and public folder database are visible in EMC. not a problem…. well wait because THE inexplicable flaw here is that those databases do not physically exist!!!! in other words when I type my command “Get-MailboxDatabase | fl *Path* the EdbFilePath leads me to an empty folder”
        I think I’ve never seen something like this before.

        Your help would be so much appreciated Steve, you have no idea! 🙁

Comments are closed.