Upgrading SharePoint Portal Server 2003 to Microsoft Office SharePoint Server 2007 (MOSS 2007) using Database Migration

Assumptions

This post assumes that you are currently running Microsoft SharePoint Portal Server 2003 (SPS 2003) and that you want to upgrade to Microsoft Office SharePoint Server 2007 (MOSS 2007). I assume you are running in a pretty simple configuration such that you have a separate database server, and the rest of your SPS2003 installation is on one server.

One thing that I have that many people may not have that I have is two installations and versions of SharePoint on one server. I have SPS 2003 and MOSS 2007 on the same server. There are separate urls for both instances of SharePoint. I don’t want to do anything to the existing MOSS 2007 installation, but I DO want to upgrade SPS 2003 to MOSS 2007. In the end, my goal is to have one instance of MOSS 2007 with two different web applications (one that already existed and one that was converted from SPS2003) and no instances of SPS2003. FYI, this is all because I inherited the MOSS 2007 instance after I already had SPS2003, and I only had the one server to run both instances.

You should be running Service Pack 3 for SPS2003 and WSS 2.0. Click here for instructions.

The biggest assumption is that you want to do a database migration. The advantage is that you can work at the upgrade and still have your old installation available. Read this page before you decide if this strategy meets your needs:

If you don’t want to do a database migration, I read that you can use psconfig –upgrade to do an inplace or side by side (gradual) upgrade. You can use stsadm –o upgrade to force the the upgrade again from where it left of (assuming it fails) and I assume if you didn’t start it yet also. In my case I am doing a database migration and need to make it run again. I think this is the same as doing an inplace and giving it the new url that points to MOSS site we create (later). 

NOTE: Once a site definition has been converted you need to start the conversion over again on a virgin SPS2003 content database. It doesn’t reconvert the site definitions. In other words any custom mapping file you created can only be applied once before you have to start over. For more information on migrating Custom Site Definitions, see my other blog entry.

.

Summary of Actions

Below is a quick overview of what you need to do. Please see the sections after this section for more details on pre-requisites, more details, etc for each of these steps

On SharePoint Server

Run PRESCAN

cd C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>

PRESCAN.EXE /c preupgradescanconfig.xml /ALL

Review log / summary generated by it. Search for word error and review end of file for summary counts.

On Database Server (or any SQL Server Management Studio)

Backup content database

Restore content database under another name (i.e. CompanyName_WSS_Content)

On MOSS 2007 Central Administration

Create new web application in MOSS

WARNING: Do NOT specify the content database as the one from above (i.e. CompanyName_WSS_Content)

WARNING: Do NOT click on the Create the Site Collection link

On SharePoint Server

Install Custom Site Definitions, Mapping files, etc

Make sure the following paths have your necessary files

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates\MyCustomSiteDefinition

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\1033\XML\WEBTEMPMyCustomSiteDefinition.xml

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\CONFIG\UPGRADE\MyCustomSiteDefinitionWssUpgrade.xml

Add Content Database

cd C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN

stsadm.exe -o addcontentdb -url http://hostname:port -databasename CompanyName_WSS_Content -databaseserver host\instancename

Review C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\LOGS\Upgrade.log for errors, exceptions, warnings, etc. These are good keywords to search on

Remove Temp Content Database

Pre-upgrade requirements

Since I am running MOSS 2007 already, I by definition have the .NET Framework 3.0 and ASP.NET 2.0 enabled in IIS. For more information on how to install these, see here.

According to this page you also should make sure that the minimum permissions for the account that you use to run Setup and the SharePoint Products and Technologies Configuration Wizard are:

• Member of the Administrators group on the local computer that runs Office SharePoint Server 2007.

• In SQL Server, the account must be:

• Authorized to access all SharePoint Products and Technologies databases.

• Granted the Database Creators (dbcreator) fixed server role.

• Granted the Security Administrators (securityadmin) fixed server role.

I would also recommend reviewing  this: http://technet.microsoft.com/en-us/library/cc303417.aspx for more details on pre-upgrade steps.

If you are doing this on a production system, PLEASE backup your stuff; at least the databases. Please understand, I can’t begin to explain everything in one blog post that takes Microsoft 238 pages in this ebook. This is just what I did.

Backing up your Content database

NOTE: This step is optional, but is always a good idea. Backup up your content database BEFORE you run the Prescan tool since it does change your content database a little.

The content database in SharePoint Portal Server typically end in _SITE. For example, SPS2003_SITE. Just use the typical database tools (For example Enterprise Manager or MS SQL Server Management Studio) to do the backup.

Install Custom Site Definitions, Mapping files, etc

Make sure the following paths have your necessary files

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates\MyCustomSiteDefinition

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\1033\XML\WEBTEMPMyCustomSiteDefinition.xml

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\CONFIG\UPGRADE\MyCustomSiteDefinitionWssUpgrade.xml

Run Prescan

If you didn’t do this, do it now. Open a command prompt and execute the following (assuming you installed MOSS 2007 in the default location, otherwise you’ll need to adjust the path to your location).

cd C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>

PRESCAN.EXE /c preupgradescanconfig.xml /ALL

NOTE: The /c preupgradescanconfig.xml must be used if you are upgrading from SPSS 2003 like we are.

Running it with the /ALL will not affect your MOSS 2007 installation. It only scans the older versions of SharePoint (SPS2003) from what I can tell. At least that is all it did for me.

You can also use the /V to specify the path. Use /? switch to get all the command line options

Search the generated file for the word error to see if you have any big errors

I recommend keeping the command prompt open. We’ll need it later.

Set the previous version databases to be read-only

Be sure you capture all of the data in your backup so that you are restoring and upgrading the current state of your environment. Therefore, set the previous version databases to read-only so that users cannot add or change information in the sites. With the databases set to read-only, users can continue to view content, but they are not able to add or change content.

Important:

Before you perform these procedures, be sure that you have run the pre-upgrade scan tool on the sites stored in the databases. If you do not run the pre-upgrade scan tool on the data before you back the databases up, you will not be able to upgrade the data on your new server farm.

Set a database to read-only in SQL Server 2000

1. In Microsoft SQL Server Enterprise Manager, right-click the name of the database that you want to set to read-only, and then click Properties. 2. In the Properties dialog box, click the Options tab. 3. Under Access, select the Read-only check box, and then click OK.

Set a database to read-only in SQL Server 2005

1. In SQL Server Management Studio, right-click the name of the database that you want to set to read-only, and then click Properties. 2. In the Select a page section, click Options. 3. In the right pane, under Other options, in the State section, next to Database Read-Only, click the down arrow, and then select True.

NOTE: This section was taken directly from age 138 of Upgrading to Office SharePoint Server 2007 (Free eBook).

 

Backing up your Content database

NOTE: This backup is a MUST because it will be used to restore from. Run prescan BEFORE you do this backup. You will now have one or two back ups depending if you did the optional backup or not. This is the backup you will use for the restore and upgrade. The other backup is just incase you need to rollback to SharePoint Portal Server 2003.

The content database in SharePoint Portal Server typically end in _SITE. For example, SPS2003_SITE. Just use the typical database tools (For example Enterprise Manager or MS SQL Server Management Studio) to do the backup.

NOTE: While backing up the database the web front end will likely fail when loading a page.

Restore your Content Database

Use the typical database tools (For example Enterprise Manager or MS SQL Server Management Studio) to restore the database you just backed up. Be sure to restore it as a different name from the database name. This will change the filename as well, which is good. Be sure to uncheck the Overwrite the existing database checkbox so you don’t accidentally forget to change the database name and overwrite the original or other database.

You will want to give some thought as to what the name of this will be since it will be your new content database. I like CompanyName_WSS_Content.

NOTE: Be sure to change the database (that you just restored) from read-only to read write again. Otherwise the upgrade probably won’t work too well.

Create a new web application in MOSS 2007

NOTE: If you want to save yourself some steps later, you can stop your SharePoint Portal Server 2003 IIS web application (using IIS). Then when you create your application, you can use the old url that you had for SPS2003. Be aware that this has the downside of everyone seeing the new site immediately, and the old site being down longer. If you have scheduled downtime and good communication this method may be acceptable, but it does pose the opportunity for users to make change when you are not ready for it to go live yet.

Go to SharePoint Central Administration | Application Management tab | SharePoint Web Application Management section | Create or extend Web application link. Click that link. Now, click the Create a new Web application link. Set the url to a new url (it can’t be the existing one).  Set the Database Name to Temp_WSS_Content or something since we won’t keep it for long. In particular, do not set it to what the name of the database we restored above. After filling in all the fields, click OK.

When the web application is finished being created, you will see a page like the following:

Do NOT create a new Site Collection! This is VERY important. In other words do NOT click the Create Site Collection link shown above

Add Content Database

Now we need to add the Content database ( the one we restored) to SharePoint so it knows about it. This is also the point where the database is converted from SPS2003 content database to a MOSS2007 content database. This step may take a while since this is were the actual conversion takes place. To see the status of the conversion you can check the log file (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\LOGS\Upgrade.log)

You MUST use stsadm to add the content database. The UI will let you try but tell you it can’t because the operation could timeout if you are using a browser and that you should use stsadm instead.

Be sure to specify the Web Application is the one you just created above when you created the web application!

Be sure you specify the information to the database that we restored.

cd C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN

stsadm.exe -o addcontentdb -url http://hostname:port -databasename CompanyName_WSS_Content -databaseserver host\instancename

NOTE: \instance name for the databaseserver parameter is optional and only needed if your database is not on the default instance of SQL Server. This is common for a Clustered environment or where you have more than one SQL Server installed on one ser

The databaseuser and databasepassword are for SQL database users only. If you are using Windows Authentication, you don’t need these parameters since they will be passed from the context that you execute stsadm. So, make sure that user has the correct permissions.

Depending on the size of your content database the addcontentdb command may take a while. I have a 10GB on a low end server and it took well over 35 minutes (I had to leave while it was processing after 35, so I don’t know how long it actually took). Your mileage may vary.

For more info see here.

Remove the Content database

Now we need to remove the temporary Content database that was created in the above step from SharePoint and also delete it. You can use stsadm to do this, but I like the UI. Your choice, to use the UI do the following. Go to SharePoint Central Administration | Application Management tab | SharePoint Web Application Management section | Content databases link.

Be sure to change the Web Application to the one you just created above!

Click the Content database that you specified (Temp_WSS_Content ) when you created the web application above.

Click the Remove content database checkbox. This won’t delete the database from SQL Server though. You will need to do that once you have removed it. You may also want to wait until you are finished with the upgrade.

Estimated Time

There is no real way for me to say how long it will take you, but my test server is fairly old and my content database is 10GB. It roughly took me 1 hr 43 minutes to do it. Below is the breakdown of the time per step. Don’t forget to leave time to review the conversion. Also, the content will need to be indexed again so make sure customers are aware of it.

Prescan - 5 minutes
Review prescan - 3 minutes
Backup db (11GB) - 7 minutes
Restore db (11GB) - 10 minutes
Web App Creation - 8 minutes
Install Site Definitions - 8 minutes (approximately)
Add Content DB - 60 minutes (approximately)
Delete Temp Content DB - 2 minutes

Troubleshooting

I did run the prescan tool, but it says I didn’t.

Q: I get the following message:

The pre-upgrade scan tool has not yet been run on this database SPContentDatabase Name=CompanyName_WSS_Content Parent=SPDatabaseServiceInstance Name=dbhostname. You must run the pre-upgrade scan tool before you can continue with the upgrade process. Run the tool from the following path: C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\bin\prescan.exe. Troubleshoot issues with Windows SharePoint Services.

A: I got this problem when I applied MOSS 2007 Service Pack 3 then backed up and restored the database. I guess Service Pack 3 changes the database. It is also possible I forgot to run prescan.exe on the database before I backed it up.

 

References

• SharePoint 2003 to MOSS 2007 upgrade: Database migration This is an excellent blog on what I said. It is what I based most of my knowledge on.
• MSDN explains in detail how to do what I explain here. Detailed doc on what I explained above, but from Microsoft.
• Upgrading to Office SharePoint Server 2007 (Free eBook) 
  This will provide you with more than you ever wanted to know on how to upgrade. If you have a question, I recommend checking it. It is 238 pages and is very comprehensive.
• Upgrading to Office SharePoint Server 2007 (Online version)
  Use this for an shorter version of the above. I found it easier to follow because of less stuff to read through that didn’t match my scenario.

Converting a Site Definition from SharePoint Portal Server 2003 to MOSS 2007

Unfortunately, the Site Definitions from SharePoint Portal Server 2003 (SPS2003) to Microsoft Office SharePoint Server 2007 (MOSS 2007) have changed quite a bit. Well, unfortunately because it means more work for the conversion. The good news is that it is a much better model and easier to work with.

There are two things to consider here. One thing is that you will need an updated Site Definition, and the other is that you will need to map all previous standard SPS2003 paths to the ones used by MOSS 2007. The two things are related, but understanding the differences can help in troubleshooting any problems you may have.

Let’s start by looking at the files we need to have to do each.

Site Definition Files

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates\MyCustomSiteDefinition

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\1033\XML\WEBTEMPMyCustomSiteDefinition.xml

Mapping files for Site Definition

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\CONFIG\UPGRADE\MyCustomSiteDefinitionWssUpgrade.xml

Your Site Definition files can be added to any MOSS 2007 installation and then you will be able to create new sites that use your Site Definition. If you can do that and the site you create works as expected then this means that your Site Definition files are good. These are all you would need if you were just creating a new Site Definition for MOSS 2007.

The Mapping files are needed for when the site is migrated by the upgrade process from SPS2003 to MOSS2007. It needs to map all path that were used in SPS2003 to MOSS2007 compatible paths. This file is only applied once when the upgrade process runs. In fact, once it is applied I can’t find a way to apply it again without starting over with a virgin SPS2003 content database.

Creating a new Site Definition

I am by no means an expert at getting site definitions to work. All I have done is in SPS2003, I copied the STS Site Definition and made some very minor tweaks to it. Now I am in MOSS2007 and want to again make those very basic tweaks so that I can have all the functionality of MOSS2007 team sites. This means in my scenario the easiest way to continue is to not try to convert my existing Site Definition from SPS2003, but to just copy the STS Site Definition from MOSS2007 and make my tweaks to it.

Here is what I did.

1. Go to C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates and copy the sts directory and call it something unique like MySts. Make your tweaks as needed.
2. Go to C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\1033\XML\ and create a file called something like WEBTEMPMySts.xml. It MUST MUST MUST start with WEBTEMP and end in .xml, the rest is not important but should be meaningful to you. If you are copying the STS Site Definition like I am, here is all you need in that file. If you are doing something else, you’ll have to decide what you put in it.
   
   

   <?xml version="1.0" encoding="utf-8" ?>
   <!-- _lcid="1033" _version="12.0.4518" _dal="1" -->
   <!-- _LocalBinding -->
   <Templates xmlns:ows="Microsoft SharePoint">
    <Template Name="MySts"     ID="30001">
       <Configuration ID="0" Title="My Team Site" Hidden="FALSE" ImageUrl="/_layouts/images/stsprev.png" Description="This template creates a site for teams to create, organize, and share information quickly and easily. It includes a Document Library, and basic lists such as Announcements, Events, Contacts, and Quick Links." DisplayCategory="Collaboration">   </Configuration>
       <Configuration ID="1" Title="My Blank Site" Hidden="FALSE" ImageUrl="/_layouts/images/stsprev.png" Description="This template creates a Windows SharePoint Services-enabled Web site with a blank home page. You can use a Windows SharePoint Services-compatible Web page editor to add interactive lists or any other Windows SharePoint Services features." DisplayCategory="Collaboration">   </Configuration>
       <Configuration ID="2" Title="My Document Workspace" Hidden="FALSE" ImageUrl="/_layouts/images/dwsprev.png" Description="This template creates a site for colleagues to work together on documents. It provides a document library for storing the primary document and supporting files, a Task list for assigning to-do items, and a Links list for resources related to the document." DisplayCategory="Collaboration">   </Configuration>
    </Template>
   </Templates>
   

   You don’t have to use 30001, but it does have to be unique with regards to Microsoft Standard ids (so don’t use 1, etc). I think it is anything over 10000 is for user created Site Definitions, but 30000+ is definitely a safe range.
   Check out Upgrading SharePoint Server 2003 Customizations to SharePoint Server 2007 under the Upgrading Custom Site Templates section more information.

Creating the mapping file

The mapping file is used for conversion only.

Go to the C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\CONFIG\UPGRADE directory

Since we are doing the STS site, all we have to do is copy the WssUpgrade.xml to another file and call it something like MyStsWssUpgrade.xml. The name is not important, but must be in this directory in order for the upgrade process to find it.

I then did the following

1. Replace All: FromPath="{LocaleId}\STS with FromPath="{LocaleId}\MySts
2. Replace All: ToPath=  "SiteTemplates\STS with ToPath=  "SiteTemplates\MySts
3. Replace All: FromPath="{LocaleId}\STS\Lists with FromPath="{LocaleId}\MySts\Lists
4. Change ID=”1” to ID=”30001”

In the end all paths should reference your MySts directory, not the STS directory

For more information on how to migrate from SPS2003 to MOSS2007, check out my other blog entry.

Verifying your conversion

For starters the site definition should look like you expect it to. In our scenario, it should look and act like all other MOSS 2007 sites, not like it used to.

Also, go to the MOSS 2007 content database using SQL Management Studio. Run the following query.

select fullurl, webtemplate, ProductVersion from webs order by productversion

If all the sites where converted to MOSS2007, they will have a ProductVersion = 3. If there are any that are not converted, they will have ProductVersion = 2. This will cause you grief down the road if you don’t get it to convert to version 3 and will also cause MOSS to run those sites in compatibility mode and there is a negative performance hit with that.

You can also review the log carefully. It should say that it upgraded each site. The log file is usually C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\LOGS

Reset password for Remote Desktop Connection

Newer versions of Remote Desktop don’t save the password in the .rdp file anymore. I don’t know where it is stored. If anyone knows, please let me know.

I have a bunch of .rdp files that I want to not prompt me for a password, but instead just open the Remote Desktop session. The problem is that when my password changes in Active Directory, where ever my password is saved for these .rdp files, it is not updated automatically. This means that when I open the .rdp file it says the remote server says I have not entered the correct password, check my username, domain, etc.

The workaround to this is to just wait for the alert to pop up, click ok, and then type in my new password. The problem is that it does not remember the new password.

The easiest solution I have found is to do the following.

1. Select all your .rdp files in Windows Explorer.
   
2. Right-click on of them and choose Edit. This will open the edit window for each of these .rdp files.
   
3. For each window click the edit link as shown below.
   
   
   
4. You will be prompted to enter your username and password. Be sure the username has the domain\username as the username if you are using Active Directory username. Otherwise, the local login on that server will be assumed.
   
5. Click Cancel. No need to Save unless you change other values on the window shown above.
    

While it is not automated, it should only take about 5 seconds per .rdp file once they are all open which should not take long either. This assumes you can type your password really faster or better yet, open up notepad type your password, copy it, and then instead of typing your password for each .rdp file, you can just paste it. This will reduce your errors and make your life easier.

FieldValue in DynamicData ForeignKeyField is null, but it should not be

I am using ADO.NET Entity Framework and Dynamic Data in a ASP.NET Formview. It works great! Let’s pretend I am working with a Person entity. Then I added a Foreign key to my table (and EF Model) to another table called it Gender. I add the DynamicControl to my formview for the foreign key by setting the DataField to the name of the relationship from Person to Gener, in this case it is called Gender.

I added this to EditItemTemplate

<asp:DynamicControl ID="GenderDynamicControl" runat="server" DataField="Gender" Mode="Edit" ValidationGroup="Edit"  />

The two-way binding works great.

Then I go to the ItemTemplate and add

<asp:DynamicControl ID="GenderDynamicControl" runat="server" DataField="Gender" Mode="ReadOnly" ValidationGroup="ReadOnly"  />

I then test, and see that the ItemTemplate / Read only version of this doesn’t show any value. After debugging and looking at the FieldValue property in ForeignKey.ascx.cs I see that it is null when it should not be since the row I was looking at had a GenderID specified in the Person table.

Then I looked at the CRUD interface that you get for free when you create a Dynamic Data Web Site. It works great! That made no sense to me, but then I realized that the EntityDataSource I was using did NOT include the Gender entity, so this entity was not loaded. This is VERY characteristic of ADO.NET Entity Framework since it does lazy loading and requires that you specify what you want loaded. If you don’t, you don’t get the data.

So, the real solution was to just modify my EntityDataSource so that I include the Gender table as shown below. Some of the properties have been removed for simplicity in the code.

<asp:EntityDataSource ID="dsPerson" runat="server" 
EntitySetName="Person" 
Include="Gender">

Change Max file upload size in MOSS 2007

If you can’t upload a file larger than say 50MB or 150MB for then it is likely you are running into the max file upload size that SharePoint sets.

Microsoft Office SharePoint Server 2007 (MOSS 2007) supports file upload sizes of 2GB. This is really a SQL Server limitation since files are written to the database. There are some related settings in web.config for the upload.aspx that you could change, but mine are already set to 2GB so that would not do any good.

For additional information on uploading file in general and this point, you may want to check out this post.

To check or change what the max size set in MOSS is, just open your Central Administration for the installation you want to check. Then go to Application Management tab | Web application general settings link and you will get the Web Application General Settings page.

In the example below you can see that I set mine to 1.5GB or really 1500MB.

Keep in mind this is set at the site collection level, not the site level.