Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation – Part II

This is one entry in a series of blog entries:

• Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation - Part I
• Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation - Part II
• Why aren’t my ForeignKeys being loaded on my Custom Dynamic Data page?

Adding Labels based on MetaData from Model

In Part I we did some pretty neat stuff with Dynamic Data. In this blog entry I will show how to have your labels be set to what is set in the MetaData.

Let’s do some additional clean up here. Let’s remove the ProductCategoryID and ProductModelID fields since they will never be used.

Next we need to replace static text labels with a Label control so that we can set it via code. For example, replace ProductID with

<asp:Label ID="lblProductID" runat="server" AssociatedControlID="ProductIDDynamicControl"  OnLoad="LoadLabelForDynamicControl"></asp:Label>

I recommend a consistent naming convention to make remembering control ids. Once you have replaced one template such as the EditItemTemplate you can copy it to the InsertItemTemplate and the ItemTemplate, but be sure to change the Mode property to Insert and ReadOnly respectively.

Since we will have to get the metadata for each column on the FromView we will create a method that does this for us. Below is that method.

/// <summary>
/// Assuming you have a Label and DynamicControl (with the AssoicatedControlID on the label control set to the DynamicControl)
/// on your FormView it will set the text of the label control to the DisplayName which is from the MetaColumn for the column
/// used by the DynamicControl.
/// </summary>
/// <param name="labelControl">The label control that will have its text changed</param>
private void SetLabelBasedOnMetaData(Label labelControl)
{
           
    string dynamicControlID = labelControl.AssociatedControlID;
    if (string.IsNullOrEmpty(dynamicControlID)) throw new Exception("The AssociatedControlID must be set for Label control with ID: " + labelControl.ID);

    var container = labelControl.NamingContainer;
    var userControl = container.FindControl(dynamicControlID) as DynamicControl;
    if (userControl == null) throw new Exception("Could not find dynamic control with id: " + dynamicControlID);

    labelControl.Text = userControl.Column.DisplayName;
}

This will be called from the OnLoad event for each label.

protected void LoadLabelForDynamicControl(object sender, EventArgs e)
{
    Label labelControl = sender as Label;
    SetLabelBasedOnMetaData(labelControl);
}

If you run this now you will see that there is no real difference from the user’s perspective. However, the labels are being pulled from the MetaData (the Model). Now we just need to specify the label we want to display. This is the same label that is used in generated error messages.

 

To specify the labels, open the AWDomainService.metadata.cs and find the property you want to set the label for and add the following line.

[Display(Name="Product ID")]
public int ProductID { get; set; }

Do this for each property. Re-run and this time you will see that the labels are the values you set in the Display attribute.

It is a little more work, but in the end your labels will always be in sync from screen to screen and your error messages will reflect the name as well. When you need to make a change, you just do it once in your model. It is also the same information that your scaffolded screens will use as well.

 

All source for this project can be downloaded here.

Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation – Part I

This is one entry in a series of blog entries:

• Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation - Part I
• Create CRUD UI using ASP.NET 4.0, FormView, Dynamic Data, DomainDataSource, Entity Framework, WCF Domain Service, LINQ, and custom validation - Part II
• Why aren’t my ForeignKeys being loaded on my Custom Dynamic Data page?

The title is a mouthful and thus the topic can be kind of overwhelming. With ASP.NET 4 there are so many different technology frameworks that you need to be familiar with. The problem that I have found is that most examples only use a one or two of these frameworks. In my world, I want to use them all and the examples are never as simple as the examples. In this blog entry, I will give a read world example of how to combine all these technologies. I find myself having to figure out how to do this over and over again. Hopefully, this will be of help to others.

This blog entry is NOT about how to create a Dynamic Data project in VS2010 and create an admin UI or something like that. This is about creating a page from scratch that uses this technology. To better understand the different scenarios that Dynamic Data can be used in and steps on how to extend it, see here.

Technology Overview

I am writing in the context of ASP.NET 4.0 and VS2010 (Visual Studio 2010). These technologies are relatively new in some cases and the api’s have changed since betas. I am only showing stuff that works in VS2010 (the final released version).

• ASP.NET 4.0 – This is the version of ASP.NET that I am targeting.
• FormView – This is the control that we will be using as a basis for the CRUD UI (User Interface).
• Dynamic Data – This is the technology that allows you to use DynamicControl on your FormView instead of the standard ASP.NET Forms controls. BTW, you don’t have to have anything installed (not even the Dynamic Data files and templates). You get basic implementation without them, but with them you can customize and get a more full-featured set of tools. See here for a GREAT video on using Dynamic Data in “old” applications that don’t have all these cool technologies. For the most up to date resources, click here.
• Entity Framework 4.0 – The standard with VS2010 for database to object mapping. Click here if you want to see how to use Dynamic Data with an ObjectDataSource and GridView.
• WCF Domain Service – This is your middle tier where you would put your business logic, complex validation, etc. It provides a consist way to access your perform CRUD operations on your data. Abstraction layer before the Entity Framework.
• LINQ – We will use LINQ to Entities in the WCF Domain Service to query database through the Entity Framework.
• DomainDataSource – This is a control that works much like the ObjectDataSource or LinqDataSource or EntityDataSource except that it can connect to your WCF Domain Service from ASP.NET. Please note that there is also one for Silverlight and it has the same name so when you are Googling be sure to check what the context is. This is a good doc on using this control.

Getting Started

You can select most any type of web application project in VS2010 to get started. My instructions will be for the Dynamic Data project so that I don’t have to explain how to manually move over the Dynamic Data files. If you have an existing project and need to copy over the Dynamic Data files, click here for detailed instructions. Please note, I have copied some of these steps (and written others) from instructions I have read on MSDN, but have combined them together in one continuous instruction set. For example, most of the instruction are from: here and here.

1. Open Visual Studio 2010 and create a new project of type ASP.NET Dynamic Data Domain Service Web Application. I’m calling my project MyDynamicDataExample.
2. Optional - Create two Directories: Models and Services.
3. Optional - Download and install the AdventureWorksLT database if you don’t already have it. The particular database is not really that important to understand the concepts. Notice I am using the Lite version since it is a bit simpler to follow, etc.
4. Add a New Item… and select ADO.NET Entity Data Model. I called MyModel.edmx. We will generate from Database. Select the database you want to work with (add a new connection if it is not there). On the Choose Your Database Objects screen, select Product and ProductCategory. Keep everything else default.
5. You must build your project so that the next step will work.
6. Add (to Services folder) a New Item… and select Domain Service Class. I called mine AWDomainService.cs. Select the options as shown below:
   
7. Add (to Models folder) a New Item… and select Web Form using Master Page. Call it DDTest.aspx. Select the Site.master on the next screen.
9. From the General or Data group of the Toolbox, add a DomainDataSource control to the page. If you have never used the DomainDataSource before you will need to add it to your toolbox. To add it to your toolbox. Click the Tools menu, and then click Choose Toolbox Items. In the Choose Toolbox Items dialog box, click the .NET Framework Components tab, select the DomainDataSource check box, and then click OK.
10. Add a DomainDataSource to the page. Configure the datasource to point to GetProducts() method on the service you created. Enable Inserts, Updates and Deletes.
11. Add a QueryExtender to the page. Set the TargetControlID to the id of the DomainDataSource (mine is called DomainDataSource1).
12. Add a FormView (though a GridView or DetailsView could also be used). Check the Enable Dynamic Data Support checkbox. Please note it will not stay checked when you come back to it later (in some cases). Set the Data Source to the DomainDataSource you added earlier.

13. Add a DynamicDataManager control to the page. Chose Register Controls… and add the FormView as shown below.
    
    This registers the data-bound control and enables dynamic behavior for it.

14. Open the code-behind. Add a Page_Init method and add the following line:
    

    protected void Page_Init(object sender, EventArgs e)
    {
        FormView1.EnableDynamicData(typeof(MyDynamicDataExample.Models.Product));
    }

15. Go to the .aspx page again and go to the Source view. Find the QueryExtender you added earlier. Add the following line to it:
    

    <asp:QueryExtender ID="QueryExtender1" runat="server" TargetControlID="DomainDataSource1">
            <asp:DynamicRouteExpression ColumnName="ProductID" />
    </asp:QueryExtender>
    
    This will allow us to pass the ID that we want to edit via the url as a query string.

16. Now we need to do some clean up. In each of the templates in the FormView remove the Text and Dynamic controls for: ProductCategoryReference, EntityState, and EntityKey. Also, remove the rowguid from the insert template.

17. Let’s test to see if this works now. You will see that you get the YSOD (Yellow Screen Of Death) when you click the Edit button and then the Update button. You will get an EntityOperationException and the last method that was called was HandleValidationErrors. This means that something in the model failed.

18. Create event handlers on the DomainDataSource for the Updated and Inserted events. They should look like this when you are done:
    

    protected void DomainDataSource1_Inserted(object sender, Microsoft.Web.UI.WebControls.DomainDataSourceStatusEventArgs e)
    {
        if (e.ChangeSetEntry.HasError)
        {
            foreach (var error in e.ChangeSetEntry.ValidationErrors)
            {
                AddValidationSummaryItem(error.Message);
            }
        }
    }

    protected void DomainDataSource1_Updated(object sender, Microsoft.Web.UI.WebControls.DomainDataSourceStatusEventArgs e)
    {
        if (e.ChangeSetEntry.HasError)
        {
            foreach (var error in e.ChangeSetEntry.ValidationErrors)
            {
                AddValidationSummaryItem(error.Message);
            }
        }
    }
    

19. Next add this supporting method.
    

    public void AddValidationSummaryItem(string errorMessage)
    {
        var validator = new CustomValidator();
        validator.IsValid = false;
        validator.ErrorMessage = errorMessage;
        this.Validators.Add(validator);
    }

20. Now put a breakpoint on the Updated event handler and debug your application. Do the same test again, you’ll see there is some error about the field: ThumbnailPhoto. Note that the error does not display on the page, but it doesn’t go to the YSOD either. Like any other validation exception that is NOT column specific, we need to use the ValidationSummary control to view this error.

21. Drag a ValidationSummary control to your page. Re-run and you’ll see the exception. This does give little bit of database schema information. This may be considered by some to be bad. I’ll leave that up to you. If you don’t do this then you won’t get validation exceptions that are thrown from your domain service which is how you will implement your business logic. My experience shows that nothing too serious comes out of here once you have everything wired up correctly. If you don’t like that you’ll have to come up with another solution.

22. Let’s quickly fix this by removing the ThumbNailPhoto field from each of the FormView templates. Re-run, and this time you won’t get any errors. We can do this because it is an optional field.

23. Let’s add some validation to see how to do that. Go to the AWDomainService.cs and find the UpdateProduct method. Modify it so that it looks like this:
    

    
    public void UpdateProduct(Product currentProduct)
    {
        this.ObjectContext.Products.AttachAsModified(currentProduct, this.ChangeSet.GetOriginal(currentProduct));
        if (currentProduct.ListPrice < 2000)
            throw new ValidationException("List Price is too low.");
    }
    Re-run, change the List Price to less than 2000 and update and you’ll get the message shown in the ValidationSummary control.

24. You can also use ValidationAttributes such as Required, Range, RegularExpression, etc in the Model to declaratively validate the data. To do this open your AWDomainServices.metadata.cs. Find the StandardCost property and modify it so that it looks like this:
    

    [Range(0, 200, ErrorMessage="{0} must be between 0 and 200.")]
    public decimal StandardCost { get; set; }
    
    Re-run and and change the standard cost to something greater than 200 if it isn’t already and Update. You’ll see it says StandardCost must be between 0 and 200. Notice it filled in the name of the field for us. It is the actual property name though.

25. To display a user-friendly version of the property name you can add Display information as shown below.
    

    [Display(Name="Standard Cost")]
    [Range(0, 200, ErrorMessage="{0} must be between 0 and 200.")]
    public decimal StandardCost { get; set; }
    
    Re-run and you’ll see the same message but with the friendly name of the property.

26. To Edit a particular record such as the Product with ID = 800, just go to the same url, but with DDTest.aspx?ProductID=800.

27. If you try to run your application without specifying the specific test page you will get an error like this “There are no accessible tables. Make sure that at least one data model is registered in Global.asax and scaffolding is enabled or implement custom pages.”
    
    To fix that you just need to go to your Global.asax.cs and uncomment the line that start with DefaultModel.RegisterContext. Change it so that it looks like this:
    
    DefaultModel.RegisterContext(new DomainModelProvider(typeof(MyDynamicDataExample.Services.AWDomainService)), new ContextConfiguration() { ScaffoldAllTables = true});
    
    When you do this you’ll have some CRUD pages for each of the tables you included. This is separate from what we are doing here which is a custom page, but let’s fix the errors anyway. Another option would be to not have this default page and leave that line above commented. That way you won’t have all these admin page open to the world. Yes you can secure them, but they are not by default.
    

28. Back on track, when you access the Default.aspx page you will get an error like this: “The method 'Skip' is only supported for sorted input in LINQ to Entities. The method 'OrderBy' must be called before the method 'Skip'.”
    
    To fix this just go to the AWDomainService.cs and add a sort method to the default query methods which are GetProducts and GetProductCategories. Just make the methods look like these:
    
    

    public IQueryable<ProductCategory> GetProductCategories()
    {
        return this.ObjectContext.ProductCategories.OrderBy(o => o.Name);
    }
    

    public IQueryable<Product> GetProducts()
    {
        return this.ObjectContext.Products.OrderBy(o => o.Name);
    }

29. Re-run. This time go to the Product link. The same validation we added before is also here. This is because the validation is written at the Model, not the UI. This is AWESOME in my opinion. I love it. This is why I love Dynamic Data.

I hope this has been beneficial to all.

The entire source for this example (minus the AdventureWorksLT database) can be downloaded from here.

AzGroups 8th Annual Scott Guthrie Event

Things to check out:

NUGET

(pronounced NEWGET) – it is built into VS2010 (Visual Studio 2010) and makes finding, downloading, and installing libraries (open source included) into your project. It even does some configuration in the config files, etc. Pretty awesome. You can get it here.

elmah

It is an error logging facility that is completely pluggable that can added to a running ASP.NET web application without re-compilation or deployment. When an exception occurs it logs it, but allows you decide where to log it to such as a database, RSS, email, etc. It also records A LOT of VERY useful information for determining what happened.  See here for more details.

IIS Express

Can be used side-by-side with the built in web server (Cassini) in VS2010. It launches fast just like the web server built into VS2010, but doesn’t require you setup virtual directories, etc like you do with IIS (full version). You now get best of both worlds. It now let’s you develop using the IIS 7.x feature-set. It is now as easy to use as the web server built into VS2010. It doesn’t require admin account. You don’t have to change any code or the way you work to take advantage of it. See Scott’s blog for more details.

Web Matrix

You should still continue to use Visual Studio for professional work, but Web Matrix project are great for hobby, family sites, etc. It has been totally reworked to be a task based tool. It makes publishing very easy. If the site gets more complex the project can be opened up in Visual Studio.

SQL Server Compact Edition 4

It is great for ASP.NET development. It is light weight and free. It is an embedded database (no SQL Server required) so it is in the binaries (bin directory) that you can xcopy to your server. You can use Standard SQL Server syntax so anything that supports the ADONET provider model (such as LINQ and EF) will work. You are also licensed to redistribute it. All this means you can now deploy this in a medium trust ASP.NET shared web hosting such as DiscountASP, GoDaddy, etc. This means you don’t have to pay for the SQL database. It integrates into VS2010 (coming soon). It is different from previous version because it is safe to use with multi-threaded environments (such as ASP.NET). and no longer blocked from being used with ASP.NET. The catch is that it is for light-usage production (4GB or less for datafile) usage scenarios and it does NOT support stored procedures. For high-volume usage usage, you’ll probably want to use SQL Server Express (which is still free), SQL Server, or SQL Azure. Also, since stored procs are not supported the ASP.NET Membership is not supported (though they will be shipping new providers so that they do). The only change to your code when you migrate is your configuration string. See Scott’s blog for more details.

 

Portable Library Project

It is a special kind of project in VS2010 that only allows code that will work on particular targets. In particular, the targets are Silverlight and Silverlight for Window Phone 7. Intellisense only lets you see Intellisense on methods, properties, classes, namespaces, etc that are compatible with both. The most I could find on it was here.

Silverlight 5

Great stuff coming with Silverlight 5. Better DataBinding support for MVVM. Improved Printing and Text. Finally there will be a 64-Bit version of Silverlight. 3D Modeling that use Hardware acceleration. 3D and 2D can be used together. Breakpoints on XAML Bindings.

MVC 3

I am really impressed with MVC3. I am very tempted to try a project with MVC3. It takes advantage of HMTL 5 including Data validation, round corners, etc.

MVC Basics

The basic idea is that a url is mapped to a controller and method on that controller. That method makes calls to the model where most of the business logic resides, and then typically returns a view which is responsible for rendering the response (page). It maps url to action methods on controller class. In general the best practice with MVC is to have skinny controllers and fat models. Default method is called Index() and is called when no action is specified. Can pass entities as parameters to methods; just need to map them. Default parameters can be used as well. (i.e. int num = 5). ASP.NET MVC forces separation, while ASP.NET Forms does not.

Routing

When determining which rule matches it starts with the one that was added first. It then proceeds down the list until one matches. Routing is used to map urls to controllers and actions, but it is also used to generate urls in the application.

Glimpse

It is a an open source tool that allows you to diagnose what is happening on the server, but from the client. It is very useful when trying to figure out what happened once your request made it to the server. For example, what route was executed. It is basically Fiddler for the server.

For more information see here.

Razor

It is a View Engine for MVC. For more info see ScottGu’s blog. Razor files are .cshtml. Tags start with @ and it automatically figures out where code ends and content begins. Content and code can be intermingled. A layout page is basically like a master page in ASP.NET Forms. You can use traditional parameters or Named parameters. Use Custom helpers for custom rendering; refactoring in a way.

MVCScaffolding package

It is a code generator for MVC; a smart one at that. Highly recommended so you don’t have to write so much code to do CRUD UIs. Use NuGet Package Manager to get and install it.

Entity Framework 4.1

Great code first support. You can use POCO objects, make changes and the database just updates. It is so cool. This behavior is optional, but very nice. You can setup database initialization code to populate some basic data when the database is dropped.

Modernizr

A great tool that allows you to take advantage of HTML 5 when it is available and downgrade when it is not. An example of this is round corners. Very nice tool that can be used in most any ASP.NET project. http://www.modernizr.com/

Windows Phone 7

Later this year there will be a new release called Mango. It is a very exciting release as it will bring the Windows Phone 7 much closer to the competition as far as its offering. To get started with Windows Phone 7 development go here. Everything that is developed for Windows 7 Phone falls into two categories: Silverlight or XNA. XNA is for games (also used for XBOX), and Silverlight is for everything else. You can import some XNA libraries (such as audio) into Silverlight. Tombstoning is basically storing state to appear like the application never quit when in fact it did. It basically works by providing a state bag that most anything can be written/read to when different events such as (navigate to and from are called). It works much like Viewstate does in ASP.NET except it is not automatic. So more specifically it works like adding and retrieving stuff to/from viewstate in ASP.NET. The cost is $99 per year to put your application (or as many as you want except not more than 15 free ones) in the Marketplace. You can register to developer unlock your Windows Phone 7. Visual Studio 2010 comes with an emulator, but the one that will be in the mango release is sooo much better because it simulates things like sensors. The talk was given by Jeff Wilcox.

Tips on Windows Phone 7 development

Change splashscreen (an image) to all black if you app launches immediately. That way it is perceived as faster and cleaner. If your app takes a few seconds to load then put in a custom splashscreen (change the image) to give a more professional look. Add scrollviewer or list so that scrolling will always be available if something doesn’t fit on the screen. Use the metro look and feel styles, don’t hard code styles. If things don’t align as expected, then use 12px margins. Promote your app by creating a site for it. Prompt a user for review of your app after you think they have used it for a while. Statistically most apps are only used once and never used again. Use Watson to collect crash information, bugs, etc. Use Google Analytics to track page navigation within your app. Keep your main dll as small as possible since the entire thing is checked before loading into memory. Delay load pages / dlls on an as needed basis to help accomplish this. This also helps with startup times. Use a tombstoning helper. Use caching whenever possible.

Windows Azure – by Scott Guthrie and Mark Russinovich

Windows Azure is a great technology that needs some work with regards to deployment speed, and in general ease of use, but it is extremely well architected for scaling your application in a very elastic way. It allows you to worry about your app and not the infrastructure it is running it. It is Microsofts offering of cloud computing. The idea is that if you need 100 servers for an event you have them. Then after the event you don’t need them. With traditional models you would have to have servers ready to go at all times and pay for that much power. Since Microsoft has a giant server farm called Windows Azure and a middle tier between you and the server OS you only pay for what you use for the length of time you use it. Everything is continually monitored for 99.? uptime.

Mark went into all the details or the architecture. He convinced me it is very complex and robust, but not so fun to listen to a discussion about it. way too much details for my interest level. However, here as some of the things I found interesting. SQL is actually much slower than the way Windows Azure does it. They actually have everything as read only. Once it is read the objects are de-serialized and used. If they need to update, they are written back to the cloud as serialized objects. This takes only a trillionth of a second instead thousandths of a second that SQL takes. This allows them to scale fast. Their belief is that in the next 10 years or so this will scenario will be common more so than the relational database. Currently options in Azure are SQL, Blobs, and Queries. I believe SQL Azure is part of Windows Azure.

Some Barriers / solutions to get over before cloud computing will be accepted are: Trust of proprietary data being outsourced; loss of control; confidence (ISO and other certifications will help); Private clouds may be an alternative; license to run in own data center may help also.

Zune software is not installed. Install the latest version of Zune software

If you are using Visual Studio and opened an existing or created a new Windows Phone 7 project you may get the following build error.

Zune software is not installed. Install the latest version of Zune software

The reason it is complaining about the Zune software is that it is assuming that you want to deploy to a Windows Phone 7 DEVICE and NOT to the Windows Phone 7 EMULATOR. IF you are deploying to a REAL Windows Phone 7 DEVICE then you should install the Zune software. More likely, you want to test your project in the Windows Phone 7 EMULATOR that works with Visual Studio 2010. To target the Emulator, you just need to the target in your Drop Down List from Windows Phone 7 Device to Windows Phone 7 Emulator. After making the change, try debugging again and it should work much better.

300 blog entries now

I can’t believe I have written 300 blog entries over the years. WOW! This is SO COOL! Given the large number of people that visit my blog I am so thrilled to be able to give back to the community.

How to change the display format of a field using MetaData

Let’s assume you have a Domain Service class that you are access from your ASP.NET Dynamic Data application (should work for Silverlight client as well (I think)) and that you have a table called Person. The Person table has a field / property / column that is called BirthDate. By default it is displayed as a date AND time in the GridView and the Edit and Detail Forms. The way to change the display format is in the MetaData for the Person table. In particular the MetaData for the property BirthDate. The basic MetaData class is generated for you when you create the Domain Service class.

The question is what Attribute do we use to communicate what we want. The answer is: DataFormatAttribute. Use it like you would a string format in most other places in the .NET framework. In the example below only the date (not the time) will always be shown.

[DisplayFormat(DataFormatString = "{0:d}", ApplyFormatInEditMode=true)]
public DateTime PerformedOnDateTimeUtc { get; set; }

If we remove or set the ApplyFormatInEditMode parameter to false (as shown below) then insert and edit will still show the time, but the gridview will still only show the date (no time).

[DisplayFormat(DataFormatString = "{0:d}", ApplyFormatInEditMode=false)]
public DateTime PerformedOnDateTimeUtc { get; set; }

You can use custom date formats, use on different datatypes, etc.

How to determine if your Windows 7 Firewall is blocking your connection

If you are like me, you cannot turn off your Windows 7 Firewall for even a minute on your work computer. If you are lucky enough to have admin rights on your computer or you can add a rule to your firewall then you are in luck. It just so happens that you can add a rule that allows most anything to open a connection from your laptop to just about anywhere.

To add a rule, go to the Start Menu | Administrative Tools | Windows Firewall with Advanced Security. Right-click the Inbound Rules on the left navigation area and then select New Rule…

Next choose Custom rule. Click the next button.

Leave all programs setting. Click the next button.

Leave Protocol type to Any. Click the next button.

Leave IP addresses as Any. Click the next button.

Make sure Allow the Connection radio button is check. Click the next button.

Choose all three (Domain, Private, Public). Click the next button.

Give the rule something descriptive.

You have now opened up your computer for potential attack (at least from inbound). I highly recommend you not keep this rule enabled.

How to consume a SharePoint Web Service with WCF Client

SharePoint Web Services are great because they can be called from any computer that can connect to SharePoint, not just the server that is running SharePoint. This is nice for installations of SharePoint like MOSS 2007 or 2010, but it is critical for SharePoint Online (BPOS) or SharePoint 360 where you will NOT have access to the server that SharePoint runs on. The problem is they are not nearly as easy to work with as just using the SharePoint object model, and they don’t expose everything through the web services. This is a real bummer! It is still a very useful alternative for situations where you cannot use the SharePoint Object Model.

There are a few tricks for working with SharePoint Web Services. There are lots of ways you can access the web services. I prefer to use WCF to access the services so that I get cool syntax from XElement.

Here is a list of web services that are available. Let’s use the Webs web service. Below are the step by step (close anyway) to using the web service in Visual Studio 2010.

1. Create a console (for simplicity) or any other project type in Visual Studio 2010. NOTE: This can be on your laptop that is NOT running SharePoint.
   
2. Go to the docs for the Webs Web Service. Here you will see that the url is: http://<Site>/_vti_bin/Webs.asmx you need to replace <Site> with your hostname or dns name, etc.
   
3. Add a Service Reference to the above url (after you replaced <Site> with your site info). I am naming my MyWebs, but you can call it whatever you like, just make the appropriate changes to the code I show later.
   
4. If you require all users to be authenticated (don’t accept anonymous users) then you will need to change the WCF bindings that were automatically created in your app.config such that it will pass security information to the web service. To do this, just search for the url of the web service. That will take you to a XML tag called endpoint. In that same XML tag you will see an attribute called WebsSoap. In this same app.config do a search for WebsSoap. This will take you to a binding tag. In that binding tag you will see a security tag. That security tag should have a mode=”None” attribute. Change mode=”None” to mode=”TransportCredentialOnly”. Then just below that you will see a transport tag, change the clientCredentialType=”None” to clientCredentialType=”Ntlm” and change the proxyCredentialType=”None” to proxyCredentialType=”Ntlm”. 
   
5. All SharePoint services are work on Site Collections or Sites (webs) so really the docs should say http://<Site>/<subsite>/_vti_bin/Webs.asmx because if you are dealing with a subsite you need to use that url. You don’t need to add a service reference for each site, you can change it in the code. Be sure to change the endpoint address line in the example below.
   
6. Since we are using WCF, we get XElement which gives us LINQ syntax and objects from XML. How cool is that! When determining what is available as attributes it is easiest to just look in the debugger (use the visualizer).
   
7. Now that we the results as objects we can do whatever we want with them.

NOTE: You can also do this with the old Web Service client, but you don’t have XElement, LINQ support, but changing the URL is as easy as just changing the url and the Credentials property is all you need to change the credentials. You also don’t really have to worry about the WCF app.config stuff, but in the end I still like WCF.

 

private static void GetWebs()
{
    using (var ws = new MyWebs.WebsSoapClient())
    {
        // pass the proper credentials. Comment/Uncomment the proper lines depending on your situation
        ws.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials; // use current security context
        //ws.ClientCredentials.Windows.ClientCredential = new NetworkCredential("username", "password", "domain"); // use another Active Directory account
        //ws.ClientCredentials.Windows.ClientCredential = new NetworkCredential("username", "password"); // use account that is used for BPOS

        // change url to the site we want to work with
        ws.Endpoint.Address = new System.ServiceModel.EndpointAddress("http://myhost/sites/SomeSiteHere/_vti_bin/Webs.asmx");

        // get all the webs at and below the specified site
        var results = ws.GetAllSubWebCollection();
               
        // use XElement and LINQ to get our results and create objects from the XML.
        var test = from r in results.Elements()
                    orderby r.Attribute("Url").Value
                    select new { Title = r.Attribute("Title").Value, Url = r.Attribute("Url").Value };

        // the results are just objects now, so do whatever you want with them.
        foreach (var item in test)
        {
            //Console.WriteLine(item.Url + " | " + item.Title);
            Console.WriteLine(item.Url);
        }

    }
}

Migrating MOSS to BPOS –Part II

This is a great video on how you can pull data from SharePoint using LINQ to SharePoint. I think for an Alpha it is quire stable from what I can tell. To better understand LINQ to SharePoint, the video below goes into how to use Web Services, the SharePoint Object Model, writing your own CAML by hand, how to write you own LINQ syntax and how LINQ works. Very good course on LINQ itself, lamdba expressions, etc.

Highlights:

• CAML Builder Tool if you want to use Object Model and not hand write the CAML code
• Everything that can be done in the UI of SharePoint can be done using the Object Model (Sharepoint.dll)
• LINQ to SharePoint
• Intro into Extension Methods and Lamdba Expressions (shorter syntax for specifying delegate methods).
• How to write your own LINQ language such as LINQ to SharePoint which is really just mapping LINQ to CAML
• Can use LINQ to SharePoint in a SharePoint web part which uses the Object Model, but can ALSO do the same code but have it use the web services model by sending all the CAML to SharePoint at once and getting back result (using LINQ to SharePoint).
• Linq to SharePoint is not created by Microsoft, but is available on codeplex.
• Once LINQ to SharePoint is installed, there is a new type (template) that you can select in Visual Studio called LINQ to SharePoint File.
• LINQ to SharePoint is nice syntax and easy to work with and it is also very fast like CAML.

After watching this, I am excited to use it. I installed LINQ to SharePoint and I like it a lot. It is very nice. Be sure to use the newest example, don’t go to the samples page until you understand that SharePointDataSource is now a SharePointDataContext and the code is a little bit different. I did find out that it ONLY works on lists. It does NOT work on Shared Document Libraries, but it does work on Events and Tasks. Basically, I believe it works on anything that shows up under the List of a site. Also, from what I can tell, it only works on a particular web. This can be good and bad.

In the end, this will NOT give me a list of users or anything like that because it is not a list. It also doesn’t appear to support any kind of updating of data. I think I am back to the SharePoint web services for updating or getting users or document libs, etc. I think LINQ to SharePoint is a great tool for querying list data in SharePoint.

Migrating MOSS to BPOS (Microsoft SharePoint Online) –Part I

MOSS 2007 is also known as Microsoft Office SharePoint Server, and BPOS is also known as Business Productivity Online Suite, but more specifically Microsoft SharePoint Online since BPOS is actually more than just SharePoint. Every since the beginning Microsoft has never really known how to market / name SharePoint, and they still don’t IMHO, but that isn’t the point of this blog.

When you start Googling around you will most likely come across Metalogix tools. The two most notable are BPOS Commander and SharePoint Migration Manager. I have tried to get pricing or a demo on both of these products and the company has been 100% non-responsive other than an automated email that says an account rep will contact me shortly. It has been a while now, and no response, so I tried again. Still no response. I expect the product is priced like Oracle… “how much money do you have to spend with us” is the answer you will get when you read between the lines. With that said, the tooling looks WONDERFUL and should make the task much simpler than many other ways.

The big issues to consider are

• Mapping Users from current domain to BPOS users.
• Cleaning up users that are no longer with company
• Keeping Permissions
• Domain Groups and SharePoint groups
• User account activation since they are not active initially
• Hard coded Urls in things such as Wikis, Announcements, Tasks, HTML Content Areas, etc.
• Changing Site Hierarchy
• Identifying Owners of sites… Abandoned sites, SPS 2003 migrated sites so no Owner groups in some cases, etc
• Determining what sites are still used
• Converting custom Site Definitions that are basically the same as original Site Definitions to a standard Site Definition that BPOS will accept.

 

Given that I have not heard back from Metalogix and I don’t even know if we will have a budget for such a tool, I decided to start looking elsewhere. This brought me to the SharePoint Web Services 3.0 SDK which is a Windows Help file. It is much easier to use than the online version in my opinion. Though if you want just the docs for each of the web services, it is pretty good. Though, the downloaded versions has code samples for each web service, I find that VERY useful to get started. It does a good job of getting you started. There are two section of interest. Programming Tasks –> Programming Web Services for Windows SharePoint Services and Reference –> Web Services.

Two other good places to start are below. They do a good job of explaining what is available and how to start coding:

• Talk to SharePoint Through Its Web Services
• Getting Started With SharePoint Web Services

I wrote up an Example of how to use the SharePoint Web Services here.

In the past I used the SharePoint Object Model and .NET to develop migration scripts. I also did some direct database updates. The problem with these is that they require console (Server) access and I won’t have that with BPOS. So, the only option if I want to use the SP Object Model is to do the change before hand, then export the data to BPOS and hope for the best.

Since I won’t have console access, I think it makes sense to look at the web services route. The best part about this is that I can do the development (including debugging) on my laptop instead of the SharePoint server itself. The big downside is that the web services seem to be much more complicated to use. Everything seems to be XML based and a special schema just for querying. I would much prefer working with strongly typed objects and being able to use Intellisense.

One helpful thing is that InfoPath can be used. It appears dealing with the namespaces can be a bit tricky, but can be done and hopefully abstracted away. Here and here are posts on how to use XPath and SharePoint Web Services.

In the quest for being able to use strongly-typed objected I thought maybe someone created a LINQ to SharePoint provider. That would be great! As it turns out, someone did. It can be seen here. The problem is that it is ALPHA and appears to have died quite a while ago. The good news is that SharePoint 2010 has been released and it appears that they have included the LINQ to SharePoint Provider. Click here to see it. It looks similar, but I’m not sure if it is the same as the ALPHA one or not.

If you are not working with lists specifically then SharePoint Web Services are a good option.

Can’t access SQL Server Cluster or non-standard port or named instance after upgrading to Windows 7

After I migrated to Windows 7 we were not able to connect to the SQL Cluster using MS SQL Management Studio, Linqpad,Visual Studio, my apps, odbc, etc. The reason is that Windows firewall blocks non-default ports for SQL Server.

Determine if the Windows Firewall is the cause

The first thing you probably want to do is confirm that it is actually a Firewall issue. The easiest way (if you are lucky) is to just turn off the Windows Firewall. If you are not lucky enough to be able to turn off your firewall due to corporate policies then there is still hope. To help troubleshoot this, I created a rule in the Firewall that opened up all ports for all programs. Click here to see the details of how to do this.

Once you have determined that the firewall is the issue, you may want to open the port instead of opening it for individual programs or open all ports between your computer and the server you are trying to access.

Option 1: Open Firewall for server

This seems to be the easiest by far since you only have to do it once unless you change server ip addresses or ports.

To add a rule, go to the Start Menu | Administrative Tools | Windows Firewall with Advanced Security. Right-click the Inbound Rules on the left navigation area and then select New Rule…

Next choose Custom rule. Click the next button.

Leave all programs setting. Click the next button.

Leave Protocol type to Any. Click the next button.

Leave local IP addresses as Any. Add ALL IP addresses that you server has to the list of remote IP Addresses. I was surprised that I needed all the ip addresses for the server (use ipconfig from the command line if you don’t know them) and not just one, so to be safe add them all. Click the next button.

Make sure Allow the Connection radio button is check. Click the next button.

Choose only Domain (you may need to add private if you run into issues with vpn, etc). Click the next button.

Give the rule something descriptive.

NOTE: A slightly more secure version of this would be to specify TCP as the Protocol instead of Any. Copy the rule and edit it such that you change TCP to UDP. Then you’ll have one rule for TCP and one for UDP. This is the method I personally chose.

NOTE: You could do the same, but do it for you subnet, but that is a bit riskier from a security standpoint, though not bad on a corporate network. Just be certain to select domain (no public or private).

Option 2: Open Firewall for each program

To fix the issue, go to your Start menu | Control Panel | Windows Firewall | Allow a program or feature through Windows Firewall | Change Settings | Allow another program…

Next choose your program such as MS SQL Management Studio, Linq pad, Visual Studio, odbc, etc. Be sure to click the Network location types… button and select Home Network if you connect from home (and have selected that connection to be a home connection). FYI, to create an odbc connection using a non-standard port you will need to add the C:\Windows\System32\odbcad32.exe program.

Side note on MS Access: If you are trying to have your MS Access application access a linked server that uses ODBC to connect to the SQL Server you will need to create a new ODBC Data Source, but be sure NOT to make it a System DSN. You want to select User DSN otherwise MS Access will not be able to access it. Though, I imagine if you ran MS Access as administrator you could make it work that way with a System DSN.

I still had trouble when running in Visual Studio 2010 when debugging (worked fine if just ran without debugging). I never did figure out how to get past this. If anyone knows, please let me know.

Option 3: Hardcode server ip address

Workaround, if you can’t change your Firewall, you need admin rights on your machine I think. If you can’t do that, then you can always access the server using the ip address and port number separated by a comma. I don’t know why it works, but it does (at least on my machine).

 

Legal like warning: I am by no means a security or firewall guru. Please don’t take these as security recommendations. I simply figured out how to solve the problem while taking the security issues into consideration the best way I know how (I’m a programmer not a security expert). So, please don’t make these changes unless you understand the risks as you are messing with firewall rules that are meant to protect your computer from attack. You assume all responsibility for your changes.

Tips on How to install SQL Server 2008 R2

First this is NOT a step by step installation. I assume you can find the setup.exe and run it. I do address some of the less obvious issues that you may encounter.

BEFORE you run setup.exe: If you have Visual Studio 2008 installed be sure that you also have Service Pack 1 installed for VS2008 (Visual Studio 2008). Otherwise, you will have to stop the SQL Server 2008 R2 installation, go install SP1 for VS2008, then run the SQL Server 2008 R2 installer again.

Once you launch the setup.exe you will see the SQL Server Installation Center. Click the Installation navigation link on the left, and then select New installation or add features to an existing installation.

 

After a number of straight-forward screens you’ll come to this screen. This is the screen were you determine what you want to install. Basically the first 5 checkboxes (the ones under Instance Features) are for the server and the rest are for the most part for client, though they are “shared” so some of them are for server I assume also.

 

 

Check the items you want. I suggest the following.

 

NOTE: In the installation it will create a Integration Services (service). I guess it is needed even though I am not hosting it.

Visual Studio 2010 won’t debug under Windows 7

I found out that Visual Studio 2010 won’t let the debugger started until I run Visual Studio 2010 with Run as Administrator. I think I figured out why. Here is an account of what I was seeing and how I fixed it.

I could open my C# ASP.NET web application just fine using Run (Control-F5). Then I tried to do it with Debug (F5) and I got the following message:

Unable to launch the ASP.NET Development Server.Uncaught  exception thrown by method called through Reflection.

I then Googled that error message. I found this link. I go to the section 2.4.11.2 (shown below).

2.4.11.2 Debugger start fails if the IntelliTrace recordings directory is Read-Only

By default, IntelliTrace files are located in the "%ProgramData%\Microsoft Visual Studio\10.0\TraceDebugging\" directory.  If this directory is made read-only, or the directory is changed to a read-only location, an uncaught exception gets raised to the user.  "Error while trying to run project: Uncaught exception thrown by method called through Reflection."

To resolve this issue:

Option 1: Modify the permissions of the directory to be writable.

Option 2: Change the directory where IntelliTrace recordings are stored.

• ○ In Visual Studio, open the Options dialog box.
• ○ Expand the IntelliTrace item in the left pane of the dialog box.
• ○ Click the Advanced item under IntelliTrace.
• ○ Change the IntelliTrace recordings directory to one that is not read-only.
• ○ Click OK.

I tried both option 1 and Option 2. The problem is that when I tried Option 1 and went to the directory suggested, it was not there. When I tried Option 2 I got the following message where I was expecting to see some kind of useful UI:

An error occurred loading this property page.

Then I got the idea, what if Visual Studio 2010 needs to be run as administrator even though I am an admin under. I am knew to Windows 7, but I am quickly learning that most techie tasks require the programs to be Run As Administrator; Visual Studio is no exception to this. It seems that I am not the only one to come to this conclusion. Here Microsoft suggests changing Visual Studio 2010 to run as Administrator and I have shown instructions below:

If you are running Windows Vista, you must start Visual Studio by going to the Start menu and right clicking Visual Studio 2010 and selecting Run as Administrator. To always launch Visual Studio 2010 as an administrator you can create a short cut, right click the short cut, select properties, select the Compatibility tab, and check the Run this program as an administrator checkbox. When you start Visual Studio 2010 with this shortcut, it will always run as administrator.

The good news is that after to run VS2010 (Visual Studio 2010) as Administrator the proper directory structure will be created and the if you happen to run VS2010 without running it as Administrator you can now run as Debug (F5) and you can also go to the Options | IntelliTrace | Advanced and see a useful UI with no errors.

I think to keep things simple I will run VS2010 as administrator at all times.