Hide Columns in a SharePoint list based on SharePoint group

It seems to be a fairly common request to hide particular columns on a SharePoint form based on the SharePoint group that a user is a member of. One solution to this problem would be to create custom forms for Add, Edit, and Display in SharePoint using SharePoint designer. Then figure out a way to hide and show the columns. This requires SharePoint Designer access and may not always be an option. It also requires the user to read through ASP.NET / XSL markup and figure out how to do it. This is a very tedious task and hardly easy to maintain. The good news is that this is a truly secure solution since the data is not sent to the client (browser). If you have these strict security requirements then you may want to go this route. FYI, I have read that InfoPath is also an option, but I don't like going that route either.

I am working on an internal instance of SharePoint 2010 and simply hiding the data on the forms using CSS is sufficient because the user would have to look in the source of the page to see the data.

WARNING:
Note: This is NOT a true security solution! All data (even for hidden columns) is sent to the browser, but is just hidden from user's view in the browser.

In my case, it would not be the end of the world if users viewed source to see all the data. We are all part of the same company after all. If this is your scenario, then you may want to consider the solution outlined below.

My solution uses a JavaScript library called SPUtility.js. It makes hide and showing a field simple. It also makes changing a field to readonly simple as well. I can't take credit for the library, but I am very happy I found it. The only problem left to solve was how to get the current user and the groups that the user is in. This is something I had to come up with. It requires CSOM (Client Side Object Model) which requires SharePoint 2010 or later. I read that you can use SPServices if you are using MOSS 2007, but I have not tried to do it.

To implement the functionality to get user and group information, I wrote the following methods.

You can download the source here.

function getCurrentUserObject(callbackFunc)

function isCurrentUserInSharePointGroup(sharePointGroupID, callbackFunc)

function isUserInSharePointGroup(userLogin, sharePointGroupID, callbackFunc)

CSOM uses asynchronous calls for everything which makes it difficult to code if you are not accustomed to it. The good news is all you have to do is provide the callback function to make use of these functions. While that may sound difficult, just follow the example I have provided. In the example code the first thing it does is make sure the CSOM library (sp.js) is loaded and then executes our code. It takes the approach of getting the current user then seeing if the user is in the owner group and setting column visibility appropriately. If the user is not in the owner group it then looks in the members group and sets the column visibility appropriately. If they are not in either of those groups it sets the column visibility appropriately. You could continue on with different checks if desired.

Here is a snippet of what the solution code looks like:

function doOwnerChanges(userLogin, isInGroup)
{
  if (isInGroup)
  {
 
   SPUtility.GetSPField('Title').Show();
   SPUtility.GetSPField('Cost').Show();
   SPUtility.GetSPField('Customer Comment').Show();
   SPUtility.GetSPField('Assigned To').Show();
   SPUtility.GetSPField('Internal Comments').Show();
  }
  else // not in owner group let's see if they are in members group
  {
 
   isUserInSharePointGroup(userLogin, 7, Function.createDelegate(this, this.doMemberChanges));
 
  }
}

As you can see it is easy to maintain because all the complexity is abstracted away. If another column is added you would just add a line to the above code.

Added solution to your page

1. Download the following:
   • SPUserGroupUtility.js My code that gets the current user and checks the groups the user is in.
   • ShowHideColumnExample.js (only needed if you want an example of how to use this solution). You can replace or not use this in your specific implementation. This is the file you will need to specify your columns. The name is not important except that you reference it later.
   • SPUtility.js This provides the functionality for hide/showing fields and making them readonly, etc.
   • Prototype.js Required by SPUtility.js.
   • sp.js - actually you don't need to download it. Ultimately, this is the CSOM library. It is provided by SharePoint 2010. Just use it in your code as needed.
2. Upload the .js file to somewhere on SharePoint where all users can access the files. I recommend installing your .js file in your Site Assets directory on your site. You can find it by going to Site Action | View All Site Content | Site Assets.
3. Navigate to form (EditForm.aspx or NewForm.aspx)
   In SharePoint 2010, you can choose Form Web Parts -> Default whatever Form
4. Add a Content Editor Web Part to the page
5. Edit the web part
   In SharePoint 2010, click the arrow -> Edit Web Part.
6. Edit the content editor's HTML to add some JavaScript
   In SharePoint 2010, under Editing Tools -> Format Text click the HTML button -> Edit HTML Source
7. Follow the instructions on installing the SPUtility.js
8. <script src="/sites/someSite/SiteAssets/prototype.js" type="text/javascript"></script>
   <script src="/sites/someSite/SiteAssets/SPUtility.js" type="text/javascript"></script>
   <script src="/sites/someSite/SiteAssets/ShowHideColumnExample.js" type="text/javascript"></script>
   <script src="/sites/Brent/SiteAssets/SPUserGroupUtility.js" type="text/javascript"></script>
9. Save the page / stop editing.

Troubleshooting

• NOTE: To just try SPUtility.js you can try the install instructions.
• If nothing happens on the page, make sure you don't have any errors. You may need to do an F12 in IE to see the error console. Also, you can look at the F121 | Network tab to see if the JavaScript files are able to be found.
• You can always start the JavaScript debugger and see where it is breaking.

Uploading large files to SharePoint Online (O365)

It seems amazing to me that Microsoft did such a poor job of giving developers a good way to upload large documents to Microsoft SharePoint Online (aka Office 365 or SharePoint 365). Ideally I would like to use the

Microsoft.SharePoint.Client.File.SaveBinaryDirect

method that is part of the CSOM (Client Side Object Model), but this does not work with SharePoint Online and only seems to work on SharePoint 2010 (probably 2013, but have not tested). I did get

Microsoft.SharePoint.Client.File uploadedFile = docLib.RootFolder.Files.Add(newFileFromComputer);

to work on SharePoint Online, but it not very useful because it is limited to files of about 3MB in size.

After lots of trial and error, I figured out that the most reliable was to do it is using a standard PUT request and passing the Claims Authentication cookies that are required to make most any request to SharePoint Online. This works well unless you are debugging and you still have the default exception warning enabled. In that case, for large files that take more than 60 seconds to upload you will get a message similar to this:

The CLR has been unable to transition from COM context 0x1fe458 to COM context 0x1fe5c8 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.

Depending on what you are doing this may be okay. In my case, I don’t care if my command line application waits for a long request to continue. This is caused by using the STA threading attribute on the main method of the application.

Below are two methods (one just an overload that calls the other basically) that upload a document to a shared library in SharePoint Online. It should also work on SharePoint 2010 plus if you replace the ClaimClientContext with a standard ClientContext that is needed for SharePoint 2010+. I’ll leave that to you to try on your own for now. You’ll notice I have also added functionality to change the created and modified date of the files after they are uploaded. For details (and source code click here). Or if you just want to understand more about Claims based authentication, sample code, options, etc definitely check out here.Lastly, the keyValues dictionary is just a dictionary of actual internal field names (see CAML field names) as keys and the values for each field are the values of the dictionary.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint.Client;
using MSDN.Samples.ClaimsAuth;
using System.Xml.Linq;
using System.Net;
using System.IO;
...

public static void AddDocument(string fileToUpload, string webUrl, string docLibInternalName, string docLibUIName, string documentSetName, string filenameToSaveAs, DateTime? createdDate, DateTime? modifiedDate, Dictionary<string, object> keyValues, int timeoutInMilliseconds)
{
string urlToSaveAs = webUrl + "/" + docLibInternalName + "/" + documentSetName + "/" + filenameToSaveAs;
AddDocument(fileToUpload, webUrl, urlToSaveAs, createdDate, modifiedDate, keyValues, timeoutInMilliseconds);
}

// Uploads most any size file to SharePoint Online (O365) using Claims Authentication. It does NOT use CSOM, and instead uses a standard PUT request
// that has the cookies from the Claims based authentication added to it.
// This solution is based on http://stackoverflow.com/questions/15077305/uploading-large-files-to-sharepoint-365
// and
// To get the claims authentiation cookie, this solution requires: http://msdn.microsoft.com/en-us/library/hh147177.aspx#SPO_RA_Introduction
// or if you want to get the cookies for claims authentication antoher way, you can use
// http://www.wictorwilen.se/Post/How-to-do-active-authentication-to-Office-365-and-SharePoint-Online.aspx
public static void AddDocument(string fileToUpload, string webUrl, string urlToSaveAs, DateTime? createdDate, DateTime? modifiedDate, Dictionary<string, object> keyValues, int timeoutInMilliseconds)
{

//For example: byte[] data = System.IO.File.ReadAllBytes(@"C:\Users\me\Desktop\test.txt");
byte[] data = System.IO.File.ReadAllBytes(fileToUpload);

// get the cookies from the Claims based authentication and add it to the cookie container that we will then pass to the request
CookieCollection cookies = ClaimClientContext.GetAuthenticatedCookies(webUrl, 200, 200);
CookieContainer cookieContainer = new CookieContainer();
cookieContainer.Add(cookies);

// make a standard PUT request
System.Net.ServicePointManager.Expect100Continue = false;
HttpWebRequest request = HttpWebRequest.Create(urlToSaveAs) as HttpWebRequest;
request.Method = "PUT";
request.Accept = "*/*";
request.ContentType = "multipart/form-data; charset=utf-8";
request.CookieContainer = cookieContainer;
request.AllowAutoRedirect = false;
request.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)";
request.Headers.Add("Accept-Language", "en-us");
request.Headers.Add("Translate", "F");
request.Headers.Add("Cache-Control", "no-cache");
request.ContentLength = data.Length;
request.ReadWriteTimeout = timeoutInMilliseconds;
request.Timeout = timeoutInMilliseconds;

using (Stream req = request.GetRequestStream())
{
    req.ReadTimeout = timeoutInMilliseconds;
    req.WriteTimeout = timeoutInMilliseconds;
    req.Write(data, 0, data.Length);
}

// get the response back
HttpWebResponse response = null;
try
{
    response = (HttpWebResponse)request.GetResponse();
    Stream res = response.GetResponseStream();
    using (StreamReader rdr = new StreamReader(res))
    {
        string rawResponse = rdr.ReadToEnd();
       
    }
}
catch (Exception ex)
{
    throw ex;
}
finally
{
    if (response != null)
    {
        response.Close();
    }
}

// NOTE: the file that was uploaded is still checked out and must be checked in before it will be available to others.
// The method includes a checkin command. If the method below is removed for some reason,
// a checkin method call should be added here so that the file will be available to all (that have access).
// NOTE: The method calls add keyValues passed in as well. These would need to be done also if the method is removed.
UriBuilder urlBuilder = new UriBuilder(urlToSaveAs);
string serverRelativeUrlToSaveAs = urlBuilder.Path;
ChangeCreatedModifiedInfo(webUrl, serverRelativeUrlToSaveAs, createdDate, modifiedDate, keyValues);

}

 

…

Programmatically finding and using a person using Client Side Object Model

I have a custom table and one of the fields is a Person column column called Answered By. In the SharePoint UI the person can type in the persons username (same as email if you are using SharePoint Online) or use the Person Picker to find the person. In the code below we want to find them by username.

using (ClientContext clientContext = ClaimClientContext.GetAuthenticatedContext("https://myco.sharepoint.com/site1/site2"))
{
    var user = clientContext.Web.EnsureUser("me@myco.com");
}

Once you have the person you can then use it when setting the value for that field programmatically. In this example I am creating a new list item and setting the Answered By field using the person object that I find using the EnsureUser() method.

using (ClientContext clientContext = ClaimClientContext.GetAuthenticatedContext("https://myco.sharepoint.com/site1/site2"))
    {
        var destList = clientContext.Web.Lists.GetByTitle("FAQs");
        var itemCreateInfo = new ListItemCreationInformation();
        var newItem = destList.AddItem(itemCreateInfo);
        newItem["Title"] = "Some title here";
        var user = clientContext.Web.EnsureUser(brent.vermilion@nxp.com);
        newItem["Answered_x0020_By"] = user;
        newItem.Update();
        // actually create the records in the destination list
        clientContext.ExecuteQuery();
    }

Take note, I could have also put the integer instead of the user object, but I think this actually saves a trip to the database and is cleaner. I also found it interesting that spaces in column names in SharePoint have the space replaced by the space with a notation that uses the hex value for space (20).

To learn more about CRUD operation and other useful topics see this intro.

Best tool for CAML query generation

After lots of looking around, I finally found a CAML query generation tool that supports SharePoint Online authentication and Client Side Object Model api. It also works with SharePoint 2010. It is well designed, easy to use, and very powerful. At the click of a button it gives you the applicable Server Object Model code or Client Side Object Model code or even Web Service code. Unfortunately, the generated code does not work with SharePoint Online, but that is because it uses different authentication.However take a look at my post on using authenticating against SharePoint Online when using the Client Side Object Model.

Thank you to Karine Bosch for writing the CAML Designer and making it freely available AND giving good documentation on it.

Click here to learn more and download.

Getting Started with SharePoint Online Development

Authenticating against SharePoint Online

The first place I would start is here to learn how to authenticate against SharePoint Online (O365). This will give you a good understand of what libraries are available to do the claims based authentication SharePoint Online uses. It also explains it in an easy to understand manner. Remote Authentication in SharePoint Online Using Claims-Based Authentication – This is a article, but really what you want is the code / solution you can use to connect to your SharePoint Online site and get its title. Here is the direct link to download the solution for Visual Studio. You can also check out this library instead. They both do similar authentication. These libraries are great because they make authenticating against SharePoint Online which is a MAJOR pain in the rear, much easier.

If you are doing your development on a machine that does not have SharePoint 2010 installed you will need the SharePoint Foundation 2010 Client Object Model Redistributable. This gives you among other things the Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll assemblies which you will need. If you have an installation of SharePoint 2010 on another machine you can copy these files from the c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI directory. If you have trouble, you may want to look here first for a code tweak that brings up the login screen. I did not find it necessary to do the tweak, but some people do.

As it turns out the SharePoint Online requires a bit more work to get the authentication to work than the standard example for SharePoint 2010 show. This is because SharePoint Online uses Claims-Based Authentication. For whatever reason it is not implemented and certainly is not transparent in the SharePoint Client Object Model. What I recommend is downloading the solution from above if you have not already done so since this has the Claims-Based Authentication implemented for you. You should still be able to use most samples for SharePoint 2010 regarding the CSOM (SharePoint Client Object Model), but you will need to do two things to make them work (which is related to authentication).

1. Add a reference to the ClaimsAuth (that is from the solution above).
2. In the code replace any calls to new ClientContext(url) with ClaimClientContext.GetAuthenticatedContext(url);

Once you are authenticated, the api should be the same.

Technology Overview

The SharePoint Foundation provides an api to interact with SharePoint (both SharePoint 2010 and SharePoint Online) data remotely from script that executes in the browser, Silverlight, or .NET applications. The api includes a subset of the features available in the SharePoint Object Model (Microsoft.SharePoint.dll) that you would use if you were running your code on the SharePoint Server itself (which is not an option if you are using Microsoft SharePoint Online). Each language uses the same api to make switching between them easier. Obviously, the syntax is different for each language.

Microsoft decided to make the client object model instead of constantly trying to extend the web services. The reason is that with the client object model it is very similar to the server object model that was available on the SharePoint server itself. This makes providing new features much easier and also easier for developers since there is one object model needed for both client and server (in many cases).

You can do most things you can do with the server object model. The following areas are supported by the client object model:

• Site Collections and Sites
• Lists, List Items, Views, and List Schemas
• Files and Folders
• Web, List, and List Item Property Bags
• Web Parts
• Security
• Content Types
• Site Templates and Site Collection Operations

This has some advantages over the web services that Microsoft used to require developers to use. The biggest advantage may be that it does not have the overhead associated with web services (xml bloat).

A good resource is the Client Object Model Resource Center.

A great video to get more details.

IMPORTANT: It only works with .NET 3.5 right now. It does NOT work with a .NET 4.0 project.

Do your first project.

1. Create a new .NET Command Line application (or other type if you prefer) using the 3.5 (4.0 is the default and will not work as of 2012-07-26)
2. Make sure you compiled the ClaimsAuth project that is located in the Remote Authentication in SharePoint Online Using the Client Object Model solution and reference the ClaimsAuth.dll, or just add that project to your project if you prefer to have the source code.
3. Add a reference to the Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll to your project as well. Remember, you can get that from SharePoint Foundation 2010 Client Object Model Redistributable installation or from c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI on an existing SharePoint 2010 installation.
4. Add [STAThread] attribute to the Main method.
5. Make your Main method look like this (you’ll need to change the url)
   [STAThread]
   static void Main(string[] args)
   {
       using (ClientContext clientContext = ClaimClientContext.GetAuthenticatedContext(https://yoursite.sharepoint.com/teams/site1/site2/etc))
       {
           Web site = clientContext.Web;
           clientContext.Load(site); // tell model we want that data
           clientContext.ExecuteQuery(); // actually go to SharePoint and get data
           Console.WriteLine("Title: {0}", site.Title);
       }
   }
6. I have highlighted the most important pieces that are different from what you would do in SharePoint 2010. When you run it you will see a browser object popup and then continue. That is all there is to it.

Where to go from here

Using the SharePoint Foundation 2010 Managed Client Object Model – good tutorial that covers lots of the CRUD type operations you may need to do.

You may prefer to use LINQ instead of CAML to do your queries. This may be a good place to start.