How to add temporary columns to a List View using CSR, in order to display dynamic information not stored in the list.

I have found it useful sometimes to use the CSR framework to display extra information in a List View that isn't necessarily stored on each list item.  This is especially true of values that are related to the list item but may be dynamic or conditional in some way, and it makes more sense to evaluate those conditions using JavaScript at render time than to use something like, say, a Calculated Column, because a Calculated Column is only re-calculated when the item is updated.

My original approach to implementing this kind of thing was to add a column to the list as a place-holder for where my dynamic value would be rendered, and then link the CSR override script to that field using the field's JSLink property.

However, at the recent SharePointFest DC conference, I was at a talk about CSR, and it was mentioned that, from a data integrity standpoint, that is a bad practice because you are essentially adding a field to the list in which you are never going to store any data.  It was also mentioned that you could probably just add a field directly in the view you do your rendering on.

I thought that sounded reasonable, and was an interesting approach, so I played around with the idea for a little bit and came up with this general pattern for doing just that.

And instead of attaching to the JSLink property of a field, these scripts are intended to be added directly to the list Views through the List View Web Part's JSLink property (which is exposed in the web part's Editor pane).

During my experimentation, I was trying to add the temporary column in OnPreRender, but still register a field override for that field in the normal manner.  That turned out to be problematic, so eventually I settled on using OnPreRender to add the temporary field to the view, and using OnPostRender to do all the calculations and render the final HTML.

Here's a code snippet that outlines the basic pattern, which I have commented heavily so you can follow along:

1. var DEC = DEC || {};
2.  
3. DEC.FakeField = (function () {
4.  
5.     function addTemporaryField(ctx) {
6.         // make sure we haven't added the field already,
7.         // in case OnPreRender fires twice, which it does sometimes
8.         if (ctx.ListSchema.Field.filter(function (fld) {
9.             return fld.Name == 'FakeField';
10.         }).length == 0) {
11.  
12.             // create the field schema object to insert
13.             var FakeFieldObj = {
14.                 AllowGridEditing: 'FALSE',
15.                 DisplayName: 'Fake Field',
16.                 RealFieldName: 'FakeField',
17.                 Name: 'FakeField',
18.                 FieldType: 'Text',
19.                 Type: 'Text',
20.                 Filterable: 'FALSE',
21.                 Sortable: 'FALSE',
22.                 ReadOnly: 'TRUE',
23.             };
24.  
25.             // find the index of the field to insert next to,
26.             // based on that field's DISPLAY name.
27.             var insertIdx = ctx.ListSchema.Field.map(function (fld) {
28.                 return fld.DisplayName;
29.             }).indexOf('Some Field Name');
30.  
31.             // if you want to insert *before* the field do not add anything
32.             // but, if you want to insert *after* the field, add 1
33.             insertIdx++;
34.  
35.             // insert your fake field schema object into the list schema
36.             ctx.ListSchema.Field.splice(insertIdx, 0, FakeFieldObj);
37.         }
38.     }
39.  
40.     function renderFakeFieldValues(ctx) {
41.         // get the index of the fake field column in the table
42.         // by looking for the header with the DISPLAY NAME of our fake field,
43.         // so we know where to insert the final rendered value for each item
44.         var colIndex = 0;
45.         var headers = document.querySelectorAll('.ms-listviewtable th');
46.         var len = headers.length;
47.         for (var idx = 0; idx < len; idx++) {
48.             if (headers[idx].innerHTML.indexOf('Fake Field') != -1) {
49.                 colIndex = idx;
50.             }
51.         };
52.  
53.         // go through the rows and do whatever calculations we need
54.         // to get the value we want to render in the added field.
55.         // at this point ctx.ListData.Row represents a collection
56.         // of list items for all the rows currently visible in the view.
57.         ctx.ListData.Row.forEach(function (listItem) {
58.  
59.             // here, listItem will have all the data of all
60.             // the columns visible in the view, which we
61.             // can access by listItem.FieldInternalName
62.  
63.             // we can then use those values to do our conditional formatting
64.             // and calculate / generate the value to render in the fake field
65.             var helloWorldHtml = '<span>Hello ' + listItem.WorldField + '</span>';
66.  
67.             // find the row in the table that corresponds to this list item
68.             var iid = GenerateIIDForListItem(ctx, listItem);
69.             var row = document.getElementById(iid);
70.  
71.             // insert the value we want in the fake field column
72.             // of the row representing this list item
73.             row.children[colIndex].innerHTML = helloWorldHtml;
74.         });
75.     }
76.  
77.     return {
78.         render: function () {
79.             SPClientTemplates.TemplateManager.RegisterTemplateOverrides({
80.                 OnPreRender: addTemporaryField,
81.                 OnPostRender: renderFakeFieldValues
82.             });
83.         }
84.     }
85. })();
86.  
87. RegisterModuleInit(SPClientTemplates.Utility.ReplaceUrlTokens("~site/SiteAssets/Scripts/AddFakeField.js"), DEC.FakeField.render);
88. DEC.FakeField.render();

So where is this useful?  Well, let's say you want to highlight list items that are overdue.  Sure, you could come up with a rendering override for the due date field itself, and either render the text in red, or add a red background, or something like that.  But what if you wanted to display a message to the users that spelled out exactly how many days overdue the item was, like "XX days overdue".

The following code snippet shows how to do exactly that:

1. var DEC = DEC || {};
2.  
3. DEC.DaysOverdueField = (function () {
4.  
5.     function addTemporaryField(ctx) {
6.  
7.         if (ctx.ListSchema.Field.filter(function (fld) {
8.             return fld.Name == 'DaysOverdueField';
9.         }).length == 0) {
10.  
11.             var daysOverdueFieldObj = {
12.                 AllowGridEditing: 'FALSE',
13.                 DisplayName: 'Days Overdue',
14.                 RealFieldName: 'DaysOverdueField',
15.                 Name: 'DaysOverdueField',
16.                 FieldType: 'Text',
17.                 Type: 'Text',
18.                 Filterable: 'FALSE',
19.                 Sortable: 'FALSE',
20.                 ReadOnly: 'TRUE',
21.             };
22.  
23.             // in this case i am going to show the "XX days overdue"
24.             // message directly next to the due date field
25.             var insertIdx = ctx.ListSchema.Field.map(function (fld) {
26.                 return fld.DisplayName;
27.             }).indexOf('Due Date');
28.  
29.             // insert *after* the due date field
30.             insertIdx++;
31.  
32.             ctx.ListSchema.Field.splice(insertIdx, 0, daysOverdueFieldObj);
33.         }
34.     }
35.  
36.     function calculateOverdueDays(ctx) {
37.         // get the index of the "Days Overdue" column in the table
38.         var colIndex = 0;
39.         var headers = document.querySelectorAll('.ms-listviewtable th');
40.         var len = headers.length;
41.         for (var idx = 0; idx < len; idx++) {
42.             if (headers[idx].innerHTML.indexOf('Days Overdue') != -1) {
43.                 colIndex = idx;
44.             }
45.         };
46.  
47.         // go through the rows and display an overdue message if it is overdue
48.         ctx.ListData.Row.forEach(function (listItem) {
49.             // get the due date
50.             var duedate = new Date(listItem.DueDate);
51.  
52.             // use the built in GetDaysAfterToday function to calculate
53.             // how many days overdue an item is.  if it is truly overdue,
54.             // this function will return a negative number.
55.             var overdueDays = GetDaysAfterToday(duedate);
56.             if (overdueDays < 0) {
57.  
58.                 // change the negative to a positive
59.                 overdueDays = Math.abs(overdueDays);
60.  
61.                 // find the row in the table that corresponds to this list item
62.                 var iid = GenerateIIDForListItem(ctx, listItem);
63.                 var row = document.getElementById(iid);
64.  
65.                 // insert the overdue message in the <td> element in our temporary "Days Overdue" column
66.                 row.children[colIndex].innerHTML = '<span style="color:red;">' + overdueDays + ' days overdue</span>';
67.             }
68.         });
69.     }
70.  
71.     return {
72.         render: function () {
73.             SPClientTemplates.TemplateManager.RegisterTemplateOverrides({
74.                 OnPreRender: addTemporaryField,
75.                 OnPostRender: calculateOverdueDays
76.             });
77.         }
78.     }
79. })();
80.  
81. RegisterModuleInit(SPClientTemplates.Utility.ReplaceUrlTokens("~site/SiteAssets/Scripts/ShowDaysOverdue.js"), DEC.DaysOverdueField.render);
82. DEC.DaysOverdueField.render();

So, that is how you can add a temporary column to a list view, and then access it to add dynamic information related to the list items into the view.  As requested, here is a screen shot of the "days overdue" code in action.  The column "Days Overdue" does not exist on this list:

In a subsequent post I will show how to use this technique to do something a little more complex: display file counts next to folders in a Document Library view that includes all files within all subfolders within any particular folder.

A Better Pattern for Client Side Rendering Scripts

Recently I did some work for a client that required combining multiple List View Web Parts on one page, and using Client Side Rendering (CSR) to apply custom styling to each of the the various views. The site was a Team Site that had Minimal Download Strategy (MDS) enabled, and this led to some interesting challenges. The MDS feature is supposed to speed up browsing by having the client only process the differences between the current page and the new page. However, this can lead to situations where custom CSR scripts do not get executed correctly.

In a nutshell, MDS registers JavaScript files as it downloads/executes them, and on a subsequent page load, if the request calls for the same file, the MDS engine will check the list, see that it already has the script, and not download/execute it a second time. Thus, the CSR code is not run and the custom rendering is not applied.

There is a way around this. SharePoint provides a function called RegisterModuleInit() which you can use to tell MDS that a particular function from a particular file should always be executed. RegisterModuleInit() takes two arguments – the path of the file that has the function to execute, and the name of the function. Now, let’s take a look at one of the popular patterns for constructing CSR template overrides, which happens to be the one I used up until this project:

1. (function () {
2.     var overrideCtx = {};
3.     overrideCtx.Templates = {};
4.     overrideCtx.Templates.Item = customItem;
5.  
6.     SPClientTemplates.TemplateManager.RegisterTemplateOverrides(overrideCtx);
7. })();
8.  
9. function customItem(ctx) {
10.     var itemHtml = "";
11.     // do whatever processing is necessary
12.     // here to generate the custom html
13.     return itemHtml;
14. }

We can see two things here:

1. The template overrides are defined and registered with the template manager in an anonymous self-executing function, thus there is no named entry point to register with RegisterModuleInit(). In order to use RegisterModuleInit() to get the code functioning properly with MDS, the template override code needs to be changed to have a named entry point.

2. The customItem() function is defined in the global namespace. Keep in mind, for this project I was working with several template override files being loaded on the same page, many with multiple rendering override functions defined. If not handled correctly, the global namespace was going to get polluted.

So, in order to fix both of these problems, I came up with the following pattern, which I now use for all of my CSR template override scripts:

1. var DEC = DEC || {};
2.  
3. DEC.thisOverride = (function () {
4.     function customItem (ctx) {
5.         var itemHtml = "";
6.         // do whatever processing is necessary
7.         // here to generate the custom html
8.         return itemHtml;
9.     }
10.  
11.     return {
12.         render: function () {
13.             SPClientTemplates.TemplateManager.RegisterTemplateOverrides({
14.                 Templates: {
15.                     Item: customItem
16.                 }
17.             });
18.         }
19.     }
20. })();
21.  
22. // register for MDS enabled sites
23. RegisterModuleInit(SPClientTemplates.Utility.ReplaceUrlTokens("~site/SiteAssets/Scripts/myOverrideScript.js"), DEC.thisOverride.render);
24.  
25. // fallback for non-MDS enabled sites
26. DEC.thisOverride.render();

Let’s look at what’s going on in the code.

First, I register (or retrieve) my own namespace, so I can stay organized (line 1).

Second, I add a property thisOverride (line 3). I named it generically for this example, but the name of this property should reflect the specific SharePoint asset you are applying the override to. That way many overrides can co-exist on the same page within the custom namespace (e.g. DEC.calendarOverride, DEC.contactsOverride, DEC.customFieldXOverride, DEC.customListYOverride).

I then use the revealing module pattern to create a closure in which I contain all the functions I might need for the custom rendering. By keeping those functions private, I can still use the same internal naming convention for the actual override functions (e.g. customItem, render, etc.).  The only publicly exposed member is the render function which registers the overrides with the template manager, and becomes the named entry point.

Lastly, I register the entry point function with the MDS system (line 23), and provide a fallback for non-MDS enabled sites (line 26), ensuring that the template overrides will get applied no matter what.

Notice also that the SPClientTemplates.Utility namespace offers a function to replace URL tokens, so you can use tokens when defining the path to your override file. This allows for much greater flexibility when including the CSR script as part of a feature that may be activated on a number of sites.

So there you have it – a CSR template override pattern that is flexible enough to work on MDS enabled or disabled sites, while maintaining responsible use of the global namespace.

Bigger editing area for the Script Editor Web Part

This is not new or revelatory, but I needed to jot it down somewhere I can find it when I forget.

When working with the Script Editor Web Part, you can get a bigger text box by pasting this in as the first thing:

<style>
    .ms-rte-embeddialog-textarea {
        width: 1000px;
        height: 500px;
    }
</style>


It gets applied to the editing dialog fairly quickly, so you have a much easier time adding script snippets.

Hiding form fields using CSR without jQuery

A recent question on SharePoint.SE about the best way to hide fields on forms got me thinking.  Many people know about how to hide fields on forms using jQuery, by using some selector that gets you to the field's label on the form, and then hiding the containing <tr> to hide both the label and the field's control:

$('.ms-formtable nobr:contains("Field Display Name")').closest('tr').hide();

And this article shows how to apply that same jQuery technique using client side rendering / template overrides.  But I wondered, is there a way to hide fields on forms using just what you have available during CSR code execution?  Is there a way to do it without needing jQuery?

Indeed there is:

(function () {

    var overrideCtx = {};
    overrideCtx.Templates = {};
    overrideCtx.Templates.Fields = {
        "Field1": {
            "NewForm": hideField,
            "EditForm": hideField
        },
        "Field2": {
            "NewForm": hideField,
            "EditForm": hideField
        }
    };

    SPClientTemplates.TemplateManager.RegisterTemplateOverrides(overrideCtx);
})();

// the hideField function uses ctx.CurrentFieldSchema.Name,  
// so it is generic enough to be used for multiple fields

function hideField(ctx) {

    // get the span that represents the field's actual edit control
    var span = $get(ctx.FormUniqueId + ctx.FormContext.listAttributes.Id + ctx.CurrentFieldSchema.Name);

    // still need to go up two levels to hide the entire <tr>
    span.parentNode.parentNode.setAttribute("style", "display:none");
    return "";
}

I guess it just goes to show you how much you can actually do with CSR.

Interesting code usements, and structure: Usement 1 - safely allowing unsafe updates

This is intended to be an ongoing series where I post code patterns that I find interesting.  Vets may find what I post here to be old hat, and newbs may find something useful.  Regardless, these are patterns that I have personally found to be interesting, for whatever reason.  If you have a helpful comment that will elaborate on what's here and educate me, bring it on.  My ultimate goal is to become a better coder.

The first code pattern in this series has to do with SharePoint, and the SPWeb.AllowUnsafeUpdates property.

AllowUnsafeUpdates = true.  We've all done it, despite the fact that it's there to protect us from cross-site scripting attacks.  There are just times when you want to update something and AllowUnsafeUpdates is set to false.  It happens.  And you need to get around it.

EDIT: Scott Brickey (see comments below) wanted me to remind all you SharePointers out there that changing the AllowUnsafeUpdates is only necessary when updating data in an HTTP GET request.  Any other time you are updating data, you should be able to do it without changing AllowUnsafeUpdates.

This is a pattern that leaves me feeling a little more reassured about protecting the safety of the SPweb object during an "unsafe" update.  The trick is to put your updating code in a try{} block, and whether or not you attempt to catch any exceptions, use a finally{} block to reset AllowUnsafeUpdates to false.  That way, if something goes wrong, at least your web object is re-protected.

EDIT: Thanks to an anonymous commenter for pointing out that the AllowUnsafeUpdates value does not persist outside the scope of the instance of SPWeb that you're working on.  So if you are not doing anything else with the SPWeb after you finish your updates, you don't need to set AllowUnsafeUpdates = false.

Here's an example:

using (SPSite site = new SPSite("http://asharepointsite"))

using (SPWeb web = site.OpenWeb())

{

    // do most of the work out here

    try

    {

        web.AllowUnsafeUpdates = true;

        // do the minimum amount of work that

        // necessitated allow unsafe updates

    }

    finally

    {

        web.AllowUnsafeUpdates = false;

    }

    // If you are still going to use your

    // SPWeb object here and do more work,

    // you want to use that finally above to

    // set AllowUnsafeUpdates to false.

    // But, if you are done with the object and

    // are going to let the SPWeb go out of scope

    // and get disposed, you don't need to worry

    // about setting it to false.

}

(Because of the using block, the web and site objects will be disposed of properly even if an exception is thrown, so you don't need to worry about that.)

How to successfully publish custom Document Set content types - Part 2: It's like butter.

(For a recap of the situation and the problems I encountered, refer to the previous post - Part 1: I got 99 problems...)

So there I was, desperately needing to set Inherits="TRUE" on my custom Doc Set content type definitions so that I could get them to publish through a Content Type Hub, and knowing that the consequence of that setting would be losing all my custom XML Documents. What to do? I had already written all the definitions in CAML/XML, and I sure didn't want to re-do all that work in C# in a feature receiver -- I had over 40 definitions! Hard-coding the construction of all of those CTypes would be a nightmare.

Well, the answer was staring me right in the face. I already had the XML. Even if I set Inherits="TRUE" and caused SharePoint to conveniently ignore all my work, it didn't actually erase my work.  SharePoint would build the CType definitions in the site level Content Type collection without using my code, but once my feature got activated, the Elements.xml files that contained my definitions would be deployed to the 14 hive.  Therefore, they would be accessible in SharePoint. And I knew all the parts would be in place by the time a feature receiver FeatureActivated() method got fired, so all I had to do would be to grab the files with my custom CType definitions, cycle through them and copy the XML Documents from the Elements.xml files into the definitions in the site.

So I started preparing an event receiver to do just that.  And I was getting ready to reach right into the 14 hive and pull my files out, but let's face it, that would be kind of ugly.  Luckily, as I was poking around the internet looking for ideas on how to pull this whole thing off, I ran across this great post by Stefan Stanev, which has one particularly beautiful (albeit long) line of LINQ.  Lo and behold, with a little tweaking, I could pull my CType definitions right out of the feature receiver's properties, and never have to touch the file system!  I was sold.

Here, now, is my event receiver.  Comments provided to help explain things step by step.

using System;

using System.Xml;

using System.Xml.Linq;

using System.Linq;

using System.Runtime.InteropServices;

using System.Security.Permissions;

using Microsoft.SharePoint;

using Microsoft.SharePoint.Security;

using Microsoft.SharePoint.Utilities;

using System.Collections.Generic;

using Microsoft.SharePoint.Administration;

using System.Globalization;

using System.IO;

namespace My.ContentTypes.Features.MyCTypeDefinitions

{

    [Guid("a4cc1ae2-d1c6-4e7a-bef2-da1c26a0c96f")]

    public class MyCTypeDefinitionsEventReceiver : SPFeatureReceiver

    {

        // the id of the description field, so we can set it to HIDDEN later

        protected readonly String _docSetDescFieldID = "CBB92DA4-FD46-4C7D-AF6C-3128C2A5576E";

        public override void FeatureActivated(SPFeatureReceiverProperties properties)

        {

            using (SPSite site = properties.Feature.Parent as SPSite)

            {

                // pull all the content type definitions from our elements.xml

                // files as XElements, and put them in a list

                List<XElement> cTypeDefElements =

                    properties.Definition.GetElementDefinitions(CultureInfo.GetCultureInfo((int)site.RootWeb.Language)).Cast<SPElementDefinition>()

                    .Select(def => XElement.Parse(def.XmlDefinition.OuterXml))

                    .Where(ctdefel => ctdefel.Name.LocalName == "ContentType" && ctdefel.Attribute("ID").Value.Contains("0x0120D520"))

                    .ToList();

                // iterate through all my content type definitions

                foreach (XElement myCTypeDef in cTypeDefElements)

                {

                    // apply my XML docs to the definition on the site

                    CopyXMLDocsToSite(myCTypeDef, site);

                }

                site.RootWeb.Update();

            }

        }

        private void CopyXMLDocsToSite(XElement myCTypeDef, SPSite site)

        {

            // get all the XmlDocuments from my custom content type definition

            List<XElement> myXmlDocs = myCTypeDef.Descendants().Where(doc => doc.Parent.Name.LocalName == "XmlDocument").ToList();

            // get the actual content type from the site

            SPContentTypeId cTypeID = new SPContentTypeId(myCTypeDef.Attribute("ID").Value.ToString());

            SPContentType installedCType = site.RootWeb.ContentTypes[cTypeID];

            // get the existing XmlDocuments in the installed definition on the site

            // as XElements, and add them to a list

            List<XElement> installedDocs = installedCType.XmlDocuments.Cast<string>().Select(elem => XElement.Parse(elem)).ToList();

            // iterate through the ones we are customizing and add them

            foreach (XElement docToAdd in myXmlDocs)

            {

                // find the doc we want to replace

                XElement docToDelete = installedDocs.Find(doc => doc.Name.LocalName.Equals(docToAdd.Name.LocalName));

                // if there is one, delete it from the collection

                if (docToDelete != null)

                    installedCType.XmlDocuments.Delete(docToDelete.Name.NamespaceName);

                // if it is the receivers, insert ours into the existing "Receivers" doc

                // so we keep the default, inherited ones as well

                if (docToAdd.Name.LocalName == "Receivers" && docToDelete != null)

                {

                    AddNodesIntoExistingXElement(docToAdd, docToDelete, "Receiver");

                    SaveXElementAsXMLDocument(docToDelete, installedCType);

                }

                else

                {

                    // otherwise, just clean our definition and add it

                    CleanXElement(docToAdd);

                    SaveXElementAsXMLDocument(docToAdd, installedCType);

                }

            }

            // while we have a reference to the content type definition from the site,

            // change any other default field settings

            ChangeInheritedFieldSettings(installedCType);

            // write the changes into the database

            installedCType.Update(true);

        }

        private void AddNodesIntoExistingXElement(XElement xElementToAdd, XElement existingXElement, string nodeName)

        {

            // extract the nodes

            List<XElement> nodesToAdd = xElementToAdd.Descendants().Where(node => node.Name.LocalName == nodeName).ToList();

            // add into existing xdoc

            foreach (XElement node in nodesToAdd)

            {

                existingXElement.Add(node);

            }

        }

        private void CleanXElement(XElement xElement)

        {

            // check for attributes

            List<XAttribute> atts = null;

            if (xElement.HasAttributes)

                atts = xElement.Attributes().ToList();

            // replace the guts of the XElement using descendants

            // so we get rid of comments and only keep useful XML

            xElement.ReplaceAll(xElement.Descendants());

            // re-add the attributes if there are any

            if (atts != null)

                xElement.ReplaceAttributes(atts);

        }

        private void SaveXElementAsXMLDocument(XElement xElement, SPContentType installedCType)

        {

            // turn the XElement into an XmlDocument

            XmlDocument newDoc = new XmlDocument();

            newDoc.Load(xElement.CreateReader());

            // add it back to the content type definition on the site

            installedCType.XmlDocuments.Add(newDoc);

        }

        private void ChangeInheritedFieldSettings(SPContentType installedCType)

        {

            // get the installed content type definition schema

            // so we can access the default fields that were

            // inherited from the parent content type

            XElement cTypeSchema = XElement.Parse(installedCType.SchemaXmlWithResourceTokens);

            // set the "Document Set Description" field to be Hidden

            cTypeSchema.Descendants().Single(node => node.Name.LocalName == "FieldRef" && node.Attribute("ID")

                                     .Value.ToUpper().Contains(_docSetDescFieldID.ToUpper()))

                                     .SetAttributeValue("Hidden", "TRUE");

            /*

             *  make any other changes to inherited fields here

             */

            // save the edited schema back into the site

            installedCType.SchemaXmlWithResourceTokens = cTypeSchema.ToString(SaveOptions.DisableFormatting);

        }

    }

}

A couple more comments about what's going on there -- I had more than just the Doc Set content types in the feature, but none of the other content types had XML Documents, so I was only concerned with my custom types that derived from Doc Set. Which is why I only pull the content type defs that have an ID that contains the base ID for Doc Set. And I knew that I didn't want any default values for any of the XML Documents, which is why I delete the installed XML Documents and just replace them with mine, except for the Event Receivers, where I did want to keep the default XML Docs, so in that case I just add mine in to what's already there. Also, I had to use the ReplaceAll() method to get rid of comments that I had in my definitions, because ultimately pushing comments into the CType defs on the site did not work well. However, ReplaceAll() also got rid of the attributes, which led to a couple null reference exceptions when SharePoint choked on certain XML Docs not having the LastModified attribute.  So I had to save the attributes and re-add them after using ReplaceAll().  And finally, even though I tried to set a particular default field to be Hidden in the FieldRefs section, having Inherits="TRUE" means that SharePoint ignores that customization as well, so I had to set that attribute through code.

And with that, I have my custom XML Documents, and Inherits="TRUE", which ultimately makes publishing the content types smooth like butter.

EDIT (18 Dec 2012) : Decided to refactor the code a little bit to try and push it in a slightly more flexible, usable direction.  And for some, uh, other, personal reasons as well.  In any case, this code is not fully tested.  I did deploy and activate it once in debug to make sure it didn't choke on anything, and it ran fine.  So I'm assuming that means everything went well and if I were to publish the CTypes and then start using them, everything would work smoothly.  But, that's just an assumption.  So if you decide to borrow some of this code, just know that it has changed from when I originally wrote the post, and be ready for a bug or two.  (I'd like to hear about them if they pop up, so I can keep improving on it.)

How to successfully publish custom Document Set content types - Part 1: I got 99 problems...

I recently worked on a project where we had a large number of custom content types, all based on Document Set, and we needed to publish them through a Content Type Hub. Sounds simple, right? As it turns out, it's not so straightforward.

If you've ever tried to create a custom Document Set content type through code you may have run into the problem described here -- if you do what you really should do when creating a derivative content type and set Inherits="TRUE", you end up losing all your custom XML documents. For those who don't know, let's take a look at the ramifications of that little problem.  Here's a sample custom Doc Set definition the way we would want to make it:

<?xml version="1.0" encoding="utf-8" ?>

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <!-- Parent ContentType: Document Set (0x0120D520) -->

  <ContentType ID="0x0120D52000E295900502E84310BD8863E5E3468033"

               Name="My Custom Document Set"

               Group="My Custom Content Types"

               Description="A custom content type derived from Document Set"

               Inherits="TRUE"

               ProgId="SharePoint.DocumentSet"

               Version="0">

    <Folder TargetName="_cts/My Custom Document Set" />

    <FieldRefs>

      <FieldRef ID="{CBB92DA4-FD46-4C7D-AF6C-3128C2A5576E}" Name="DocumentSetDescription" Hidden="TRUE" />

      <FieldRef ID="{BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218}" Name="CustomField1" DisplayName="Custom Field 1" Required="TRUE" />

      <FieldRef ID="{6692F1B7-8087-4E65-B509-C819D694FAED}" Name="CustomField2" DisplayName="Custom Field 2" Required="FALSE"/>

      <FieldRef ID="{7EC6256B-F521-4D43-B346-FE010478DDCF}" Name="CustomDateField" DisplayName="Custom Date Field" Required="TRUE" Format="DateOnly" />

    </FieldRefs>

    <XmlDocuments>

      <!-- List of all fields [site columns] shared between all content types and the document set. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/sharedfields">

        <sf:SharedFields xmlns:sf="http://schemas.microsoft.com/office/documentsets/sharedfields" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add shared fields here using the syntax below-->

          <!--<SharedField id="00000000-0000-0000-0000-000000000000" />-->

          <SharedField id="BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218" Name="CustomField1" />

          <SharedField id="6692F1B7-8087-4E65-B509-C819D694FAED" Name="CustomField2" />

          <SharedField id="7EC6256B-F521-4D43-B346-FE010478DDCF" Name="CustomDateField" />

        </sf:SharedFields>

      </XmlDocument>

      <!-- Add Event Receivers-->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/events">

        <Receivers xmlns:spe="http://schemas.microsoft.com/sharepoint/events">

          <Receiver>

            <Name>MyCustomItemAddedEventReceiver</Name>

            <Type>ItemAdded</Type>

            <SequenceNumber>10001</SequenceNumber>

            <Assembly>MyCustomCode.MyEventReceivers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=756a9f17c5fe9a3b</Assembly>

            <Class>MyCustomCode.MyEventReceivers.ItemAddedEventReceiver.ItemAddedEventReceiver</Class>

            <Data></Data>

            <Filter></Filter>

          </Receiver>

        </Receivers>

      </XmlDocument>

      <!-- List of all content types that are allowed in the document set. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/allowedcontenttypes">

        <act:AllowedContentTypes xmlns:act="http://schemas.microsoft.com/office/documentsets/allowedcontenttypes" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add content types that will be used in the document set using the syntax below -->

          <!--<AllowedContentType id="00000000-0000-0000-0000-000000000000" />-->

          <AllowedContentType id="0x0101"/>

          <AllowedContentType id="0x01010069928169381543AC927453E7A865E300"/>

        </act:AllowedContentTypes>

      </XmlDocument>

      <!-- List of all fields [site columns] that should appear on welcome page. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/welcomepagefields">

        <wpFields:WelcomePageFields xmlns:wpFields="http://schemas.microsoft.com/office/documentsets/welcomepagefields" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add welcome fields here using the syntax below -->

          <!--<WelcomePageField id="00000000-0000-0000-0000-000000000000" />-->      

          <WelcomePageField id="BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218" Name="CustomField1" />

          <WelcomePageField id="6692F1B7-8087-4E65-B509-C819D694FAED" Name="CustomField2" />

          <WelcomePageField id="7EC6256B-F521-4D43-B346-FE010478DDCF" Name="CustomDateField" />

        </wpFields:WelcomePageFields>

      </XmlDocument>

      <!-- List of all default documents associated with the content types. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/defaultdocuments">

        <dd:DefaultDocuments xmlns:dd="http://schemas.microsoft.com/office/documentsets/defaultdocuments" AddSetName="TRUE" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add default documents using the syntax below -->

          <!--<DefaultDocument name="Sample Document.docx" idContentType="0x0101" />-->

        </dd:DefaultDocuments>

      </XmlDocument>

      <!-- Documents used (OOB) -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">

        <FormUrls xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">

          <New>_layouts/NewDocSet.aspx</New>

        </FormUrls>

      </XmlDocument>

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">

        <FormTemplates  xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">

          <Display>ListForm</Display>

          <Edit>ListForm</Edit>

          <New>DocSetDisplayForm</New>

        </FormTemplates>

      </XmlDocument>

    </XmlDocuments>

  </ContentType>

</Elements>

Now take a look at the XML Documents section... it's huge! If we lose our customization there, we lose the key things that make a Doc Set a Doc Set: the shared fields, the welcome page fields, the default documents, etc. Plus, if we have any custom event receivers or forms, we lose those too. That's quite a lot to lose for trying to do things correctly.

It would seem that the easy solution is to just set Inherits="FALSE", which means you then have to remember to explicitly add into the definition all the stuff you should be inheriting, like the default Doc Set event receivers.  (Which, truth be told, I did not do, and because of that ended up stumbling into my solution for all these problems, which I will describe in the next post.)  But at least all your customization remains intact. And indeed that does work if you are only deploying to one site. But what if you need to syndicate your Doc Sets through content type publishing? Well, then you might run into another problem, which shows up in the content type publishing error logs on the subscribing sites after the publishing doesn't work:

Content type 'My Custom Document Set' cannot be published to this site because feature 'MyContentTypeDefinitionsFeature' is not enabled.

Wait, what? I need to enable the feature that has my content type definitions on the site I'm trying to publish to? If I need to do that, why am I even bothering with the publishing? When I first got this error, I was left scratching my head a bit, but went ahead and obliged SharePoint and enabled the feature.  And then the publishing worked... sometimes.  And sometimes it wouldn't, and there seemed to be no rhyme or reason to when it would and when it wouldn't.  The content type publishing error logs only reported an "unknown error."  So I started digging a little deeper and cracked open the ULS logs, where I found this error:

Unable to locate the xml-definition for CType with SPContentTypeId <Content Type ID>

And to top that off, in the cases where the publishing would actually succeed, I started getting yet another problem -- anytime anyone would try to create a new Doc Set based on one of our custom types, it would fail with an error stating that the content type definition was read-only. Well of course it's read-only, it's been syndicated, that's exactly what's supposed to happen! But why is that causing a problem? If I'm getting a read-only error, that implies that SharePoint is trying to modify the content type definition itself.  Why?

I then proceeded to google the google out of Google.

After a while, I finally found a few things that helped me paint a picture of what was happening.  A very speculative picture, no doubt. I won't claim that what I thought I understood about the problems is anywhere near accurate. But, in some strange way, it made sense to me, and eventually led me to figuring out a real solution to the problem.

First, I found this question and answer on StackExchange about needing to enable the feature that held the CType definitions when trying to publish. Someone was having the same problem I was, and the answer made sense to me.  Because I had set Inherits="FALSE", SharePoint couldn't pull the XML Schema from the original definition on the Hub (or from the parent CType), but it did know what feature held the Schema, so it demanded that the feature be activated so it could access the Schema there.

Then I found this blog post about the "unable to locate the xml-definition" error in the ULS logs. It doesn't offer much information about the problem itself, but it does have one little gem that I latched on to -- the author has "seen these errors happen in environments where deployed package [sic] was deployed before." In a certain sense, during the publishing process, SharePoint is trying to deploy a content type definition to a subscribing site by pulling it from the Hub.  However, to get past the previous error, we've enabled the feature containing the content type definition on the subscribing site already, and therefore have "deployed the package before," so to speak. Now, maybe what's happening is not that SharePoint can't find the XML definition for the content type -- it's actually finding two, and can't decide which is the right one to use.

The final piece of the puzzle (third strike? last straw?) came when I cracked open a reflector and started to trace through what happens when a new Doc Set is created.  As it turns out, as part of the creation process, the DocumentSet.Provision() method is called, which will cause SharePoint to check to see if the content type definition has the default Doc Set event receivers.  If it doesn't, even though you are technically creating the Doc Set based off a list level CType definition, SharePoint will reach up to the parent definition at the site level and try to provision the event receivers there. And, since my site level definitions were read-only because they were syndicated through content type publishing... I would get a read-only error. Because (as I said earlier) I had neglected to include the default Doc Set event receivers when I originally set Inherits="FALSE".

So what it all boiled down to at this point was that I absolutely had to get things working with Inherits="TRUE". It would give me the default Doc Set event receivers which would get rid of my read-only error.  And most importantly, it would allow SharePoint to know where to pull the Schema from and enable successful publishing, while only activating the feature on the one site it should be activated on - the Content Type Hub. But...what to do about inevitably losing the custom XML Documents? I had already written all that CAML, I really didn't want to have to write some crazy huge feature receiver just to re-do all that work in code.

As it turned out, the answer was right in front of my face.

To read about what it was, and how I got everything working, stay tuned for part 2...