How to successfully publish custom Document Set content types - Part 2: It's like butter.

(For a recap of the situation and the problems I encountered, refer to the previous post - Part 1: I got 99 problems...)

So there I was, desperately needing to set Inherits="TRUE" on my custom Doc Set content type definitions so that I could get them to publish through a Content Type Hub, and knowing that the consequence of that setting would be losing all my custom XML Documents. What to do? I had already written all the definitions in CAML/XML, and I sure didn't want to re-do all that work in C# in a feature receiver -- I had over 40 definitions! Hard-coding the construction of all of those CTypes would be a nightmare.

Well, the answer was staring me right in the face. I already had the XML. Even if I set Inherits="TRUE" and caused SharePoint to conveniently ignore all my work, it didn't actually erase my work.  SharePoint would build the CType definitions in the site level Content Type collection without using my code, but once my feature got activated, the Elements.xml files that contained my definitions would be deployed to the 14 hive.  Therefore, they would be accessible in SharePoint. And I knew all the parts would be in place by the time a feature receiver FeatureActivated() method got fired, so all I had to do would be to grab the files with my custom CType definitions, cycle through them and copy the XML Documents from the Elements.xml files into the definitions in the site.

So I started preparing an event receiver to do just that.  And I was getting ready to reach right into the 14 hive and pull my files out, but let's face it, that would be kind of ugly.  Luckily, as I was poking around the internet looking for ideas on how to pull this whole thing off, I ran across this great post by Stefan Stanev, which has one particularly beautiful (albeit long) line of LINQ.  Lo and behold, with a little tweaking, I could pull my CType definitions right out of the feature receiver's properties, and never have to touch the file system!  I was sold.

Here, now, is my event receiver.  Comments provided to help explain things step by step.

using System;

using System.Xml;

using System.Xml.Linq;

using System.Linq;

using System.Runtime.InteropServices;

using System.Security.Permissions;

using Microsoft.SharePoint;

using Microsoft.SharePoint.Security;

using Microsoft.SharePoint.Utilities;

using System.Collections.Generic;

using Microsoft.SharePoint.Administration;

using System.Globalization;

using System.IO;

namespace My.ContentTypes.Features.MyCTypeDefinitions

{

    [Guid("a4cc1ae2-d1c6-4e7a-bef2-da1c26a0c96f")]

    public class MyCTypeDefinitionsEventReceiver : SPFeatureReceiver

    {

        // the id of the description field, so we can set it to HIDDEN later

        protected readonly String _docSetDescFieldID = "CBB92DA4-FD46-4C7D-AF6C-3128C2A5576E";

        public override void FeatureActivated(SPFeatureReceiverProperties properties)

        {

            using (SPSite site = properties.Feature.Parent as SPSite)

            {

                // pull all the content type definitions from our elements.xml

                // files as XElements, and put them in a list

                List<XElement> cTypeDefElements =

                    properties.Definition.GetElementDefinitions(CultureInfo.GetCultureInfo((int)site.RootWeb.Language)).Cast<SPElementDefinition>()

                    .Select(def => XElement.Parse(def.XmlDefinition.OuterXml))

                    .Where(ctdefel => ctdefel.Name.LocalName == "ContentType" && ctdefel.Attribute("ID").Value.Contains("0x0120D520"))

                    .ToList();

                // iterate through all my content type definitions

                foreach (XElement myCTypeDef in cTypeDefElements)

                {

                    // apply my XML docs to the definition on the site

                    CopyXMLDocsToSite(myCTypeDef, site);

                }

                site.RootWeb.Update();

            }

        }

        private void CopyXMLDocsToSite(XElement myCTypeDef, SPSite site)

        {

            // get all the XmlDocuments from my custom content type definition

            List<XElement> myXmlDocs = myCTypeDef.Descendants().Where(doc => doc.Parent.Name.LocalName == "XmlDocument").ToList();

            // get the actual content type from the site

            SPContentTypeId cTypeID = new SPContentTypeId(myCTypeDef.Attribute("ID").Value.ToString());

            SPContentType installedCType = site.RootWeb.ContentTypes[cTypeID];

            // get the existing XmlDocuments in the installed definition on the site

            // as XElements, and add them to a list

            List<XElement> installedDocs = installedCType.XmlDocuments.Cast<string>().Select(elem => XElement.Parse(elem)).ToList();

            // iterate through the ones we are customizing and add them

            foreach (XElement docToAdd in myXmlDocs)

            {

                // find the doc we want to replace

                XElement docToDelete = installedDocs.Find(doc => doc.Name.LocalName.Equals(docToAdd.Name.LocalName));

                // if there is one, delete it from the collection

                if (docToDelete != null)

                    installedCType.XmlDocuments.Delete(docToDelete.Name.NamespaceName);

                // if it is the receivers, insert ours into the existing "Receivers" doc

                // so we keep the default, inherited ones as well

                if (docToAdd.Name.LocalName == "Receivers" && docToDelete != null)

                {

                    AddNodesIntoExistingXElement(docToAdd, docToDelete, "Receiver");

                    SaveXElementAsXMLDocument(docToDelete, installedCType);

                }

                else

                {

                    // otherwise, just clean our definition and add it

                    CleanXElement(docToAdd);

                    SaveXElementAsXMLDocument(docToAdd, installedCType);

                }

            }

            // while we have a reference to the content type definition from the site,

            // change any other default field settings

            ChangeInheritedFieldSettings(installedCType);

            // write the changes into the database

            installedCType.Update(true);

        }

        private void AddNodesIntoExistingXElement(XElement xElementToAdd, XElement existingXElement, string nodeName)

        {

            // extract the nodes

            List<XElement> nodesToAdd = xElementToAdd.Descendants().Where(node => node.Name.LocalName == nodeName).ToList();

            // add into existing xdoc

            foreach (XElement node in nodesToAdd)

            {

                existingXElement.Add(node);

            }

        }

        private void CleanXElement(XElement xElement)

        {

            // check for attributes

            List<XAttribute> atts = null;

            if (xElement.HasAttributes)

                atts = xElement.Attributes().ToList();

            // replace the guts of the XElement using descendants

            // so we get rid of comments and only keep useful XML

            xElement.ReplaceAll(xElement.Descendants());

            // re-add the attributes if there are any

            if (atts != null)

                xElement.ReplaceAttributes(atts);

        }

        private void SaveXElementAsXMLDocument(XElement xElement, SPContentType installedCType)

        {

            // turn the XElement into an XmlDocument

            XmlDocument newDoc = new XmlDocument();

            newDoc.Load(xElement.CreateReader());

            // add it back to the content type definition on the site

            installedCType.XmlDocuments.Add(newDoc);

        }

        private void ChangeInheritedFieldSettings(SPContentType installedCType)

        {

            // get the installed content type definition schema

            // so we can access the default fields that were

            // inherited from the parent content type

            XElement cTypeSchema = XElement.Parse(installedCType.SchemaXmlWithResourceTokens);

            // set the "Document Set Description" field to be Hidden

            cTypeSchema.Descendants().Single(node => node.Name.LocalName == "FieldRef" && node.Attribute("ID")

                                     .Value.ToUpper().Contains(_docSetDescFieldID.ToUpper()))

                                     .SetAttributeValue("Hidden", "TRUE");

            /*

             *  make any other changes to inherited fields here

             */

            // save the edited schema back into the site

            installedCType.SchemaXmlWithResourceTokens = cTypeSchema.ToString(SaveOptions.DisableFormatting);

        }

    }

}

A couple more comments about what's going on there -- I had more than just the Doc Set content types in the feature, but none of the other content types had XML Documents, so I was only concerned with my custom types that derived from Doc Set. Which is why I only pull the content type defs that have an ID that contains the base ID for Doc Set. And I knew that I didn't want any default values for any of the XML Documents, which is why I delete the installed XML Documents and just replace them with mine, except for the Event Receivers, where I did want to keep the default XML Docs, so in that case I just add mine in to what's already there. Also, I had to use the ReplaceAll() method to get rid of comments that I had in my definitions, because ultimately pushing comments into the CType defs on the site did not work well. However, ReplaceAll() also got rid of the attributes, which led to a couple null reference exceptions when SharePoint choked on certain XML Docs not having the LastModified attribute.  So I had to save the attributes and re-add them after using ReplaceAll().  And finally, even though I tried to set a particular default field to be Hidden in the FieldRefs section, having Inherits="TRUE" means that SharePoint ignores that customization as well, so I had to set that attribute through code.

And with that, I have my custom XML Documents, and Inherits="TRUE", which ultimately makes publishing the content types smooth like butter.

EDIT (18 Dec 2012) : Decided to refactor the code a little bit to try and push it in a slightly more flexible, usable direction.  And for some, uh, other, personal reasons as well.  In any case, this code is not fully tested.  I did deploy and activate it once in debug to make sure it didn't choke on anything, and it ran fine.  So I'm assuming that means everything went well and if I were to publish the CTypes and then start using them, everything would work smoothly.  But, that's just an assumption.  So if you decide to borrow some of this code, just know that it has changed from when I originally wrote the post, and be ready for a bug or two.  (I'd like to hear about them if they pop up, so I can keep improving on it.)

How to successfully publish custom Document Set content types - Part 1: I got 99 problems...

I recently worked on a project where we had a large number of custom content types, all based on Document Set, and we needed to publish them through a Content Type Hub. Sounds simple, right? As it turns out, it's not so straightforward.

If you've ever tried to create a custom Document Set content type through code you may have run into the problem described here -- if you do what you really should do when creating a derivative content type and set Inherits="TRUE", you end up losing all your custom XML documents. For those who don't know, let's take a look at the ramifications of that little problem.  Here's a sample custom Doc Set definition the way we would want to make it:

<?xml version="1.0" encoding="utf-8" ?>

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <!-- Parent ContentType: Document Set (0x0120D520) -->

  <ContentType ID="0x0120D52000E295900502E84310BD8863E5E3468033"

               Name="My Custom Document Set"

               Group="My Custom Content Types"

               Description="A custom content type derived from Document Set"

               Inherits="TRUE"

               ProgId="SharePoint.DocumentSet"

               Version="0">

    <Folder TargetName="_cts/My Custom Document Set" />

    <FieldRefs>

      <FieldRef ID="{CBB92DA4-FD46-4C7D-AF6C-3128C2A5576E}" Name="DocumentSetDescription" Hidden="TRUE" />

      <FieldRef ID="{BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218}" Name="CustomField1" DisplayName="Custom Field 1" Required="TRUE" />

      <FieldRef ID="{6692F1B7-8087-4E65-B509-C819D694FAED}" Name="CustomField2" DisplayName="Custom Field 2" Required="FALSE"/>

      <FieldRef ID="{7EC6256B-F521-4D43-B346-FE010478DDCF}" Name="CustomDateField" DisplayName="Custom Date Field" Required="TRUE" Format="DateOnly" />

    </FieldRefs>

    <XmlDocuments>

      <!-- List of all fields [site columns] shared between all content types and the document set. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/sharedfields">

        <sf:SharedFields xmlns:sf="http://schemas.microsoft.com/office/documentsets/sharedfields" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add shared fields here using the syntax below-->

          <!--<SharedField id="00000000-0000-0000-0000-000000000000" />-->

          <SharedField id="BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218" Name="CustomField1" />

          <SharedField id="6692F1B7-8087-4E65-B509-C819D694FAED" Name="CustomField2" />

          <SharedField id="7EC6256B-F521-4D43-B346-FE010478DDCF" Name="CustomDateField" />

        </sf:SharedFields>

      </XmlDocument>

      <!-- Add Event Receivers-->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/events">

        <Receivers xmlns:spe="http://schemas.microsoft.com/sharepoint/events">

          <Receiver>

            <Name>MyCustomItemAddedEventReceiver</Name>

            <Type>ItemAdded</Type>

            <SequenceNumber>10001</SequenceNumber>

            <Assembly>MyCustomCode.MyEventReceivers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=756a9f17c5fe9a3b</Assembly>

            <Class>MyCustomCode.MyEventReceivers.ItemAddedEventReceiver.ItemAddedEventReceiver</Class>

            <Data></Data>

            <Filter></Filter>

          </Receiver>

        </Receivers>

      </XmlDocument>

      <!-- List of all content types that are allowed in the document set. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/allowedcontenttypes">

        <act:AllowedContentTypes xmlns:act="http://schemas.microsoft.com/office/documentsets/allowedcontenttypes" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add content types that will be used in the document set using the syntax below -->

          <!--<AllowedContentType id="00000000-0000-0000-0000-000000000000" />-->

          <AllowedContentType id="0x0101"/>

          <AllowedContentType id="0x01010069928169381543AC927453E7A865E300"/>

        </act:AllowedContentTypes>

      </XmlDocument>

      <!-- List of all fields [site columns] that should appear on welcome page. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/welcomepagefields">

        <wpFields:WelcomePageFields xmlns:wpFields="http://schemas.microsoft.com/office/documentsets/welcomepagefields" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add welcome fields here using the syntax below -->

          <!--<WelcomePageField id="00000000-0000-0000-0000-000000000000" />-->      

          <WelcomePageField id="BCD93B9E-9DFF-4AE1-BC0E-607D3ACC9218" Name="CustomField1" />

          <WelcomePageField id="6692F1B7-8087-4E65-B509-C819D694FAED" Name="CustomField2" />

          <WelcomePageField id="7EC6256B-F521-4D43-B346-FE010478DDCF" Name="CustomDateField" />

        </wpFields:WelcomePageFields>

      </XmlDocument>

      <!-- List of all default documents associated with the content types. -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/office/documentsets/defaultdocuments">

        <dd:DefaultDocuments xmlns:dd="http://schemas.microsoft.com/office/documentsets/defaultdocuments" AddSetName="TRUE" LastModified="1/1/2010 08:00:00 AM">

          <!-- Add default documents using the syntax below -->

          <!--<DefaultDocument name="Sample Document.docx" idContentType="0x0101" />-->

        </dd:DefaultDocuments>

      </XmlDocument>

      <!-- Documents used (OOB) -->

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">

        <FormUrls xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">

          <New>_layouts/NewDocSet.aspx</New>

        </FormUrls>

      </XmlDocument>

      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">

        <FormTemplates  xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms">

          <Display>ListForm</Display>

          <Edit>ListForm</Edit>

          <New>DocSetDisplayForm</New>

        </FormTemplates>

      </XmlDocument>

    </XmlDocuments>

  </ContentType>

</Elements>

Now take a look at the XML Documents section... it's huge! If we lose our customization there, we lose the key things that make a Doc Set a Doc Set: the shared fields, the welcome page fields, the default documents, etc. Plus, if we have any custom event receivers or forms, we lose those too. That's quite a lot to lose for trying to do things correctly.

It would seem that the easy solution is to just set Inherits="FALSE", which means you then have to remember to explicitly add into the definition all the stuff you should be inheriting, like the default Doc Set event receivers.  (Which, truth be told, I did not do, and because of that ended up stumbling into my solution for all these problems, which I will describe in the next post.)  But at least all your customization remains intact. And indeed that does work if you are only deploying to one site. But what if you need to syndicate your Doc Sets through content type publishing? Well, then you might run into another problem, which shows up in the content type publishing error logs on the subscribing sites after the publishing doesn't work:

Content type 'My Custom Document Set' cannot be published to this site because feature 'MyContentTypeDefinitionsFeature' is not enabled.

Wait, what? I need to enable the feature that has my content type definitions on the site I'm trying to publish to? If I need to do that, why am I even bothering with the publishing? When I first got this error, I was left scratching my head a bit, but went ahead and obliged SharePoint and enabled the feature.  And then the publishing worked... sometimes.  And sometimes it wouldn't, and there seemed to be no rhyme or reason to when it would and when it wouldn't.  The content type publishing error logs only reported an "unknown error."  So I started digging a little deeper and cracked open the ULS logs, where I found this error:

Unable to locate the xml-definition for CType with SPContentTypeId <Content Type ID>

And to top that off, in the cases where the publishing would actually succeed, I started getting yet another problem -- anytime anyone would try to create a new Doc Set based on one of our custom types, it would fail with an error stating that the content type definition was read-only. Well of course it's read-only, it's been syndicated, that's exactly what's supposed to happen! But why is that causing a problem? If I'm getting a read-only error, that implies that SharePoint is trying to modify the content type definition itself.  Why?

I then proceeded to google the google out of Google.

After a while, I finally found a few things that helped me paint a picture of what was happening.  A very speculative picture, no doubt. I won't claim that what I thought I understood about the problems is anywhere near accurate. But, in some strange way, it made sense to me, and eventually led me to figuring out a real solution to the problem.

First, I found this question and answer on StackExchange about needing to enable the feature that held the CType definitions when trying to publish. Someone was having the same problem I was, and the answer made sense to me.  Because I had set Inherits="FALSE", SharePoint couldn't pull the XML Schema from the original definition on the Hub (or from the parent CType), but it did know what feature held the Schema, so it demanded that the feature be activated so it could access the Schema there.

Then I found this blog post about the "unable to locate the xml-definition" error in the ULS logs. It doesn't offer much information about the problem itself, but it does have one little gem that I latched on to -- the author has "seen these errors happen in environments where deployed package [sic] was deployed before." In a certain sense, during the publishing process, SharePoint is trying to deploy a content type definition to a subscribing site by pulling it from the Hub.  However, to get past the previous error, we've enabled the feature containing the content type definition on the subscribing site already, and therefore have "deployed the package before," so to speak. Now, maybe what's happening is not that SharePoint can't find the XML definition for the content type -- it's actually finding two, and can't decide which is the right one to use.

The final piece of the puzzle (third strike? last straw?) came when I cracked open a reflector and started to trace through what happens when a new Doc Set is created.  As it turns out, as part of the creation process, the DocumentSet.Provision() method is called, which will cause SharePoint to check to see if the content type definition has the default Doc Set event receivers.  If it doesn't, even though you are technically creating the Doc Set based off a list level CType definition, SharePoint will reach up to the parent definition at the site level and try to provision the event receivers there. And, since my site level definitions were read-only because they were syndicated through content type publishing... I would get a read-only error. Because (as I said earlier) I had neglected to include the default Doc Set event receivers when I originally set Inherits="FALSE".

So what it all boiled down to at this point was that I absolutely had to get things working with Inherits="TRUE". It would give me the default Doc Set event receivers which would get rid of my read-only error.  And most importantly, it would allow SharePoint to know where to pull the Schema from and enable successful publishing, while only activating the feature on the one site it should be activated on - the Content Type Hub. But...what to do about inevitably losing the custom XML Documents? I had already written all that CAML, I really didn't want to have to write some crazy huge feature receiver just to re-do all that work in code.

As it turned out, the answer was right in front of my face.

To read about what it was, and how I got everything working, stay tuned for part 2...