SharePoint Add-In Governance

I’ve been working with a client that recently asked me about Governance of SharePoint Provider-Hosted Add-ins in their on-premises SharePoint 2013. Essentially, they wanted to take control over how site owners installed add-ins from the corporate SharePoint App Catalog. The App Catalog allows administrators to toggle whether site owners can install add-ins or request permission to install them, but my client was looking for a more granular solution on a site-by-site basis.

My thought process went in the direction of detecting installation of an add-in and finding a way to intercept the installation process. This led me to the App Installed event that SharePoint supports for provider-hosted add-ins. Imagine a scenario where every add-in in the catalog fired an App Installed event that then checked the add-in against a central database/list etc., to determine whether the site owner could complete the installation. Seems simple enough. However, what if I then said administrators could upload provider-hosted add-ins to the catalog that might not have App Installed event code present? Perhaps a third-party add-in or add-in developed by another person outside the governance circle.

I wondered if it’d be possible to inject logic into existing add-ins after they’d been deployed to the app catalog. Turns out this isn’t as hard as it sounds. For those of you thinking I’m about to go with a code hack and a suggestion that could compromise compiled code, just hang in a moment longer.

Provider-Hosted Add-ins consist of an APP file in the app catalog, which redirects execution to an endpoint on another sever. The APP file is really just a glorified ZIP file with AppManifest.xml and other supporting files contained. Provider-Hosted add-ins that support the App Installed event (and Upgrading and Uninstalling events) include an endpoint reference to a remote event receiver web service in the AppManifest.xml. My theory was that I could download the APP file, unzip it, make the change to the AppManifest.xml file, zip it again, and upload it back to the catalog. The new modified APP file would contain a remote event receiver location of my choosing. So, I started work…

First to illustrate what I mean, here’s a snapshot of a test Provider-Hosted Add-in:

<?xml version="1.0" encoding="utf-8" ?>
<!--Created:cb85b80c-f585-40ff-8bfc-12ff4d0e34a9-->
<App xmlns="http://schemas.microsoft.com/sharepoint/2012/app/manifest"      Name="HelloWorld"      ProductID="{316a9436-6358-4626-a007-c31fafe306a2}"      Version="1.0.0.0"      SharePointMinVersion="15.0.0.0" >
  <Properties>
    <Title>Hello World</Title>
    <StartPage>~remoteAppUrl/Pages/Default.aspx?{StandardTokens}</StartPage>
    <InstalledEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</InstalledEventEndpoint>
    <UninstallingEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</UninstallingEventEndpoint>
    <UpgradedEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</UpgradedEventEndpoint>
  </Properties>

  <AppPrincipal>
    <RemoteWebApplication ClientId="*" />
  </AppPrincipal>
  <AppPermissionRequests AllowAppOnlyPolicy="false">
  </AppPermissionRequests>
</App>

Next, I needed some plumbing that would register event receivers on the App Catalog to manipulate add-ins uploaded to the catalog. In keeping with good SharePoint 2013 development, I created a Provider-Hosted Add-in for this purpose. I called this PH add-in the App Catalog Updater, ACU for short.

Because the ACU needs to make changes to the App Catalog, this add-in requires tenant control and the ability to run under the app-only context (as opposed to app and user credentials). This prevents this add-in from ever going into the Marketplace, but for my purpose this didn’t matter.

The following is the AppManifest.xml for the ACU:

<?xml version="1.0" encoding="utf-8" ?>
<!--Created:cb85b80c-f585-40ff-8bfc-12ff4d0e34a9-->
<App xmlns="http://schemas.microsoft.com/sharepoint/2012/app/manifest"      Name="SPAppsUpdateAppCat"      ProductID="{316a9436-6358-4626-a007-c31fafe306a2}"      Version="1.0.0.0"      SharePointMinVersion="15.0.0.0" >
  <Properties>
    <Title>SPApps.UpdateAppCat</Title>
    <StartPage>~remoteAppUrl/Pages/Default.aspx?{StandardTokens}</StartPage>
    <InstalledEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</InstalledEventEndpoint>
    <UninstallingEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</UninstallingEventEndpoint>
    <UpgradedEventEndpoint>~remoteAppUrl/Services/AppEventReceiver.svc</UpgradedEventEndpoint>
  </Properties>

  <AppPrincipal>
    <RemoteWebApplication ClientId="*" />
  </AppPrincipal>
  <AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
  </AppPermissionRequests>
</App>

As mentioned, the ACU registers remote events on the App Catalog, which I achieve, using the following code:

using System;
using Microsoft.SharePoint.Client.EventReceivers;

namespace SPApps.UpdateAppCatWeb.Services
{
    public class AppEventReceiver : IRemoteEventService
    {
        ///

<summary>
        /// Handles app events that occur after the app is installed or upgraded, or when app is being uninstalled.
        /// </summary>


        /// <param name="properties">Holds information about the app event.</param>
        /// <returns>Holds information returned from the app event.</returns>
        public SPRemoteEventResult ProcessEvent(SPRemoteEventProperties properties)
        {
            var result = new SPRemoteEventResult();
            Logger.Logger.LogInfo("ProcessEvent called for AppEventReceiver", () =>
            {
                using (var clientContext = TokenHelper.CreateAppEventClientContext(properties, false))
                {
                    switch (properties.EventType)
                    {
                        case SPRemoteEventType.AppInstalled:
                            // Remove any old RER first.
                            AppHelper.UnregisterRemoteEvents(clientContext);
                            // Install a RER for the App Catalog.
                            AppHelper.RegisterRemoteEvents(clientContext);
                            // Iterate existing apps and process them.
                            AppHelper.ProcessAppList(clientContext);
                            break;
                        case SPRemoteEventType.AppUninstalling:
                            // Remove RER from the App Catalog.
                            AppHelper.UnregisterRemoteEvents(clientContext);
                            break;
                    }
                }
            });
            return result;
        }

        ///

<summary>
        /// This method is a required placeholder, but is not used by app events.
        /// </summary>


        /// <param name="properties">Unused.</param>
        public void ProcessOneWayEvent(SPRemoteEventProperties properties)
        {
            throw new NotImplementedException();
        }

    }
}

The above code resides in the App Installed event receiver code for the ACU. This code executes when the ACU is installed the first time, and then calls a handy App Helper class to register the event receivers on the App Catalog. In addition, the code looks for existing add-ins in the catalog for processing, again using the App Helper.

Let’s take a look at the App Helper class:

using Microsoft.SharePoint.Client;
using System;
using System.Xml;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.IO.Compression;
using System.ServiceModel;

namespace SPApps.UpdateAppCatWeb
{
    static class AppHelper
    {
        private const string LISTNAME = "Apps for SharePoint";
        private const string RERNAME = "Apps_Remote_Event_Receiver";

        public static void RegisterRemoteEvents(ClientContext clientContext)
        {
            if (null == clientContext) throw new ArgumentNullException("clientContext");
            try
            {
                // Get the Apps List.
                Logger.Logger.LogInfo("Registering remote events", () =>
                {
                    var appCat = clientContext.Web.Lists.GetByTitle(LISTNAME);
                    clientContext.Load(clientContext.Web);
                    clientContext.ExecuteQuery();
                    // Get the operation context and remote event service URL.
                    var remoteUrl = GetServiceUrl("ListEventReceiver.svc");
                    // Add RER for Item Added.
                    if (!IsRemoteEventRegistered(clientContext, EventReceiverType.ItemAdded))
                    {
                        appCat.EventReceivers.Add(new EventReceiverDefinitionCreationInformation
                        {
                            EventType = EventReceiverType.ItemAdded,
                            ReceiverName = RERNAME,
                            ReceiverUrl = remoteUrl,
                            SequenceNumber = 10000,
                            Synchronization = EventReceiverSynchronization.Synchronous
                        });
                        clientContext.ExecuteQuery();
                    }
                    // Add RER for Item Updated
                    if (IsRemoteEventRegistered(clientContext, EventReceiverType.ItemUpdated)) return;
                    appCat.EventReceivers.Add(new EventReceiverDefinitionCreationInformation
                    {
                        EventType = EventReceiverType.ItemUpdated,
                        ReceiverName = RERNAME,
                        ReceiverUrl = remoteUrl,
                        SequenceNumber = 10001
                    });
                    clientContext.ExecuteQuery();
                });
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                Logger.Logger.LogError(ex.ToString());
            }
        }

        public static bool IsRemoteEventRegistered(ClientContext clientContext, EventReceiverType type)
        {
            var result = false;
            if (null == clientContext) throw new ArgumentNullException("clientContext");
            try
            {
                // Get the list
                Logger.Logger.LogInfo("Checking if remote events registered", () =>
                {
                    var srcList = clientContext.Web.Lists.GetByTitle(LISTNAME);
                    clientContext.Load(clientContext.Web);
                    clientContext.ExecuteQuery();
                    // Iterate all event receivers.
                    clientContext.Load(srcList.EventReceivers);
                    clientContext.ExecuteQuery();
                    foreach (var er in srcList.EventReceivers)
                        if (0 == string.Compare(er.ReceiverName, RERNAME, true, CultureInfo.CurrentCulture) && er.EventType == type)
                        {
                            result = true;
                            break;
                        }
                });
                return result;
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                Logger.Logger.LogError(ex.ToString());
            }
            return false;
        }

        public static void UnregisterRemoteEvents(ClientContext clientContext)
        {
            if (null == clientContext) throw new ArgumentNullException("clientContext");
            try
            {
                Logger.Logger.LogInfo("Unregistering remote events", () =>
                {
                    // Get the App Catalog.
                    var appCat = clientContext.Web.Lists.GetByTitle(LISTNAME);
                    clientContext.Load(clientContext.Web);
                    clientContext.ExecuteQuery();
                    // Remove all event receivers.
                    clientContext.Load(appCat.EventReceivers);
                    clientContext.ExecuteQuery();
                    var toDelete = new List<EventReceiverDefinition>();
                    // ReSharper disable once LoopCanBeConvertedToQuery
                    foreach (var er in appCat.EventReceivers)
                    {
                        if (er.ReceiverName == RERNAME) toDelete.Add(er);
                    }
                    foreach (var er in toDelete)
                    {
                        er.DeleteObject();
                        clientContext.ExecuteQuery();
                    }
                });
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                Logger.Logger.LogError(ex.ToString());
            }
        }

        internal static void ProcessAppList(ClientContext clientContext)
        {
            if (null == clientContext) throw new ArgumentNullException("clientContext");
            try
            {
                Logger.Logger.LogInfo("Processing app catalog", () =>
                {
                    // Get the App Catalog and App List Item.
                    var appCat = clientContext.Web.Lists.GetByTitle(LISTNAME);
                    clientContext.Load(clientContext.Web);
                    clientContext.Load(appCat);
                    var query = CamlQuery.CreateAllItemsQuery();
                    var items = appCat.GetItems(query);
                    clientContext.Load(items);
                    clientContext.ExecuteQuery();
                    foreach (var item in items)
                        ProcessAppListItem(clientContext, item);
                });
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                Logger.Logger.LogError(ex.ToString());
            }
        }

        internal static void ProcessAppListItem(ClientContext clientContext, int itemID)
        {
            if (null == clientContext) throw new ArgumentNullException("clientContext");
            if (itemID <= 0) throw new ArgumentOutOfRangeException("itemID");             try {                 // Get the App Catalog and App List Item.                 var appCat = clientContext.Web.Lists.GetByTitle(LISTNAME);                 clientContext.Load(clientContext.Web);                 clientContext.Load(appCat);                 var item = appCat.GetItemById(itemID);                 clientContext.Load(item);                 clientContext.ExecuteQuery();                 ProcessAppListItem(clientContext, item);             }             catch (Exception ex)             {                 Debug.WriteLine(ex.ToString());                 Logger.Logger.LogError(ex.ToString());             }         }         internal static void ProcessAppListItem(ClientContext clientContext, ListItem item)         {             if (null == clientContext) throw new ArgumentNullException("clientContext");             if (null == item) throw new ArgumentNullException("item");             try             {                 Logger.Logger.LogInfo("Processing list item with ID {0}", () => {
                    clientContext.Load(item.File);
                    var stream = item.File.OpenBinaryStream();
                    clientContext.ExecuteQuery();
                    var fileInfo = new FileSaveBinaryInformation();
                    fileInfo.ContentStream = new System.IO.MemoryStream();
                    // Load the app manifest file.
                    ProcessManifest(stream.Value, fileInfo.ContentStream, (manifest, ns) => {
                        // Load the properties.
                        var propNode = manifest.SelectSingleNode("x:App/x:Properties", ns);
                        // Look for the endpoints.
                        var installedNode = propNode.SelectSingleNode("x:InstalledEventEndpoint", ns);
                        var upgradedNode = propNode.SelectSingleNode("x:UpgradedEventEndpoint", ns);
                        var uninstalledNode = propNode.SelectSingleNode("x:UninstallingEventEndpoint", ns);
                        if (null == installedNode)
                        {
                            installedNode = manifest.CreateElement("InstalledEventEndpoint", manifest.DocumentElement.NamespaceURI);
                            propNode.AppendChild(installedNode);
                        }
                        if (null == upgradedNode)
                        {
                            upgradedNode = manifest.CreateElement("UpgradedEventEndpoint", manifest.DocumentElement.NamespaceURI);
                            propNode.AppendChild(upgradedNode);
                        }
                        if (null == uninstalledNode)
                        {
                            uninstalledNode = manifest.CreateElement("UninstallingEventEndpoint", manifest.DocumentElement.NamespaceURI);
                            propNode.AppendChild(uninstalledNode);
                        }
                        // NOTE: We're replacing the app installing and upgrading events so we can manage app lifecycle.
                        // If the deployed originally used these events, we've overridden them.
                        installedNode.InnerText = GetServiceUrl("AppMgmtReceiver.svc");
                        upgradedNode.InnerText = GetServiceUrl("AppMgmtReceiver.svc");
                        uninstalledNode.InnerText = GetServiceUrl("AppMgmtReceiver.svc");
                    });
                    // Save the manifest back to SharePoint.
                    fileInfo.ContentStream.Seek(0, System.IO.SeekOrigin.Begin);
                    item.File.SaveBinary(fileInfo);
                    clientContext.Load(item.File);
                    clientContext.ExecuteQuery();
                }, item.Id);
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
                Logger.Logger.LogError(ex.ToString());
            }
        }

        private static void ProcessManifest(System.IO.Stream inStream, System.IO.Stream outStream, Action<XmlDocument, XmlNamespaceManager> manifestDel)
        {
            if (null == inStream) throw new ArgumentNullException("inStream");
            if (null == outStream) throw new ArgumentNullException("outStream");
            if (null == manifestDel) throw new ArgumentNullException("manifestDel");
            using (var memory = new System.IO.MemoryStream())
            {
                var buffer = new byte[1024 * 64];
                int nread = 0, total = 0;
                while ((nread = inStream.Read(buffer, 0, buffer.Length)) > 0)
                {
                    memory.Write(buffer, 0, nread);
                    total += nread;
                }
                memory.Seek(0, System.IO.SeekOrigin.Begin);
                // Open the app manifest.
                using (var zipArchive = new ZipArchive(memory, ZipArchiveMode.Update, true))
                {
                    var entry = zipArchive.GetEntry("AppManifest.xml");
                    if (null == entry) throw new Exception("Could not find AppManifest.xml in the app archive");
                    var manifest = new XmlDocument();
                    using (var sr = new System.IO.StreamReader(entry.Open()))
                    {
                        manifest.LoadXml(sr.ReadToEnd());
                        sr.Close();
                    }
                    var ns = new XmlNamespaceManager(manifest.NameTable);
                    ns.AddNamespace("x", "http://schemas.microsoft.com/sharepoint/2012/app/manifest");
                    // Call the delegate.
                    manifestDel(manifest, ns);
                    // Write back to the archive.
                    using (var sw = new System.IO.StreamWriter(entry.Open()))
                    {
                        sw.Write(manifest.OuterXml);
                        sw.Close();
                    }
                }
                // Memory stream now contains the updated archive
                memory.Seek(0, System.IO.SeekOrigin.Begin);
                // Write result to output stream.
                buffer = new byte[1024 * 64];
                nread = 0; total = 0;
                while ((nread = memory.Read(buffer, 0, buffer.Length)) > 0)
                {
                    outStream.Write(buffer, 0, nread);
                    total += nread;
                }
            }
        }

        private static string GetServiceUrl(string serviceEndpoint)
        {
            if (string.IsNullOrEmpty(serviceEndpoint)) throw new ArgumentNullException("serviceEndpoint");
            if (null == OperationContext.Current) throw new Exception("Could not get service URL from the operational context.");
            var url = OperationContext.Current.Channel.LocalAddress.Uri.AbsoluteUri;
            var opContext = url.Substring(0, url.LastIndexOf("/", StringComparison.Ordinal));
            return string.Format("{0}/{1}", opContext, serviceEndpoint);
        }
    }
}

This class, both registers event receivers on the App Catalog as well as provide logic for the injection of App Installed events for new and existing add-ins. The ProcessListItem method is the most interesting (in my opinion). This method downloads the APP file from the catalog into a memory stream, unzips the contents with the .NET compression API, makes the changes to the AppManifest.xml file, and then zips the file and re-uploads the APP to the catalog.

The location of the injected endpoint is another remote event receiver on the ACU. The governance code should reside somewhere central, and hosted in the ACU seemed like as good a place as any.

In the above code, we can trace the call from the ACU ProcessEvent method through to the code that updates the existing add-ins in the catalog. For new add-ins added to the catalog later, we also need a list remote event receiver, which the ACU registers and calls an endpoint ListEventReceiver.svc. The following is the code-behind for this service (again hosted in the ACU):

using System;
using System.Diagnostics;
using Microsoft.SharePoint.Client.EventReceivers;
using SPApps.UpdateAppCatWeb;

namespace SPApps.SubSiteCreateWeb.Services
{
    public class ListEventReceiver : IRemoteEventService
    {
        public void ProcessOneWayEvent(SPRemoteEventProperties properties)
        {
            throw new NotImplementedException();
        }

        public SPRemoteEventResult ProcessEvent(SPRemoteEventProperties properties)
        {
            var result = new SPRemoteEventResult();
            Logger.Logger.LogInfo("ProcessEvent called on ListEventReceiver", () =>
            {
                if (null == properties) throw new ArgumentNullException("properties");
                try
                {
                    switch (properties.EventType)
                    {
                        case SPRemoteEventType.ItemAdded:
                            using (var clientContext = TokenHelper.CreateRemoteEventReceiverClientContext(properties))
                                AppHelper.ProcessAppListItem(clientContext, properties.ItemEventProperties.ListItemId);
                            break;
                    }

                }
                catch (Exception ex)
                {
                    Logger.Logger.LogError(ex.ToString());
                    Debug.WriteLine(ex.ToString());
                }
            });
            return result;
        }
    }
}

Going back to the App Helper, take note of the code that registers the List Remote Event Receiver. I register the App Installed event as Synchronous. This is important, because this event is fired when uploading an APP to the catalog. SharePoint provides a dialog box to enter metadata about the add-in as part of the upload process. Leaving the event as asynchronous (default) causes a conflict error when clicking the save button on the dialog.

Lastly, let’s look at the code that is called from the injected RER endpoint – the code that allows or denies installation of the add-in:

using System;
using Microsoft.SharePoint.Client.EventReceivers;

namespace SPApps.UpdateAppCatWeb.Services
{
    public class AppMgmtReceiver : IRemoteEventService
    {
        /// <summary>
        /// Handles app events that occur after the app is installed or upgraded, or when app is being uninstalled.
        /// </summary>
        /// <param name="properties">Holds information about the app event.</param>
        /// <returns>Holds information returned from the app event.</returns>
        public SPRemoteEventResult ProcessEvent(SPRemoteEventProperties properties)
        {
            var result = new SPRemoteEventResult();
            Logger.Logger.LogInfo("ProcessEvent called for AppMgmtReceiver", () =>
            {
                using (var clientContext = TokenHelper.CreateAppEventClientContext(properties, false))
                {
                    switch (properties.EventType)
                    {
                        case SPRemoteEventType.AppInstalled:
                            result.Status = SPRemoteEventServiceStatus.CancelWithError;
                            result.ErrorMessage = "You are not allowed to install this app!";
                            break;
                        case SPRemoteEventType.AppUpgraded:
                            break;
                    }
                }
            });
            return result;
        }

        /// <summary>
        /// This method is a required placeholder, but is not used by app events.
        /// </summary>
        /// <param name="properties">Unused.</param>
        public void ProcessOneWayEvent(SPRemoteEventProperties properties)
        {
            var result = new SPRemoteEventResult();
            Logger.Logger.LogInfo("ProcessEvent called for AppMgmtReceiver", () =>
            {
                using (var clientContext = TokenHelper.CreateAppEventClientContext(properties, false))
                {
                    switch (properties.EventType)
                    {
                        case SPRemoteEventType.AppInstalled:
                            break;
                        case SPRemoteEventType.AppUpgraded:
                            break;
                    }
                }
            });
        }

    }
}

The above code is very simplistic and just denies installation of any add-in. However, this could easily be adapted (and will be for my customer) to check a central repository for before determining that an add-in is allowed or denied installation.

So, there we have it. With a little bit of trickery to the App Catalog it’s possible to implement rudimentary add-in governance. The complete project is available on github: https://github.com/robgarrett/SharePoint-Apps.

ALL CODE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND. I MAKE NO WARRANTIES, EXPRESS OR IMPLIED, THAT THEY ARE FREE OF ERROR, OR ARE CONSISTENT WITH ANY PARTICULAR STANDARD OF MERCHANTABILITY, OR THAT IT WILL MEET YOUR REQUIREMENTS FOR ANY PARTICULAR APPLICATION. I DISCLAIM ALL LIABILITY FOR DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES RESULTING FROM YOUR USE OF THE INCLUDED CODE. ALL CODE PROVIDED OR LINKED IS FREE FOR DISTRIBUTION AND IS NOT CONSIDERED PROTECTED INTELLECTUAL PROPERTY OF MINE NOR MICROSOFT CORPORATION.