Sample SOAP payload for a SharePoint Remote Event Receiver

This post is mostly about showing what an actual Remote Event Receiver message looks like but there are some of my musings too because if there's something I love more than coding it is ranting about minor annoyances with the tools we use for doing it. If you can't wait to see the actual sample - go straight ahead, it's waiting for you at the end. For the most patient ones - let the rant begin!

First, a bit of context - I've joined new team working on a super complex project and as a result they are not letting me anywhere near the actual code for a second month now. What do they have me do, then? What could be the most inconsequential thing for a noob to do that would still have them learn about the solution? You guessed right - unit tests! So I want to test a Remote Event Receiver (RER for short) and I though, well, surely there must be a sample SOAP message sent by a RER somewhere around teh interwebz. Disappointingly, if there is such a thing it is very well hidden as I couldn't find it and had to generate my own. It wasn't as simple as one might think so I'm sharing the fruit of this labour with the world.

Now is the time to rant a bit about the frustrations regarding SharePoint development. I've ranted before about how hard it is to find anything meaningful online because of the ubiquity of posts explaining trivial stuff. Seriously, who needs a million how-tos about adding a RER to an app? You just right-click on the project in Visual Studio and follow the wizard. One thing I haven't moaned about before though is the inept way Microsoft is presenting SharePoint app development. And no, I'm not even referring to the ham-fisted rebranding from "apps" to "add-ins", which left the community calling them "apps" while MS insist that they are "add-ins", in the same way that MS insisted for years that JavaScript is actually called EcmaScript.

What is SharePoint app development anyway?

If you try to teach yourself SharePoint 2013/2016 through online resources, or even if you attend a course or read a book they'll have you believe that before you even write a hello world app you need to understand about SharePoint-hosted vs Provider hosted, you need to know what's the app web and host web, the remote web app, etc. and before you even start coding you need to configure your farm for apps, which is decidedly non-trivial.

I'd say screw all this and don't confuse people with the complexities of deployed apps from day one - it's needlessly obscure and fails to drive home the main point of the SP 2013/2016 app architecture: from now on, you won't be writing applications that get deployed into SharePoint, you'll be building ASP.NET apps that talk to SharePoint through the client library.
That's what Microsoft doesn't want you to know and maybe even fail to admit it to themselves but seriously, does anyone actually build apps to be deployed through an app catalog? Almost all projects I've worked on lately involve writing code in an ASP.NET application that talks to SharePoint through the CSOM. Yeah, I get it, MS built all this app catalog infrastructure and now want to shove it down our throats so they keep talking about provider hosted and app web and whatnot when you just want to connect to SharePoint and update some files.

So, if there is a beginner SharePoint developer reading this - fear not, it's not as complicated as MS make it sound. Nowhere is this more obvious than when we bring Remote Event Receivers (RER) into the picture. If you go by MS sources alone you'll start pulling your hair in despair, expecting weeks of work just to get the damn thing to work - now you don't just have the usual moving parts and security config, you also have a service that needs yet another way to obtain a SharePoint context! Well, guess what - you don't need any of that - you just create your service that implements the correct contract, call a few lines of CSOM code to register your services as a RER and you're good to go; no apps, no app-only policy configuration, no nothing. Now is the time to give a big shout out to Dan Budimir, who went against the grain at MS and wrote a guide about this very thing:
https://blogs.msdn.microsoft.com/boodablog/2015/01/12/i-thought-i-needed-a-sharepoint-provider-hosted-app-for-that-part-1/

How do we capture the SOAP message?

When I came to be in need of a recorded SOAP payload from a RER my first thought was - hey, we already have this functionality in IIS through Failed Request Tracing, which can be used to record all traffic, including requests that didn't actually fail. However, it has a size limit and for most RER request bodies it actually cuts the ending. Therefore, Fiddler had to come to the rescue - here is the whole process if you want to capture some requests yourself:

• Setup a dummy service (it doesn't have to do anything, just implement IRemoteEventService so it can be called from a RER)
• Install Fiddler if you don't have it already
• Configure Fiddler to intercept incoming traffic – e.g. http://stackoverflow.com/questions/4428680/how-do-i-monitor-all-incoming-http-requests
• Attach the receivers – to the correct port, the one Fiddler is listening on (8888 by default), see the above link from Dan Budimir
• Add XML inspector to Fiddler e.g. http://tangoxmlview.codeplex.com/releases/view/58841>TangoXmlView – ridiculously enough, you can’t see the content without an inspector
• Do the activity you subscribed for (ItemUpdated or whatever) - it should show up in Fiddler and you can grab the coveted SOAP message dump

If all this sounds too complicated (which it is) - just make use of the few that I prepared, you should be able to adapt them if they don't cover the event you need.
Below you'll find the full text dumps for ItemDeleted (the smallest one) and ItemUpdated and here are links for two more events:
ItemAdded
ItemFileMoved (that's the event that gets fired when you rename a file)

ItemDeleted

<?xml version="1.0" encoding="UTF-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
   <s:Body>
      <ProcessOneWayEvent xmlns="http://schemas.microsoft.com/sharepoint/remoteapp/">
         <properties xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
            <AppEventProperties i:nil="true" />
            <ContextToken />
            <CorrelationId>5bccd79d-32db-e06a-0000-077828386b67</CorrelationId>
            <CultureLCID>1033</CultureLCID>
            <EntityInstanceEventProperties i:nil="true" />
            <ErrorCode />
            <ErrorMessage />
            <EventType>ItemDeleted</EventType>
            <ItemEventProperties>
               <AfterProperties xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
               <AfterUrl i:nil="true" />
               <BeforeProperties xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
               <BeforeUrl>Docs/test11.txt</BeforeUrl>
               <CurrentUserId>22</CurrentUserId>
               <ExternalNotificationMessage i:nil="true" />
               <IsBackgroundSave>false</IsBackgroundSave>
               <ListId>6980e5dd-d847-4667-8792-1e2741d6cf18</ListId>
               <ListItemId>7</ListItemId>
               <ListTitle>Docs</ListTitle>
               <UserDisplayName>Kiryazov, S. (Stefan)</UserDisplayName>
               <UserLoginName>i:0#.w|europe\d-hy32co</UserLoginName>
               <Versionless>false</Versionless>
               <WebUrl>http://sharepoint.server/sites/sitecolurl</WebUrl>
            </ItemEventProperties>
            <ListEventProperties i:nil="true" />
            <SecurityEventProperties i:nil="true" />
            <UICultureLCID>1033</UICultureLCID>
            <WebEventProperties i:nil="true" />
         </properties>
      </ProcessOneWayEvent>
   </s:Body>
</s:Envelope>

ItemUpdated

<?xml version="1.0" encoding="UTF-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Body>
    <ProcessOneWayEvent xmlns="http://schemas.microsoft.com/sharepoint/remoteapp/">
      <properties xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <AppEventProperties i:nil="true" />
        <ContextToken />
        <CorrelationId>00000000-0000-0000-0000-000000000000</CorrelationId>
        <CultureLCID>1033</CultureLCID>
        <EntityInstanceEventProperties i:nil="true" />
        <ErrorCode />
        <ErrorMessage />
        <EventType>ItemUpdated</EventType>
        <ItemEventProperties>
          <AfterProperties xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
            <a:KeyValueOfstringanyType>
              <a:Key>vti_folderitemcount</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_nexttolasttimemodified</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:30:41</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_candeleteversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">true</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_foldersubfolderitemcount</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_canmaybeedit</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">true</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>ContentTypeId</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">0x010100AE33CE7DE32F484490EEC317F89BE311</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_sourcecontrolcookie</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">fp_internal</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_filesize</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">1376</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_modifiedby</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">i:0#.w|domain\username</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_setuppathversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">15</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_level</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">1</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_timecreated</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:30:41</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_charset</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">utf-8</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_parserversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">15.0.0.4867</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_sourcecontrolversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">V1.0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_author</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">i:0#.w|domain\username</a:Value>
            </a:KeyValueOfstringanyType>
          </AfterProperties>
          <AfterUrl>Docs/soap-list1.xml</AfterUrl>
          <BeforeProperties xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
            <a:KeyValueOfstringanyType>
              <a:Key>vti_metadatanextbsn</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">100</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_backgroundsave</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_replid</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">rid:{2824DA8B-7E05-4687-A6D3-E5293D4439FC}</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_level</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">1</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>ContentTypeId</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">0x010100AE33CE7DE32F484490EEC317F89BE311</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_setuppathversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">15</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_rtag</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">rt:2824DA8B-7E05-4687-A6D3-E5293D4439FC@00000000002</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_sourcecontrolversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">V1.0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_sourcecontrolcookie</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">fp_internal</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_filesize</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">1376</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_modifiedby</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">i:0#.w|domain\username</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_metainfoversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">2</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_internalversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">513</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_nextbsn</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">162</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_parserversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">15.0.0.4867</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_docstoreversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">2</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_author</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">i:0#.w|domain\username</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_foldersubfolderitemcount</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_contentversionisdirty</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">false</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_timecreated</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:30:41</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_contenttag</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">{2824DA8B-7E05-4687-A6D3-E5293D4439FC},2,2</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_docstoretype</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_canmaybeedit</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">true</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_charset</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">utf-8</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_folderitemcount</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">0</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_timelastmodified</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:55:24</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_parentid</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">{733E0BCD-E0B4-45DA-8492-FDE35C98DD22}</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_candeleteversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">true</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_nexttolasttimemodified</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:30:41</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_contentversion</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">2</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_etag</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:string">"{2824DA8B-7E05-4687-A6D3-E5293D4439FC},2"</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_timelastwritten</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:dateTime">2017-02-01T08:55:24</a:Value>
            </a:KeyValueOfstringanyType>
            <a:KeyValueOfstringanyType>
              <a:Key>vti_streamschema</a:Key>
              <a:Value xmlns:b="http://www.w3.org/2001/XMLSchema" i:type="b:int">66</a:Value>
            </a:KeyValueOfstringanyType>
          </BeforeProperties>
          <BeforeUrl>Docs/soap-list1.xml</BeforeUrl>
          <CurrentUserId>22</CurrentUserId>
          <ExternalNotificationMessage i:nil="true" />
          <IsBackgroundSave>false</IsBackgroundSave>
          <ListId>6980e5dd-d847-4667-8792-1e2741d6cf18</ListId>
          <ListItemId>2</ListItemId>
          <ListTitle>Docs</ListTitle>
          <UserDisplayName>Kiryazov, S. (Stefan)</UserDisplayName>
          <UserLoginName>i:0#.w|europe\d-hy32co</UserLoginName>
          <Versionless>false</Versionless>
          <WebUrl>http://sharepoint.server/sites/ipp-content-stef</WebUrl>
        </ItemEventProperties>
        <ListEventProperties i:nil="true" />
        <SecurityEventProperties i:nil="true" />
        <UICultureLCID>1033</UICultureLCID>
        <WebEventProperties i:nil="true" />
      </properties>
    </ProcessOneWayEvent>
  </s:Body>
</s:Envelope>

Processing XML with .NET while preserving your sanity

XML is facing quite a bit of criticism, raging from accusations of being too complex to just being on of Micrsoft's attempts to take over the world (it's actually an open W3C standard but nevermind). The general consensus is however that it does what is says on the tin, which is to transfer data between applications, allowing humans to take a peak in-between. The focus of this post is the last part involving human beings (stupid humans, always making programming more difficult than it needs to be!)

Consider this piece of XML:

<?xml version="1.0" encoding="UTF-8"?>
<xmldata>
  <element>value</element>
</xmldata>

Nice and readable, isn't it? Well, most times at least. Now look at it without the new lines and identation:

<?xml version="1.0" encoding="UTF-8"?><xmldata><element></element></xmldata>

Much less so, indeed!
Everyone who has had their fair share of XML programming has faced this situation - for parsers the two XML documents are identical, however when we need to take a look at our data when debugging we get confused by the ugly, unformatted XML. This is made worse by the .NET Framework's tendency to produce the latter variety of XML by default.

---

There are, of course, simple ways to make our lives easier by formatting XML the way we want it - it's just less obvious than it should, so I decided to put this post together and shed some light on a few tricks and subtleties.

Dumping XML to a file & XmlWriter

It's a natural scenario to grab an XML file, do some processing, and save it. Here is a trivial piece of code that does that:

XmlDocument xml = new XmlDocument();
xml.Load(INPUT_PATH);
// ... XML processing code
xml.Save(OUTPUT_PATH);

This of course can easily be adapted to save the XML to a database, over the network or wherever else we need it to be. Most times it would appear to work fine, until you try it on our simple input. Here is what we get:

<?xml version="1.0" encoding="utf-8"?>
<xmldata>
  <element>
  </element>
</xmldata>

Looks almost the same - but not quite, the closing tag is on a new line! Curiously though, when there is an actual value between the tags we don't get a new line added to it, so it might be pretty hard to spot in a complex XML document with most elements containing values. That's actually what caused the bug I was troubleshooting when I got the inspiration for this post - it might look like it's not a big deal but what's between the opening and closing tag is our value and we just had a new line inserted into it - for most applications a new line is quite different than an empty string!
So then, what do we do about it?

Using XmlWriter

We have a class in the .NET Framework that's meant to give us more control over XML-exporting operations - XmlWriter. Here is the simplest possible way to use it, without supplying any explicit settings:

XmlDocument xml = new XmlDocument();
xml.Load(INPUT_PATH);
// ... XML processing code
XmlWriter writer = XmlWriter.Create(OUTPUT_PATH);
xml.Save(writer);
writer.Close();

And the result:

<?xml version="1.0" encoding="utf-8"?><xmldata><element></element></xmldata>

... i.e. exactly what we were trying to escape from. Looks like the XmlWriter is of not much use by itself when it comes to formatting XML for human-readability.

XmlWriterSettings.Indent

Luckily, there is the Indent property in XmlWriterSettings which is false by default but can very easily bet set to true:

// ... XML processing code
XmlWriterSettings writerSettings = new XmlWriterSettings();
writerSettings.Indent = true;
XmlWriter writer = XmlWriter.Create(OUTPUT_PATH, writerSettings);
xml.Save(writer);

Leading to, finally, a well-formatted XML output!

XmlWriter with it's settings object is nice and all, but there is one caveat that you could hit before getting to the Indent property - there is also the NewLineHandling property, which you might be tricked into thinking would achieve our goal. In fact, it only affects the new lines within actual values between tags and doesn't apply to the new lines in the markup.

Juggling with XML formats in-memory

But wait a second, this neat solution relies on XmlWriter, which - in this example, at least - only writes to a file. What if we don't actually need to write the file to the FS and we'd rather have a string, byte array or some other in-memory structure? We have a few ways to achieve this using the same XmlWriter-based logic, presented here from most to least atrocious.
One option is to just save the file and read it back like a text or binary file - this will load the well-formatted XML in memory as string/bytes/whatever. In case this solution looks attractive, I have some advice for you - don't do it. Even with an SSD drive read/write operations are expensive, and also an unnecesarry risk - the HDD might be full, we might not have access to the folder, etc. I can think of only one situation where this 'solution' would be advisable - if you actually need the physical files, e.g. for debugging or logging purposes.
Another way to harness XmlWriter for this task is to combine it with .NET's flexible, polymorphic stream architecture and create your XmlWriter around a MemoryStream. Now if using something called MemoryStream in order to just sanitize your XML doesn't sound like an overkill to you then I guess I can't argue further - if you haven't been featured on TheDailyWTF you probably will soon be.
But fear not - there is another way. Enter LINQ...

XElement/XDocument

This solution doesn't actually use LINQ itself - it just taps on the XElement-based infrastructure that LINQ to XML uses to objectify XML documents. Apart from making it possible to use LINQ on XML, these classes also use some of the more recent .NET framework additions, like object initialization and anonymous types, in order to make dealing with XML in .NET less cumbersome.
Here is how to beautify an XML document in-memory and assign it to a string:

XDocument xDoc = XDocument.Load(INPUT_PATH);
// ... XML processing code
String xmlString = xDoc.ToString();

And here is what we get:

<xmldata>
  <element></element>
</xmldata>

Almost but not quite - it's missing the XML declaration (<?xml version...). Here is how add it - we just need to change the last line to:

String xmlString = xDoc.Declaration.ToString() + Environment.NewLine + xDoc.ToString();

And voila - we have a well-formatted XML in-memory, in two lines!

Further notes on XML and strings in .NET

But why did we need to change that line in order to add the XML declaration, why doesn't it get included automatically? Upon further observation, we can see that there is the XDocument.Save(String path) method, which saves the document to disk and does include the declaration - so it starts to look like an unintentional omission to not include it in ToString()?

As it turns out, not only is there a reason for that, but there are in fact at least two good reasons to implement ToString() in such a way, and each of them reveals something interesting about the way .NET handles XML and strings in general

Handling chunks of XML in-memory

The traditional XmlElement-based approach is a traditional DOM implementation - it builds a tree of the document in memory, and deals with all pieces of XML as documents - even if it's a single element, a dummy XML document object will be created around it. That's not the case with XElement - there an XElement instance can represent just one XML element without any context.
This point of view is taken further by considering the XML declaration to not be a part of the document - it's just a header of the .xml file format to mark the content as valid XML, to indicate the version and encoding (although you need to know what's the encoding in order to read the header that gives you the encoding but nevermind). That's why you only get the XML declaration inserted when you save the thing to a file - before that it's just a document in memory that holds a piece of XML markup.

String encoding in .NET

First, a quick refresher on encodings, as I suspect that developers that work for the western market only don't deal with them in-depth on a daily basis. Encodings are ways to map characters to sequences of bits so that they can be stored in binary media. Everyone knows about ASCII, which assigns one byte per character and fits just the Latin alphabet and a bunch of funny symbols, and Un In the average .NET developer's practice, encodings are used explicitly in order to convert strings to bytes and vice versa. Let's extend our example in order to get that neat XML in a byte array, e.g. to be sent over a socket:

XDocument xDoc = XDocument.Load(INPUT_PATH);
String xmlString = xDoc.Declaration.ToString() + Environment.NewLine + xDoc.ToString();
byte[] xmlBytes = Encoding.UTF8.GetBytes(xmlString);

Now, we can of course call our good friends XmlWriter and MemoryStream but as was demonstrated we have a better way to deal with the task at hand - the only change from the previous example is the addition of the last line that uses the UTF8 encoding to convert the sequence of symbols that is represented by our string to a sequence of bytes. I bolded this because it's crucial for my next point - to understand the situation here, we need to think of strings abstractly. When we have a string object it is of course just a point to a region in memory that is filled with bytes but that's of no concern to our encoding object - it only looks at the sequence of charters, regardless of how they are represented in memory. It then generates the matching bytes for each character to give us our byte array - that's it.

OK, but what does this have to do with XML and the reason why XDocument doesn't include the XML declaration? Well, it's the same principle - XDocument is a pure soul without a body (i.e. a physical file), and it treats the encoding as a bodily concern - it's a mere physical representation of the information in the XML document. That's why the pure information contained in the XDocument object shouldn't contain the XML header, and with it - the encoding, when in fact it isn't associated with any encoding at all.

.NET does store strings as bytes in memory, so there is one more encoding operation going on all the time - the mapping of our sequence of symbols to the bytes in the managed heap. For this purpose, CLR uses UTF-16 - hence the 2 byte size of the char datatype and 2 bytes per symbol for string. What is a little bit confusing at first is that there is no specific UTF-16 encoding option, although we have UTF7, UTF8, UTF32 and Unicode - which is not even an encoding but the overall standard that covers them all. The System.Text.Encoding.Unicode encoding is in fact UTF-16, which is a glimpse into how in the .NET architects' team they assume UTF-16 to be the default encoding.

For more details and fun into how strings, and objects in general, are stored in memory in .NET you can always count on the guru Jon Skeet: http://msmvps.com/blogs/jon_skeet/archive/2011/04/05/of-memory-and-strings.aspx

Bonus: The code

Here is a simple Visual Studio 2010 solution that demonstrates all the code in this post for download

Also, in case you don't like downloading files from strangers - here is the same solution shared on CodePlex:
https://xmldemo.codeplex.com/

InfoPath - The URN specified in the XML template file does not match the URN specified in the form

You cheeky developer you, been monkeying around with your XSN files, haven't you ;-)

There comes a time in every SharePoint developer's life when he has to deal with some InfoPath. That's often a frustrating experience - especially when inheriting a middle-sized to large project. InfoPath probably does what it's supposed to do well - and this is to provide static forms for users to fill. If you swerve from the road well travelled however and try to build something more custom some opportunities (as we call problems nowadays) are likely to emerge, which get aggravated by the limited resources on the more specific scenarios.

One of those specific cases is when you need to manage form templates on the fly, programmatically. That's not particularly hard - after all InfoPath form templates are just CAB files containing XML, so you can dynamically unwrap them, do the changes needed and then put them back together with MakeCab or similar tool. Still, the process is not that hard to botch, and this "The URN specified in the XML template file does not match the URN specified in the form" error is one of the possible outcomes of getting it wrong.

This one is kind of self-descriptive, although it might not be exactly intuitive for InfoPath newbies. See, the URN (Uniform Resource Name) something like an identifier for the form template and is referenced on two different places throughout the files that comprise the XSN package and naturally, it needs to be the same in both. So, if you are spewing InfoPath form templates on the fly, with different names, you need to take care to edit the URN in sync - otherwise you end up with this error. You can also leave these alone altogether and not edit them - I haven't noticed any adverse consequences so far.

First, in minifest.xsf:

<?xml version="1.0" encoding="UTF-8"?>
<!--
This file is automatically created and modified by Microsoft InfoPath.
Changes made to the file outside of InfoPath might be lost if the form template is modified in InfoPath.
-->
<xsf:xDocumentClass solutionFormatVersion="2.0.0.0" solutionVersion="1.0.0.46" productVersion="14.0.0" name="urn:schemas-microsoft-com:office:infopath:your_form_name:-myXSD-2011-02-09T17-25-42" xmlns:xsf="http://schemas.microsoft.com/office/infopath/2003/solutionDefinition" xmlns:xsf2="http://schemas.microsoft.com/office/infopath/2006/solutionDefinition/extensions" ...

Then, in template.xml:

<?xml version="1.0" encoding="UTF-8"?>

<?mso-infoPathSolution name="urn:schemas-microsoft-com:office:infopath:your_form_name:-myXSD-2011-02-09T17-25-42" solutionVersion="1.0.0.46" productVersion="14.0.0" PIVersion="1.0.0.0" ?>
<?mso-application progid="InfoPath.Document" versionProgid="InfoPath.Document.2"?>
<?mso-infoPath-file-attachment-present?>
<my:myFields ...