The Morning Brew

Image Credit: kevinzim on Flickr

One of my basic assumptions about the .NET framework was proved incorrect last week. Up until then, I had believed that when you are using the built in framework classes for exposing web services you were always safe when it came to the output being valid XML. Sadly that turns out to be untrue, and to make matters worse, .NET will happily output XML over a web service even its own .NET web service clients can’t read.

There are certain characters that are forbidden from being in XML as per the official specification. These characters are the low ascii characters such as NULL, EOF etc. Its important to note that this is not a case of unescaped/unencoded versions of this character being disallowed, the encoded characters are also disallowed.

The problem with this isn’t that the .NET framework doesn’t understand these rules – it manages just fine when it comes to acting as a client to a web service serving these characters in content, throwing nice exceptions explaining that these characters are invalid. Additionally, the XML Text Reader has a property ‘Normalization’ which causes the XML reader to be more liberal and ignore invalid characters – but this option is not used within the automatically generated Web Service Client.

This problem isn’t limited to just the web services, standard XML serialisation also experiences the same problems. Here are bits of code that illustrate the problem:

[WebMethod()]
public string InvalidCharacter()
{
   return "" + (char)4;
}

public class MyClass
{
   public string Test = "" + (char)0;
   public static void Main()
{
     MyClass c = new MyClass();
     System.Xml.Serialization.XmlSerializer xs = new System.Xml.Serialization.XmlSerializer(typeof(MyClass));
     System.Text.StringBuilder sb = new System.Text.StringBuilder();
     System.IO.StringWriter sw = new System.IO.StringWriter(sb);
     xs.Serialize(sw, c);
     Console.WriteLine(sb.ToString());
    
     System.IO.StringReader sr = new System.IO.StringReader(sb.ToString());
     try
     {
         c = (MyClass)xs.Deserialize(sr);
     }
     catch (System.Exception ex)
     {
         Console.WriteLine(ex.ToString());
}
}
}

The little console application gives output like this:

   �

System.InvalidOperationException: There is an error in XML document (3, 12).
System.Xml.XmlException: '.', hexadecimal value 0x00, is an invalid character.
Line 3, position 12.

Unfortunately I haven’t yet found a fix for this – my only solution is to work around the problem by ensuring that these invalid characters can’t get into the system in the first place or clean the text on the get of each property of the serialized objects.

Things like this really worry me – our frameworks shouldn’t be outputting things that they can’t read – let alone outputting things that completely contravene the specifications. As always, any suggestions on an alternative solution to this problem are welcome.

6 Comments »

Setting the scene

Web Services are very simple to consume using .NET code, and its all to easy to forget what is actually going on when you add that Web Reference to your project in Visual Studio. Once you’ve entered the URL for you web service and clicked the Add Reference button, Visual Studio requests the service description WSDL file, and code generates classes to represent the data and the web service methods. I found delving into this generated code taught me quite a lot about the way XML serialization and web services actually work.

Behind the scenes the web service proxy uses classes in the System.Web.Services.Protocols namespace to actually perform the calls to the service, and these calls end up as System.Net.WebRequests containing the correctly encoded data that makes up the message.

Problems with Web Requests

Quite a lot of the problems with web services I encounter during deployment are actually problems with the System.Net.WebRequest. The most common cause of problems seems to be problems where web access is provided via some form of HTTP Proxy Server, and this typically results in an exception of the form:
[WebException: The underlying connection was closed: Unable to connect to the remote server.]

If you have access to the machine running your application its a very good idea to check if the machine can make a request to your Web Service endpoint using a web browser. This also allows you to check the settings to see if web traffic is passing through a proxy – failing this, check with the people responsible for the network.

If you find that there is a proxy involved, there are a couple of strategies available to resolve the problem.

• Make Changes in machine.config to affect all applications
  machine.Config is one of the .NET framework’s main configuration files, and can be found in the framework directory. Settings in the defaultProxy section with that attribute usesystemdefault="false"allow system wide setting of a default proxy server overriding an OS setting:
      
          
                                usesystemdefault = "false"
                    proxyaddress="http://proxyserver:port"
                    bypassonlocal="false"
                />
            
        
    
• Configuration change for your application only
  If you don't control the machine your application is running on then its unlikely you will be able to make changes to machine.config, but thankfully the above section will also work in your web.config or app.confg files
• Programmatic Solution
  You can set the proxy settings in code:

  using System.Net;

MyWebService ws= new MyWebService();
WebProxy proxyObject = new WebProxy("http://proxyservername:port", true);
MyWebService.Proxy = proxyObject;
MyWebService.MyWebMethod();