Web Services

Archived Posts from this Category

Invalid XML from a .NET web service

Posted by on 01 Oct 2007 | Tagged as: .NET, ASP.NET, C#, Development, Web Services, XML

10187684_78f140f0e2_m.jpg
Image Credit: kevinzim on Flickr

One of my basic assumptions about the .NET framework was proved incorrect last week. Up until then, I had believed that when you are using the built in framework classes for exposing web services you were always safe when it came to the output being valid XML. Sadly that turns out to be untrue, and to make matters worse, .NET will happily output XML over a web service even its own .NET web service clients can’t read.

There are certain characters that are forbidden from being in XML as per the official specification. These characters are the low ascii characters such as NULL, EOF etc. Its important to note that this is not a case of unescaped/unencoded versions of this character being disallowed, the encoded characters are also disallowed.

The problem with this isn’t that the .NET framework doesn’t understand these rules – it manages just fine when it comes to acting as a client to a web service serving these characters in content, throwing nice exceptions explaining that these characters are invalid. Additionally, the XML Text Reader has a property ‘Normalization’ which causes the XML reader to be more liberal and ignore invalid characters – but this option is not used within the automatically generated Web Service Client.

This problem isn’t limited to just the web services, standard XML serialisation also experiences the same problems. Here are bits of code that illustrate the problem:

[WebMethod()]

public string InvalidCharacter()
{
   return "" + (char)4;
}

public class MyClass
{
   public string Test = "" + (char)0;
   public static void Main()
{
     MyClass c = new MyClass();
     System.Xml.Serialization.XmlSerializer xs = new System.Xml.Serialization.XmlSerializer(typeof(MyClass));
     System.Text.StringBuilder sb = new System.Text.StringBuilder();
     System.IO.StringWriter sw = new System.IO.StringWriter(sb);
     xs.Serialize(sw, c);
     Console.WriteLine(sb.ToString());
    
     System.IO.StringReader sr = new System.IO.StringReader(sb.ToString());
     try
     {
         c = (MyClass)xs.Deserialize(sr);
     }
     catch (System.Exception ex)
     {
         Console.WriteLine(ex.ToString());
}
}
}

The little console application gives output like this:

<?xml version="1.0" encoding="utf-16"?>
<MyClass xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <Test>&#x0;</Test>
</MyClass>

System.InvalidOperationException: There is an error in XML document (3, 12).
System.Xml.XmlException: '.', hexadecimal value 0x00, is an invalid character.
Line 3, position 12.

Unfortunately I haven’t yet found a fix for this – my only solution is to work around the problem by ensuring that these invalid characters can’t get into the system in the first place or clean the text on the get of each property of the serialized objects.

Things like this really worry me – our frameworks shouldn’t be outputting things that they can’t read – let alone outputting things that completely contravene the specifications. As always, any suggestions on an alternative solution to this problem are welcome.

Web Service Connection Problems

Posted by on 06 Feb 2007 | Tagged as: .NET, C#, Development, IIS, SysAdmin, Web Services

Setting the scene

Web Services are very simple to consume using .NET code, and its all to easy to forget what is actually going on when you add that Web Reference to your project in Visual Studio. Once you’ve entered the URL for you web service and clicked the Add Reference button, Visual Studio requests the service description WSDL file, and code generates classes to represent the data and the web service methods. I found delving into this generated code taught me quite a lot about the way XML serialization and web services actually work.

Behind the scenes the web service proxy uses classes in the System.Web.Services.Protocols namespace to actually perform the calls to the service, and these calls end up as System.Net.WebRequests containing the correctly encoded data that makes up the message.

Problems with Web Requests

Quite a lot of the problems with web services I encounter during deployment are actually problems with the System.Net.WebRequest. The most common cause of problems seems to be problems where web access is provided via some form of HTTP Proxy Server, and this typically results in an exception of the form:
[WebException: The underlying connection was closed: Unable to connect to the remote server.]

If you have access to the machine running your application its a very good idea to check if the machine can make a request to your Web Service endpoint using a web browser. This also allows you to check the settings to see if web traffic is passing through a proxy – failing this, check with the people responsible for the network.

If you find that there is a proxy involved, there are a couple of strategies available to resolve the problem.

  • Make Changes in machine.config to affect all applications
    machine.Config is one of the .NET framework’s main configuration files, and can be found in the framework directory. Settings in the defaultProxy section with that attribute usesystemdefault="false"allow system wide setting of a default proxy server overriding an OS setting:<configuration>
        <system.net>
            <defaultProxy>
                <proxy
                     usesystemdefault = "false"
                        proxyaddress="http://proxyserver:port"
                        bypassonlocal="false"
                    />
                </defaultProxy>
            </system.net>
        </configuration>
  • Configuration change for your application only
    If you don’t control the machine your application is running on then its unlikely you will be able to make changes to machine.config, but thankfully the above section will also work in your web.config or app.confg files
  • Programmatic Solution
    You can set the proxy settings in code:

    using System.Net;

    MyWebService ws= new MyWebService();
    WebProxy proxyObject = new WebProxy("http://proxyservername:port", true);
    MyWebService.Proxy = proxyObject;
    MyWebService.MyWebMethod();

All of the above methods apply to Web Service classes and also the WebRequest Classes

But what if there isn’t a proxy?

I discovered this week that the proxy problem is not the only cause of a web application which calls a web service throwing the exception:
[WebException: The underlying connection was closed: Unable to connect to the remote server.]

The next step I took in investigating this problem was to create two very simple test applications – one ASP.NET based like the code I was having problems with, and another a simple console application I could run as Administrator on the machine in question.

Test.aspx

<%@Page language="c#"%>
<%
System.Net.WebRequest r = System.Net.WebRequest.Create("http://av.com");
string resp = new System.IO.StreamReader(r.GetResponse()
                   .GetResponseStream()).ReadToEnd();
Response.Write("Response:");
Response.Write(resp);
%>

test.cs

using System;
using System.Net;
using System.IO;

public class Test
{
   public static void Main()
   {
      WebRequest r = WebRequest.Create("http://av.com");
      string resp = new StreamReader(r.GetResponse().GetResponseStream()).ReadToEnd();
      Console.WriteLine("Response:");
      Console.WriteLine(resp);
   }
}

Both of these make WebRequests to the Altavista search engine, and therefore tested requests out onto the Internet, returning the HTML from the Altavista homepage. As expected the ASP.NET based version gave the same exception as before, however the console application revealed not one, but two exceptions:

Unhandled Exception: System.TypeInitializationException: The type initializer for "System.Net.Sockets.Socket" threw an exception.

---> System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full
  at System.Net.Sockets.Socket.InitializeSockets()
  at System.Net.Sockets.Socket..cctor()

Which is followed by a more standard looking timeout exception

[WebException: The operation has timed-out. ]

Changing the test code to request a page from the local IIS server to no effect confirmed that it was unlikely that this was an HTTP proxy problem.

Quite a lot of searching the web, lead me towards an Microsoft Bulletin Bulletin
BUG: You receive a “The operation has timed-out” error message when you access a Web service or when you use the IPAddress class
which sounded somewhat familiar, and suggested that the problem might be caused by have more than 50 protocol bindings . Running the enum.exe utliity linked in the MS article revealed that this machine had over 100 bindings. Performing the same check on a number of other machines revealed that a more typical value was about 20, so something was not quite right with this machine. Removing some unneeded protocols from the networking setup resolved the issue, with both console application and ASP.NET test page returned the expected HTML, and most importantly the failing web service calls in the web application now works.