SysAdmin
Archived Posts from this Category
Web Service Connection Problems
Posted by Chris Alcock on 06 Feb 2007 | Tagged as: .NET, C#, Development, IIS, SysAdmin, Web Services
Setting the scene
Web Services are very simple to consume using .NET code, and its all to easy to forget what is actually going on when you add that Web Reference to your project in Visual Studio. Once you’ve entered the URL for you web service and clicked the Add Reference button, Visual Studio requests the service description WSDL file, and code generates classes to represent the data and the web service methods. I found delving into this generated code taught me quite a lot about the way XML serialization and web services actually work.
Behind the scenes the web service proxy uses classes in the System.Web.Services.Protocols namespace to actually perform the calls to the service, and these calls end up as System.Net.WebRequests containing the correctly encoded data that makes up the message.
Problems with Web Requests
Quite a lot of the problems with web services I encounter during deployment are actually problems with the System.Net.WebRequest. The most common cause of problems seems to be problems where web access is provided via some form of HTTP Proxy Server, and this typically results in an exception of the form:
[WebException: The underlying connection was closed: Unable to connect to the remote server.]
If you have access to the machine running your application its a very good idea to check if the machine can make a request to your Web Service endpoint using a web browser. This also allows you to check the settings to see if web traffic is passing through a proxy – failing this, check with the people responsible for the network.
If you find that there is a proxy involved, there are a couple of strategies available to resolve the problem.
-
Make Changes in
machine.configto affect all applications
machine.Config is one of the .NET framework’s main configuration files, and can be found in the framework directory. Settings in thedefaultProxysection with that attributeusesystemdefault="false"allow system wide setting of a default proxy server overriding an OS setting:
usesystemdefault = "false"
proxyaddress="http://proxyserver:port"
bypassonlocal="false"
/>
-
Configuration change for your application only
If you don't control the machine your application is running on then its unlikely you will be able to make changes to machine.config, but thankfully the above section will also work in yourweb.configorapp.confgfiles -
Programmatic Solution
You can set the proxy settings in code:using System.Net;
MyWebService ws= new MyWebService();
WebProxy proxyObject = new WebProxy("http://proxyservername:port", true);
MyWebService.Proxy = proxyObject;
MyWebService.MyWebMethod();
All of the above methods apply to Web Service classes and also the WebRequest Classes
But what if there isn't a proxy?
I discovered this week that the proxy problem is not the only cause of a web application which calls a web service throwing the exception:
[WebException: The underlying connection was closed: Unable to connect to the remote server.]
The next step I took in investigating this problem was to create two very simple test applications - one ASP.NET based like the code I was having problems with, and another a simple console application I could run as Administrator on the machine in question.
Test.aspx
<%@Page language="c#"%>
<%
System.Net.WebRequest r = System.Net.WebRequest.Create("http://av.com");
string resp = new System.IO.StreamReader(r.GetResponse()
.GetResponseStream()).ReadToEnd();
Response.Write("Response:");
Response.Write(resp);
%>
test.cs
using System;
using System.Net;
using System.IO;
public class Test
{
public static void Main()
{
WebRequest r = WebRequest.Create("http://av.com");
string resp = new StreamReader(r.GetResponse().GetResponseStream()).ReadToEnd();
Console.WriteLine("Response:");
Console.WriteLine(resp);
}
}
Both of these make WebRequests to the Altavista search engine, and therefore tested requests out onto the Internet, returning the HTML from the Altavista homepage. As expected the ASP.NET based version gave the same exception as before, however the console application revealed not one, but two exceptions:
Unhandled Exception: System.TypeInitializationException: The type initializer for "System.Net.Sockets.Socket" threw an exception.
---> System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full
at System.Net.Sockets.Socket.InitializeSockets()
at System.Net.Sockets.Socket..cctor()
Which is followed by a more standard looking timeout exception
[WebException: The operation has timed-out. ]
Changing the test code to request a page from the local IIS server to no effect confirmed that it was unlikely that this was an HTTP proxy problem.
Quite a lot of searching the web, lead me towards an Microsoft Bulletin Bulletin
BUG: You receive a "The operation has timed-out" error message when you access a Web service or when you use the IPAddress class which sounded somewhat familiar, and suggested that the problem might be caused by have more than 50 protocol bindings . Running the enum.exe utliity linked in the MS article revealed that this machine had over 100 bindings. Performing the same check on a number of other machines revealed that a more typical value was about 20, so something was not quite right with this machine. Removing some unneeded protocols from the networking setup resolved the issue, with both console application and ASP.NET test page returned the expected HTML, and most importantly the failing web service calls in the web application now works.
When APIs evolve – or how I lost my lunchtime by installing the .NET 2.0 Framework
Posted by Chris Alcock on 17 Jan 2007 | Tagged as: .NET, COM Interop, IIS, SysAdmin
In the presentation about Good API Design I talked about in yesterdays post one of the key points made was that once an API is defined you should never make changes to it that will break your client’s code. An example cited throwing exceptions based on values previously considered fine.
As luck would have it I encountered an actual example of precisely this problem today while installing the .NET 2.0 Runtime on a development server. This server runs a number of .NET 1.1 applications and a number of classic ASP applications consuming COM components written in .NET 1.1.
Things didn’t start well, with the framework installer stopping the IIS instance for the better part of 10 minutes while installing, however it did restart it again once it was done (unlike MSDTC and SQL Server when installing anything from the Windows Components section of Add Remove Programs on Windows 2003).
Matters got worse when someone mentioned that one of the components on the server was now misbehaving – specifically one that uses the ASP.NET Cache to provide caching capabilities.
Whenever a web application tried to create this object (via Server.CreateObject) it was getting an invalid pointer error. Other COM components developed in a similar way were working fine, so I assumed there was something wrong with the registration of the component. Un-registering and re-registering the component gave no joy – neither did calling it from a simple VBScript file.
To make matters worse, a simple .NET test application was working just fine using the exact same library.
After a bit of head scratching and pondering the SysInternals (Now a part of Microsoft) Process Explorer revealed that instead of using the .NET 1.1 version of System.Web both CScript and the IIS DLLHost were loading the .NET 2.0 version. The code for the component hadn’t changed, so maybe the .NET framework had.
Loading the source code for the component into Visual Studio 2005 and attempting to compile and run a the simple test application revealed the problem, a Null Reference Exception from within the framework.
As the COM Component was using the ASP.NET System.Web.Cache it was creating a HTTP Context instance internally. This code looked like this:
private System.Web.HttpContext context = new System.Web.HttpContext(null);
Poking round the disassembled code of System.Web in Reflector didn’t reveal what it was that was causing the exceptions, although I did only go a few functions deep, however it did reveal an alternative way of getting to the cache.
Changing our code to use a call to System.Web.HttpRuntime.Cache to obtain the cache instance fixed our problem, and a quick rebuild of the component against .NET 2 and redeploy to the server and we were back up and running.
Lessons learned from all this:
- The .NET Framework installer will stop IIS and keep it stopped for a large part of the install – useful to know considering I’ll be installing it on some production servers soon
- Both IIS and CScript seem to run all .NET COM Components through the most up to date version of the .NET framework, regardless of the version the component is registered or compiled with
- .NET applications (like our test applet) will run in the .NET framework version they were compiled against if available
Internet Information Services Diagnostic Tools
Posted by Chris Alcock on 11 Jan 2007 | Tagged as: Development, IIS, SysAdmin
When investigating performance problems on production servers it is always very useful to have as much information about the actual work that the server is performing at any given time. Out of the box IIS 6 does not give you much to work with – at best you can identify the virtual host that is causing the issues by putting it into its own application pool, and then using Task Manager you can see PID of the w3wp.exe instance which is occupying your servers CPU. Once you have that, the iisapp.vbs administrative script will reveal the name of the application pool which is misbehaving.
I have often wished to be able to see in real time what is actually going on within the worker process or IIS instance, and with the discovery of the Internet Information Services Diagnostic Tools trace tools I have found something that comes pretty close to what I would like.
The IIS trace tools contain a command line tool that will return the details of the executing requests as XML (even obtaining the information from a remote IIS server), however the jewel in the crown is the Request Viewer – a window app that with the click of a tool bar button reveals the requests currently executing on the server. (see screen shot below)
Unfortunately the tool does not show you the name of the host that the requests relate to, just the site ID and application pool pid, but these are easily converted into the application pool name (as mentioned above, use iisapp) or the site (look the is up in the IIS Manager.
Another problem with the request viewer is that when I first ran and clicked the refresh now button it all I got was an error and no details of the requests currently running. Thankfully I found the solution on the web, and it was as simple as making sure the temp environment variable was set to a path that didn’t use long filenames.
As the IIS viewer leaves a command window in the background when it runs, I thought that the best solution would be to have a simple batch file that set the environment up, ran IISApp.VBS and then started the Request Viewer:
@echo off
set temp=c:\temp
iisapp
"C:\Program Files\IIS Resources\TraceDiag\reqviewer.exe"
So now that empty command window contains the PID values for all my application pools so its not wasting space in my RDP window 🙂
