NOAA Weather Service and .NET

I have been fighting with the national weather web service, provided by the National Oceanic and Atmospheric Administration, today.  Like a lot of developers, who like to make their web sites fancy with the current weather forecast, I have been developing a component for the web site I am working on.

NOAA seems to be the norm for providing weather data in XML via the SOAP web service protocol.  Up until a couple of weeks ago, the C# code that I had to connect to the NOAA web service and pull weather data, was working  nicely….then it stopped – bah.  The proxy code (generated from the service WSDL by .NET) kept throwing an HTTP protocol violation exception:

Unhandled Exception: System.Net.WebException: Error getting response
stream (Rea dDone2): ReceiveFailure ---> System.Exception:
at System.Net.WebConnection.HandleError ()
at System.Net.WebConnection.ReadDone ()
at System.MulticastDelegate.invoke_void_IAsyncResult ()
at System.Net.Sockets.Socket+SocketAsyncResult.Complete ()
at System.Net.Sockets.Socket+Worker.Receive ()
at System.MulticastDelegate.invoke_void ()
in <0x000ad> System.Net.WebConnection:HandleError (WebExceptionStatus
st, System.Exception e, System.String where)--- End of inner exception
stack trace ---
in <0x00158> System.Net.HttpWebRequest:EndGetResponse (IAsyncResult
asyncResult)in <0x00047> System.Net.HttpWebRequest:GetResponse ()
in <0x0004d>
System.Web.Services.Protocols.WebClientProtocol:GetWebResponse (Sys
tem.Net.WebRequest request)

It seems that NOAA changed something involving their service protocol, because the response to a SOAP request is now being returned in separate chunked HTTP responses, which causes the .NET SOAP client to puke.

After some digging on the Internet (lots of digging in fact), I came across a post on a Mono newsgroup, which gave me a work around to the problem.  The trick is to change the web request in the SOAP proxy to use HTTP version 1.0 instead of 1.1.  Since HTTP 1.0 does not support chunked response, the NOAA server sends the response data as one block, which keeps the .NET SOAP client happy. 

Below is the code for my web service proxy, drop it into your project (assuming you use C#) and instantiate the WeatherProxy class from your code.  You’ll not need to add a web reference to your project because this proxy class is attributed with the correct SOAP envelope information.

///
/// Soap Format Type.
///
[SoapTypeAttribute(“formatType”, “http://weather.gov/forecasts/xml/DWMLgen/schema/ndfdXML.xsd”)]
public enum formatType
{
///
/// 24 Hour
///
[SoapEnumAttribute(“24 hourly”)]
Item24hourly,

///
/// 12 Hour
///
[SoapEnumAttribute(“12 hourly”)]
Item12hourly,
}

///
/// Proxy class for weather service.
///
///
/// Uses the NOAA web service, this proxy derived from an example found on the .NET. The proxy that
/// VS2005 generates causes a protocol error/HTTP 502 Bad Gateway.
///
[DesignerCategoryAttribute(“code”)]
[WebServiceBindingAttribute(Name = “ndfdXMLBinding”, Namespace = “http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl”)]
public class WeatherProxy : System.Web.Services.Protocols.SoapHttpClientProtocol
{
#region Construction

///
/// Constructor.
///
public WeatherProxy()
{
this.Url = “http://www.weather.gov/forecasts/xml/SOAP_server/ndfdXMLserver.php”;
}

#endregion Construction

#region Methods

///
/// Get the weather data by day (synchronously).
///
/// Latitude coord.
/// Longitude coord.
/// Start Date.
/// Number of days.
/// Time format.
/// Soap XML
[SoapRpcMethodAttribute(“http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl#NDFDgenByDay”, RequestNamespace = “http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl”, ResponseNamespace = “http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl”)]
[return: SoapElementAttribute(“dwmlByDayOut”)]
public string NDFDgenByDay(
Decimal latitude, Decimal longitude,
[SoapElementAttribute(DataType = “date”)] DateTime startDate,
int numDays, formatType format)
{
object[] results = this.Invoke(“NDFDgenByDay”, new object[] {
latitude,
longitude,
startDate,
numDays,
format});
return ((string)(results[0]));
}

/// Creates a for the specified uri.
/// The .
/// The to use when creating the .
/// The uri parameter is null.
protected override WebRequest GetWebRequest(Uri uri)
{
HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(uri);
request.ProtocolVersion = HttpVersion.Version10;
return request;
}

#endregion Methods

}

34 thoughts on “NOAA Weather Service and .NET

  1. Ed Mooers

    Yikes! You’re code formatter really munged up those XML comments! Still, it looks good. I’ll have to give this a try over the weekend. Great job!
    <br>
    <br>Ed
    <br>
    <br>PS: Maybe you could create a little download link for code like this? 😉

  2. Rob Garrett

    Yeah, the code syntaxer is a little too smart for it’s own good, I really need to find an alternative tool, or to fix it.
    <br>
    <br>I was going to put the code on download but was in haste this afternoon (mainly because I’d spent all afternoon spitting at NOAA). I can put the file on line if you’d like it.

  3. http://

    Any idea how a lowly VB programmer can use this code? I’ve translated it into VB (using online tools) and can’t seem to get it to parse and/or work.
    <br>
    <br>GRRRRRRRRRRrrrrrrrrrrrrrrrrrrr is right!

  4. Rob Garrett

    NOAA broke their service, see my latest post…
    <br>
    <br><a target=”_new” href=”http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx”>http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx</a&gt;
    <br>
    <br>1. Is your VB code compiling?
    <br>2. I would use the Weather.com service on account of NOAA’s service being unreliable.
    <br>3. If the translation of C# to VB.NET is your issue, can you not just create a class library with my code and call it from your VB.NET code?

  5. David Berman

    Could you please provide the file with the source code for this? Do you have this working for ASP.NET 2.0 / Visual Studio 2005? Thank you for sharing this information.

  6. David Berman

    I have this compiling now using Visual Studio 2005. The trick was to paste your code into a new class file, delete the line numbers from the left side, and add the following namespace references:
    <br>
    <br>using System.Net;
    <br>using System.Xml.Serialization;
    <br>using System.Web.Services;
    <br>using System.Web.Services.Protocols;
    <br>using System.ComponentModel;
    <br>
    <br>However, now instead of the protocol violation error I get a different error:
    <br>The request failed with HTTP status 502: Bad Gateway
    <br>This comes from line 62:
    <br> object[] results = this.Invoke(&quot;NDFDgenByDay&quot;, new object[] {
    <br>
    <br>Any ideas?

  7. Rob Garrett

    A word of warning – the last time I looked at NOAA’s weather web service, they had broken the SOAP interface (<a target=”_new” href=”http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx”>http://robgarrett.com/blogs/rant/archive/2006/02/14/1847.aspx</a&gt;
    <br>). My advice would be to sign up with <a target=”_new” href=”http://weather.com”>http://weather.com</a&gt; and use their service instead.
    <br>
    <br>
    <br>BTW, if you click &quot;View Plain&quot; in the top right of the code window you do not have to remove the line numbers from my code yourself.

  8. Stephen

    This helped me out a ton!!!! Had this same problem and were it not for you, those guys down at the NWS would have received some floggings.

  9. robgarrett

    Jorge,
    This code should work the same in a winforms application. The code makes a web request to the SOAP weather service, which provides data as XML (when it is working). As long as you add a reference to System.Web to your project (for the web client) you should be good to go.

    R.

  10. http://

    Rob,

    Nice work. I found that if you insert the HTTP1.0 override into the WSDL web reference generated code, it starts to work.

    It strikes me that this is a more a .NET 2.0 missing feature that a NOAA problem.

    Henry

  11. http://

    Great tip! I took this a bit further, incorporating inheritance. Here’s my sollution (VB.NET)

    Public Class WsCustomProxy
    Inherits WsNamespace.WSProxy
    Protected Overloads Overrides Function GetWebRequest(ByVal Uri As Uri) As System.Net.WebRequest
    Dim req As HttpWebRequest = CType(MyBase.GetWebRequest(Uri), HttpWebRequest)
    req.ProtocolVersion = HttpVersion.Version10
    Return req
    End Function
    End Class

  12. http://

    Just tried to get the NDFDgen() method working using the same trick, but it generates the “Bad Gateway” exception. Not sure why NDFDgenByDay works and this doesn’t…

    Any ideas ?

    Henry

  13. http://

    The service seems to return a 502 error when you pass it bad parameters. I was calling NDFDgenByDay and finally got it to work by passing the longitude value as a negative number. (In addition to using the http 1.0 workaround discussed above)

  14. http://

    Had any luck with NDFDgenAsync?

    I’ve got:

    Private Sub NOAA_Completed(ByVal sender As Object, ByVal e As NOAA.NDFDgenCompletedEventArgs) Handles xWeather.NDFDgenCompleted
    Try
    If e.Cancelled Then

    ElseIf Not e.Error Is Nothing Then

    Else
    sWeather = e.Result
    End If

    Catch ex As Exception
    ErrorMsg(ex)
    End Try

    End Sub

    But it never seems to fire off the event…

  15. http://

    I am consuming the web service in windows app, Visual Studio 2005, C# and getting the exception –
    {“The server committed a protocol violation. Section=ResponseStatusLine”}

    What can I do to resolve this?

    Thanks
    Pooja.

  16. http://

    As other people have said, throwing this in Reference.cs file(in the main class):
    Using System.Net;
    …………
    protected override WebRequest GetWebRequest(Uri uri)
    {
    HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(uri);
    request.ProtocolVersion = HttpVersion.Version10;
    return request;
    }

    Worked for us using 2.0

  17. http://

    I struggled with this for a bit and finally put together a (good) working solution… I finally figured this out using so many sources it’s hard to give credit but this idea isn’t mine, just the implementation.

    THE REAL ASP.NET 2.0 SOLUTION:

    I fought this for a while and finally got the right code to fix this problem in ASP.Net 2.0. As many of you know, ASP.Net 2.0 generates the proxy classes dynamically so you can’t just simply edit them. The key is in a new feature, ‘partial classes’. Below is everything you need to know to fix this in ASP.Net 2.0.

    First, my web reference declaration is:
    weather.ndfdXML

    The standard instantiation would then look like:
    weather.ndfdXML2 wxService = new weather.ndfdXML2();

    With this setup, I ran into the exact same problem everyone else has written about here and rarely solved.

    First, add a new class and make it a partial class (C# code posted):
    using System;
    using System.Net;
    using System.Web.Services.Protocols;

    namespace weather
    {
    public partial class ndfdXML2 : weather.ndfdXML
    {
    protected override System.Net.WebRequest GetWebRequest(Uri uri)
    {
    //throw new Exception(“Custom WebRequest override code hit!!”);

    System.Net.HttpWebRequest webRequest = (System.Net.HttpWebRequest)base.GetWebRequest(uri);
    webRequest.KeepAlive = false;
    webRequest.ProtocolVersion = HttpVersion.Version10;
    return webRequest;
    }
    }
    }

    You can optionally uncomment that new exception to verify that the code is indeed executing.

    Second, change you instantiation inside your code to the following:
    weather.ndfdXML2 wxService = new weather.ndfdXML2();

    Now everything works great!!

  18. http://

    Thanks for the walkthrough code sample, Craig Scheets. I converted it to VB and I finally got it working. Everyone else, thanks for the discussion and learning points.

  19. http://

    Thank you so much for this solution. I was easily able to get the rest of the Methods working by pasting them from the Web Reference into the WeatherProxy class. Now you can access the Asynchronous download methods as well!

  20. http://

    Getting Error : Unable to read data from the transport connection

    I am using this same web service. NDFD service for wether data : http://www.weather.gov/forecasts/xml/

    First i got error : underlaying connection closed.

    its resolved after i puts

    protected override System.Net.WebRequest GetWebRequest(Uri uri)
    {
    System.Net.HttpWebRequest webRequest =
    (System.Net.HttpWebRequest)base.GetWebRequest(uri);
    webRequest.ProtocolVersion = HttpVersion.Version10;
    webRequest.KeepAlive = false;
    return webRequest;
    }

    But now in some cases its gives me error:

    Unable to read data from the transport connection. Connection closed.

    Plz.. can any one help me on this?

    I m using ASP.Net 2.0

  21. http://

    Thanks Craig for debugging that problem. I thought I’d post the VB. I added the web reference using http://www.weather.gov/forecasts/xml/SOAP_server/ndfdXMLserver.php?wsdl and then changed the default namespace VS2005 came up with from ‘gov.weather.www ‘to just ‘weather’.

    Namespace weather
    Partial Class ndfdXML2
    Inherits ndfdXML

    Protected Overrides Function GetWebRequest(ByVal uri As Uri) As System.Net.WebRequest

    Dim oWebReq As System.Net.HttpWebRequest

    oWebReq = MyBase.GetWebRequest(uri)

    oWebReq.KeepAlive = False

    oWebReq.ProtocolVersion = System.Net.HttpVersion.Version10

    Return oWebReq

    End Function

    End Class
    End Namespace

  22. OmegaMan

    I recently consumed the NOAA web service and began receiving the problems you had. I wanted to continue to use the proxy class which was generated by Microsoft. I simply took your override of GetWebRequest and placed in the main class of my reference.cs hidden proxy file. I have blogged here ( http://www.omegacoder.com/?p=40 ) and gave this page as a link of thanks.

  23. http://

    Just wanted to send a thank you, your code helped me with a similar problem I’ve been having the last couple of days 🙂

    Warm regards from Stockholm, Sweden.

  24. http://

    When you say proxy, do you mean your service class that inherits from System.ServiceModel.ClientBase? If I put that override there, the compiler says that there is no suitable method to override. I know this solution will work, I just need to know where exactly to implement it.

  25. http://

    Anyone have a VB.NEt version of this? And how can I make this dynamic? Send the lat long to get the weather from different places? Thanks kindly for your time.

  26. http://

    Thaks for Rob and JacDev, who write translated code. This helps me in one project.
    Greateengs from Lithuania

  27. http://

    Do you know if NOAA changed something recently? I was consuming their web service fine and then one day I started getting the “the server committed a protocol violation” error. I made the changes to the Reference.cs as above and it seemed to fix the problem. It is just odd that it worked yesterday and all the sudden it does not work today.

  28. Robert Bunge

    FYI, the NOAA feed will appear to toggle back and forth because when we either have systems problems or some extreme weather event starts to over load either our server farm(s) or available bandwidth, we switch to a Content Delivery Network. Both of the CDN’s we’ve worked with want to send the chunked data. It’s been awhile since I’ve asked them if they have a work around, if time avails, we’ll try again.

  29. inf

    getting following error when i host this in IIS
    “An existing connection was forcibly closed by the remote host “

  30. NCBuyer

    WHY on earth do people insist on using XML – its beyond me. Its SO darn cumbersome and bloated. Why not just make a straight API with black-box calls into a live system. Having worked with mainframe databases most of my early programming life, this XML stuff really doesnt make a darn bit of sense.

Comments are closed.