Web services
In this technical article, I will present the full picture of web services, basing on WCF technology over HTTP using IIS when going into details. Both beginners and experienced developers may use this article as a checklist of things they already know or want to learn.
By the full picture I mean from a need to have a web service, choosing web service type, correct configuration (e.g. security), deployment environment, to testing tools. I assume you have already created your “Hello web service” application, or just want to have a closer look at the topic.
REST vs SOAP
A web service can be SOAP-based and REST-based, and the used technology, programming language or a framework does not matter. They can’t be directly compared but the article written by the SOAP UI creators at Understanding SOAP and REST Basics And Differences will shed some light.
The often forgotten information is that SOAP protocol/standard has “versions”, so if you want to extend an old SOAP service, the possible version changes should be verified first.
Differences between 1.1 and 1.2 are nicely put in the w3.org short article From SOAP/1.1 to SOAP Version 1.2 in 9 points.
Services in the .NET Framework
The .NET Framework has four ways to create a web service:
-Web Service (SOAP) – known also as ASMX, oldest
-WCF (SOAP) – most configurable
-WCF (REST)
-Web API (REST) – lightweight, relatively new
-MVC 6 in ASP.NET Core (REST) – newest
The differences between them are explained in the short article on the dotnettricks.com Difference between WCF and Web API and WCF REST and Web Service
It is worth to mention that WCF has versions as well, and the newest one is 4.5
In addition to the pure .NET services implementation, there are frameworks like RestSharp and ServiceStack to get started quickly.
WCF and web service design – things to review
When designing a web service we have to think about how the service would be used. Do the connections to it have to be reliable? Is there a possibility for multiple requests coming at the same time? Is there a need for the server-client-server continuous communication before the connection is closed, and would that data exchange should be treated as a transaction?
When the service encounters an error, how would it be handled? What information would be sent to the client?
What is data format type and which serialization will be used? Wrong serialization attribute can make the data to not be recognized. Be sure to test the scenarios including lists of objects, nested objects, not only the “web service works” string. Will the OData (Open Data Protocol) be used?
How would the timeouts be handled by the client and the server?
Is there a WCF versioning required?
The Server
The service is going to be hosted at some point, and often using IIS. The assumption is often made that the developer using a particular technology is also able to deploy his own application, so knows the deployment environment. Administrators of the server may know nothing about IIS, or maybe the server exists only for the web service purposes, and no administrator has been assigned to the project.
The most important thing at this point is the knowledge of what elements of the environment can affect the web service.
As for the all various environments, development, testing, staging and production, the different Windows Server versions, different IIS versions, different .NET Framework versions installed, are first that can affect the behaviour. All of the environments should have the same IIS components installed, and installing them is done via “Add Roles and Features” feature.
Many issues have a root in the permissions of the deployed web service physical files and common IIS files (e.g. temp files). IIS_IUSRS group permissions may be the ones to check first.
The other permissions that may cause trouble when using Windows Authentication are Windows Account permissions, and Windows Account management overall. One of the environments having a user within a domain and the other not, password expiration policy making the authentication to fail, and so on.
The authentication can be set in many places (physical access to the files, web service being run as a certain account or pass-through, IIS Authorization rules) and it requires a detailed configuration to make it work.
The firewall can also make an IIS deployment a harder task rather than easy one, requiring to review the firewall rules and use port checker tools or the Telnet. Other services may take ports that a web service was going to use, there might be a proxy configured which redirects the traffic on its own way.
Due to all of the dependencies, it is suggested to deploy the simplest web service first (“Hello Web service”), to assess the required work at the early stages of the development. Then add needed authentication or other features.
IIS and Windows Features can be configured using cmd command line or preferably Power Shell, so there is already a space to learn how to automate the configuration.
IIS
The IIS is web apps and web services specific, so the chance anyone else than developer will know that server environment is slim. However, most used functions are at maximum 20% of all IIS possibilities.
The first thing is installing additional packages to IIS. Often it is better to download a proven IIS tool instead of developing something from a scratch. The most popular tool would be URL Rewrite, where we can use e.g. regular expressions to modify the incoming URL.
Packages can be installed from the Roles/Features at the server level (built-in packages), additional packages can be installed from the Microsoft Web Platform app or downloaded directly from the IIS website. I would suggest reviewing a list of the packages to at least know what to download when needed.
While working with IIS, always be sure that you are in the correct context/level when making any changes. There is a server level, website level, subfolder levels, and settings are inherited. If you want a security to be applied to the whole communication, configure it at a server level, so any change would not require changing all of the applications manually.
The most common issues with IIS would be applications starting slowly at first run and problems with dependencies. Both problems can be solved with Application pools, and what configuration options are possible within application pools is an important topic.
A slow start is often connected to the recycling and suspending, while dependencies are connected to the 32-bit option. The 32-bit option is useful to turn on when your 64-bit application has 32-bit dependencies and to turn off if in your application directory there are 32-bit libraries used by another application and you do not want the IIS to verify not needed files.
Web applications run within the Visual Studio IIS Express are, as you suspect, not guaranteed to work on the production.
Speaking of IIS Express, the trick is that 64-bit version has to be enabled in the Visual Studio options.
Deployment
Web services, WCF in this case, can be deployed in a few ways on the server.
The Web Deploy feature allows doing the deployment even directly from the Visual Studio to the target server. The deployment package can be created automatically, and the deployment can be made via a PowerShell script or run using WinRM. Other options are installers, the WiX Toolset has dedicated component for IIS websites configuration.
Updating the IIS configuration does not have to be a manual task, but any automation should allow going back to the previous version, at least of the web.config file!
As for the changes to the deployed application, changing the web.config file or changing .dll files in the \bin will recycle the app (recycle the pool) so it will use new configuration or libraries. It is possible to change that default behaviour e.g. for real-time applications, where reliability matters, and to recycle in the specified recycle window.
The self-hosting is a way to host a service without IIS, or even without a configuration file. It can be a console app, Windows Service or anything. It has its pros and cons.
More can be read at the Hosting and Consuming WCF Services on .NET 3.0 on MSDN and Hosting services up to IIS 7.0 on Microsoft Docs.
Security
The knowledge of certificates and authorization will become essential when at some point someone would want to add an “s” to the existing HTTP communication. Also, layers of the communication and the communication being two-sided as well. At this moment the developer should be aware how the communication between the client and the server works, at the high level, and what are the transport (point-to-point) and the message (end-to-end) layers and security configurations for them.
If the TCP communication is more internal communication, so the security design can be different.
SSL certificates
Certificates – for testing purposes (including QA, SIT and UAT) you can generate certificates on your own. Self-Signed certificates can be generated even using IIS or external tools, where more options for configuration are provided, e.g. Plularsight Self-Signed Certificate Tool.
You should know what are public and private keys, what is a difference between .cer and .pfx files, to not destroy the certificate’s value by giving the wrong key to the company connecting to your service as a client. The certificate would have to be bought again in that case, as there would be a chance where another party could be acting as a server with its certificate.
When creating applications where certificates are not managed by some administrators, the fact that certificates expire should be taken into the account. Either by having a procedure of updating all of the certificates manually or by providing a tool or configuration to manage certificates used by the application.
Imagine a situation where you hardcoded a certificate with an expiration of one year, a certificate bought by customer six months before the application went into production, application start was delayed, and finally two weeks after it has been shipped, the app stops working on HTTPS without a possibility to fix it as quickly as it should be possible.
Topics a person working with certificates should be at least familiar with are: certificate chain, wildcard certificates, browsers handling certificates differently (e.g. Chrome 58+ requiring Subject Alternative Name to be present), online SSL analyzers (e.g. SSL Labs tester), certificate storage (MMC Snap-In).
Useful tip – if your certificate is issued for the “localhost”, it will not be used for the communication with a server using its hostname.
As for .NET handling the certificates in the code, there is a possibility to use a custom certificate validation. When your .NET client connects using SSL with a server, the certificate server is presenting can be validated by a custom method which does nothing, for testing purposes. I will not recommend that approach ever, as it is easy to “forget” that the client does not validate the certificate and deploy that solution into production.
It may be useful to know the relation between X509Certificate and X509Certificate2 classes.
Authorization and authentication
After setting the secured transport channel, the next most common step is adding the message security, client credentials. The mix of transport and message gives most performance and security.
The server shows its ID (certificate), the client is checking the ID and sends the request. If we want anyone to connect to our web service, we can use the “Anonymous” authentication. However, if we want to control who is connecting to our web service, the client should have an ID as well.
IIS gives multiple authentication methods to choose from, and I will describe two of them shortly.
Basic authentication – client sends its credentials (username, password) in a plain text, and web service uses custom validation to check if those credentials match.
Windows authentication – Windows account is used as a storage of the username and password. Depending on the used negotiation mechanism (e.g. Kerberos, NTLM), credentials are encrypted and not sent with the first request (thus negotiation). This is something browsers may do not like.
Using Windows authentication is fast (how much time is needed to configure it) at the beginning (no additional code is required at the web service level), but can be problematic depending on the environment. Correct permissions should be set for the user to have access to processes and files and mentioned in this article password policies.
Kerberos (“Windows” in the web.config) is more secure than NTLM and should be used over NTLM if possible.
Access to the service can be also restricted to certain users at a server level, using the URL Authorization module. Additional restrictions can be made to limit requests to certain IP addresses (IP filtering is done before an authentication).
The security can be also enhanced by clients having their certificates, and those certificates being validated by the web service (server). It can be done in addition to the username-password authentication. Tokens could be used instead of username and password combination as well.
The interesting fact is that the POST, PUT methods have authentication set as default on IIS when sent with HttpWebRequest class – as the POST or PUT may fail article on Microsoft Support platform explains.
The web.config
The configuration file is very powerful and before deciding to use a certain option, all of the possibilities should always be reviewed.
Other benefits of the file are its .debug and .release transformations, making the development easier and with fewer mistakes of a wrong configuration being deployed.
As many configuration files within the .NET world, the web.config is also configurable via a GUI. Changes made in the IIS are affecting the web.config file, and changes made in the web.config file can be seen in IIS. Not all elements are configurable that way, but e.g. Authorization and URL Rewrite modules are.
The web services and clients can be created programatically, so most of the configurations can be done via code. This approach has advantages of static typing, Intellisense helping with choosing correct options, and more.
The possibility to write the web.config in the C# code has the potential of confusion – often there are client applications with autogenerated web service code from the WSDL, configurations present in their app.config files, but the client is written from the scratch in C#, making the saved configuration a dead code. Sometimes the configuration is used partially in the C# code, making this even weirder.
The one particularity important configuration aspect should be analyzed during the design phase of the web service: “readerQuotas”. During development, web service could be taking just “hello-webservice” data all the time, but for production purposes, the data send in one request could be very large, exceeding the default values.
Testing the solution
For the development smoke testing purposes, the built-in WcfTestClient can be enough to determine if the web service starts.
Later, the test client should be developed for the connectivity debugging (and included in the solution, even if you just develop the service), and third-party client used for data requests. I would recommend SOAP UI (which handles both SOAP and REST), whose test projects should be saved somewhere.
Unit and integration tests are always a blessing with web services, as they free the testing from the configuration hassle.
Having SOAP UI projects saved, automated tests written, and test client developed, you are ready if any issue with the web service arises.
Testing with the third party client and own client is important to determine if the web service issue is coming from the connectivity (the simplest C# app would verify that) or wrong client configuration.
Assuming you are writing a service and another company writes a client, when something does not work you have to prove that the service actually works and their client in the wrong. Then the SOAP UI or another independent client can be used.
However the client another company installed can have another version (with bugs not fixed), is configured differently, but when your SOAP UI works and their is not working, that is a start. The fact that the simple C# client works do not prove anything, as the other company may be using a Java or another technology, where e.g. credentials are set differently, the technology is not so integrated with WCF and Windows.
If the web service is not required to run in a browser, it should not be bothered. Browsers are different, they have different versions and different handling of e.g. certificates or Windows credential negotiations.
While testing connectivity, the Telnet functionality and Port Checking tools (e.g. online port checkers) will be useful.
Troubleshooting
There are a few general tips to use while working with web services:
-use the step-by-step approach, never change more than one thing before trying if something works
-test on different clients and different machines
-error messages may be misleading
-try to deploy the application on testing/staging environments as early as possible (“Hello web service” version)
Resources
Developing Service-Oriented Applications with WCF by Microsoft
IIS documentation by Microsoft
Pluralsight offers a couple of WCF, IIS and Web API courses. Some of them may feel outdated (e.g. old IIS versions) but overall concepts remain the same.
WCF Design Concepts
WCF End-to-End
WCF For Architects
WCF Power Topics
WCF 4 New Features
IIS for Developers
IIS Administration Fundamentals
IIS Administration in Depth
Introduction to the ASP.NET Web API
Leave a comment