I am looking at various testing scenarios on the new Windows Server 2008 operating system. The first step is to install Server 2008 and I ran into an interesting issue with device drivers today. First of all, I decided to follow the installation ReadMe advice and install Server 2008 onto a second partition on a machine that already has one older OS. So I grabbed an old (circa 2003) Dell Optiplex GX270 desktop machine with 1.5GB of RAM and a 60GB hard drive. I popped a bootable Windows Server 2003 disk into the machine and launched its install. I configured the drive into a 20GB partition (for Server 2003) and left the rest, about 40GB, for Server 2008. I installed Server 2003 as normal, specifying all kinds of information such as Admin password, and so on. Next I copied the Server 2008 files onto the machine, and while in Sever 2003, double-clicked on the Server 2008 setup.exe file. I aimed the install at the 40GB partition. Server 2008 installation is stunningly simple and proceeded without any problems. With Server 2008, you specify most information (machine name, administrator password, and so on) after installation rather than as part of the installation process. See the image below for the initial configuration dialog. After Server 2008 installed however, my screen resolution was stuck at an ugly 640x480. The old Dell machine has an old Intel 82865G Graphics Controller that Server 2008 didn't recognize. I churned for a couple hours searching the Internet for either a Server 2008 or even Vista driver (the idea being that Server 2008 shares much of the Vista codebase). But there is no such graphics driver. Ameet Chitre, a senior manager in the Windows group, showed me what to do. In essence, Server 2008 can use an old driver, so the trick is to swindle Server 2008 into installing an old driver not designed for Server 2008. Basically you can't directly run the driver setup program, but you can get Server 2008 to accept the driver's underlying .inf file. This is a general principle: when installing Server 2008 on an old machine, find old device drivers but trick Server 2008 into installing them by pointing directly at the associated .inf files. It's one of those things that seems obvious in retrospect. Here are the specific steps I took to install the 82865G Graphics Controller driver:

 

Download file R126991.exe from the Dell drivers page. Double click on the R126991.exe file which will extract it to the machine. Inside the R126991 folder there'll be a setup.exe but it will not install the driver -- the setup will throw a "wrong OS" error message. The trick is to go to Computer | Manage | Diagnostics | Device Manager | Display Adapters. Right-click on the current VGA entry and select "Update Driver Software". Then select the "Browse my computer for driver software" option (but not the "Search automatically" option which will throw a "the best driver is already installed" error) and then select the "Let me pick from a list of device drivers on my computer" option (but not the Browse button which will let you point to a directory but not at an .inf file which is what we need). Now select the "Have Disk" option, and then Browse to the ialmnt5.inf file in the win2000 subdirectory of the R126991 directory. Click Open and the old driver will be forcibly installed. After a reboot, the Windows Server 2008 machine now has pretty nice graphics.

 

This past week I just finished writing up an article for MSDN Magazine (http://msdn.microsoft.com/msdnmag/) that introduces the basic techniques and principles of testing Windows Communication Foundation (WCF) services. You can think of WCF services as a way to create distributed systems; or put another way, you can think of WCF as similar to Web Services but with much greater flexibility and features. Consider a WCF service which has operations that perform MD5 crypto-hashing. A crypto-hash accepts any number of bytes, which in turn can come from any type of information including text, and returns a fixed size identifying value in the form of an array of bytes. An MD5 (Message Digest version 5) crypto-hash is one of many specific crypto-hashing methods and it accepts any number of bytes and returns a 16-byte array. SHA1 (Secure Hash Algorithm) is another common crypto-hash and it returns a 20-byte array. Suppose you have several applications (perhaps Web sites and network-connected applications) that need to compute crypto-hashes. You could simply place the crypto-hash functionality into each application. Alternatively, you could use a distributed application approach (which you can also call a Service Oriented Architecture [SOA] approach) and create a single WCF service which has a crypto-hash operation. Then all you applications could call into the one server machine which hosts your WCF service. OK, so how do you test your WCF service? The most basic form of WCF testing is to verify the functional correctness of your WCF operations. One way to do this is to programmatically send a message to your WCF operation, fetch the response message, and compare the response with an expected result to determine a pass/fail result. See the screenshot below for an example. In essence the test harness is just a specialized WCF client program. In my MSDN Magazine article, which I expect to be published within about three months or so, I explain all the details of performing basic WCF testing. I also describe WCF testing scenarios that go beyond basic functionality verification, such as testing WCF security features and testing WCF transmission protocol variations. An interesting side effect of learning how to write test automation for WCF services is that the process really helps you better understand how to create and use WCF services. 

 

The new Microsoft UI Automation (MUIA) library can automate both Windows applications and Web applications. However there is painfully little documentation on exactly how to perform test automation using MUIA. When writing Web application UI test automation, you first launch the Web app under test, then get references to the user controls of interest (such as text box controls), then manipulate the controls (add text, click buttons, etc.), and then examine the resulting state of the Web app's controls to see if they match expected values to determine a pass/fail result. All these tasks are somewhat difficult but I usually have most trouble when trying to get references to the user controls. The image below comes from a little experiment program I wrote and summarizes a few of the key points about using the MUIA library to automate Web apps. The experiment program examines every AutomationElement (a MUIA concept) on a dummy Web app named MiniCalc. The program found a total of 80 elements. At the top level is the Internet Explorer browser itself, and it has a handle (a Windows concept) value of 280BD4. The IE browser has child Automation Elements, some of which are physical windows/controls (and therefore have handle values), but some child elements are just virtual AutomationElement objects (and there get a 0 window handle value). In the screenshot, notice that I used the Spy++ tool to examine the main client area in the browser (the turquoise colored body). The client area has handle value 1D1148. Now in the console app, notice that all the user controls are subchildren of the main client area, but they are not window controls (because they have no Window handle values). This is why using classical UI Windows control automation methods simply don't work for Web applications -- the user controls in a Web app are not Windows control windows. As a final observation, notice that the AutomationElement hierarchy for a Web application is moderately complex, and getting references to user controls is tricky because many user controls do not have Name properties, and there can be duplicate Name properties. I suspect that further investigation on my part will reveal that the most practical way to develop UI automation for Web applications is to make sure that developers assign to every user control, some property (like AccesssibleName perhaps) that can be accessed by MUIA to get an unambiguous reference to the control.

 

 

Here is the source code for the experiment program:

==

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using System.Windows.Automation; // MUIA; add refs to UIAutomationClient.dll & UIAutomationTypes.dll
using System.Diagnostics; // for Process

namespace Run
{
  class Program
  {
    static void Main(string[] args)
    {
      try
      {
        Console.WriteLine("\nBegin MUIA control analysis");
        Process p = System.Diagnostics.Process.Start("iexplore.exe", "http://localhost/MiniCalc/Default.aspx");

        System.Threading.Thread.Sleep(3000);

        AutomationElement aeDesktop = AutomationElement.RootElement;
        AutomationElement aeBrowser = AutomationElement.FromHandle(p.MainWindowHandle);
       
        List<Info> list = StoreElements(aeBrowser);
        Console.WriteLine("List has " + list.Count + " Info objects\n");

        foreach (Info info in list)
        {
          string s = " Name = " + info.Element.Current.Name + ", handle = " + info.Element.Current.NativeWindowHandle.ToString("X") + ", level = " + info.Level;
          Console.WriteLine(s.PadLeft(s.Length + info.Level * 2, ' '));
        }

        Console.WriteLine("\nEnd");
        Console.ReadLine();
      }
      catch (Exception ex)
      {
        Console.WriteLine("Fatal: " + ex.Message);
        Console.ReadLine();
      }
    } // Main()

    class Info // utility class to add a Level to an AutomationElement
    {
      public readonly AutomationElement Element;
      public readonly int Level;

      public Info(AutomationElement element, int level)
      {
        this.Element = element; this.Level = level;
      }
    } // class Info

    static List<Info> StoreElements(AutomationElement browserElement)
    {
      List<Info> results = new List<Info>(); // final results. Info is an AutomationElement and a Level
      Stack<Info> stack = new Stack<Info>(); // helper stack of Info objects
      stack.Push(new Info(browserElement, 0)); // the browser is at level 0

      // do a preorder, non-recursive traversal
      while (stack.Count > 0)
      {
        Info current = stack.Pop(); // get current AutomationElement info
        results.Add(current);       // add to the results List<>
        AutomationElementCollection subchildren = current.Element.FindAll(TreeScope.Children, Condition.TrueCondition);
        foreach (AutomationElement child in subchildren)
        {
          stack.Push(new Info(child, current.Level + 1));
        }
      }
      return results;
    } // StoreElements

  } // class Program

} // ns Run   

==

The topic of the low prestige of software testing (compared to software development and management) at Microsoft comes up in my conversations here in Redmond, WA all the time. It is an important topic because Microsoft has great difficulty attracting and then keeping top talent in software testing roles. The issue is relatively complex and can't be fairly addressed in the short space of a blog entry but here are brief sketches of 12 opinions that are part of the software testing perception issue. These opinions are all tightly related and in many cases different views of the same idea. Furthermore, I don't agree with all of these issues -- but they're all issues I hear repeatedly, and each idea has a strong counterargument which I'll address in some future blog entry.

 

1. Lower pay. Software testers at Microsoft do in fact receive on average lower pay than software developers with similar experience.

 

2. Self-fulfilling prophecy. Because testing is perceived as lower prestige, it does not appeal to top talent, which in turn lowers the field's prestige.

 

3. Weak testers can hide. Software testers have fewer directly measureable work outputs than developers so weak or even incompetent testers can hide within a work environment for quite some time.

 

4. Weaker technical skills. At Microsoft, prestige is most closely related to ability to code. Most testers have weaker coding skills than developers. Testers, as a whole, are sometimes seen as failed developers.

 

5. Degree of separation from product. Compared to software development, testing is at least one degree of separation removed from Microsoft's core activity -- which is software development.

 

6. Explicit career path. Software testers have fewer direct career path options than developers.

 

7. Implicit career path. Senior technical management (VP and above) at Microsoft almost always come from Development; almost none (I can't think of any) come from a Test background.

 

8. No stars. Software testing has not produced a single star similar to Kernighan, Richie, Thompson, Knuth, Hamming, Dijkstra, Newell, Simon, Backus, Naur, Hoare, Codd, Wirth, and so on.

 

9. No breakthrough technologies. Software testing has not produced any revolutionary new technologies, research, or paradigms.

 

10. Body of knowledge. There is no core body of software testing knowledge and existing certifications are essentially simple vocabulary tests.

 

11. University curriculum. Few if any major universities offer software testing classes, much less testing programs.

 

12. Declining influence. Dedicated software testing has become less important over the past few years as developers use more test driven development techniques.

 

Each of these issues requires a lot of explanation but the list above should give you a feel for many of the common arguments about why most software engineers would claim that software testing is a second class activity compared to software development. As I mentioned all of these 12 issues have strong counterarguments, which I've not listed. However, my point in this blog entry at least is that there is a serious perception issue at Microsoft that software testing is definitely inferior to software development in many ways. Now don't take this entry as software testing bashing -- quite the contrary. I am passionate about software testing and believe testing is critically important to the overall software development life cycle.

I'm pretty excited about Windows Communication Foundation (WCF) Services. I tend to think of WCF Services as highly customizable Web Services. I've been looking at testing WCF Services and am finding the main challenge is, not surprisingly, understanding WCF Services conceptually as well as learning the low-level details. A key WCF concept is bindings. A WCF binding specifies three things: the message transport (such as HTTP), the message encoding (such as text), and message protocol details (such as SOAP version). WCF bindings therefore represent sort of a meta-protocol for a client program and a server program to communicate. WCF ships with 11 system-provided bindings, and you can create custom bindings too. The most basic system-provided WCF binding is BasicHttpBinding. It specifies using HTTP, with text based SOAP version 1.1 messaging. The other 10 system-provided bindings are: WSHttpBinding, WS2007HttpBinding, WSDualHttpBinding, WSFederationHttpBinding, WS2007FederationHttpBinding , NetTcpBinding, NetNamedPipeBinding, NetMsmqBinding, NetPeerTcpBinding, and MsmqIntegrationBinding. When creating a WCF Service, you can specify the binding in one of two ways. The recommended approach is to place the binding in a configuration file. The alternative is to programmatically specify the binding. If you are hosting a WCF Service inside IIS, you could specify the binding in a configuration file by creating a Web.config file like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <services>
      <service name="MyService" >
        <endpoint
          address=""
          binding="wsHttpBinding"
          bindingConfiguration="Binding1"
          contract="IMyService" />
      </service>
    </services>
    <bindings>
      <wsHttpBinding>
        <binding name="Binding1">
        </binding>
      </wsHttpBinding>
   </bindings>
  </system.serviceModel>
</configuration>

 

If you wanted to programmatically specify the binding in the WCF Service, the code (in C#) would look like:

 

 

The main advantage to specifying the WCF Service binding in a configuration file is that is you ever want to change the binding, you can just change the configuration file without having to rebuild the Service. The main disadvantage of the configuration file approach is that you have critical information in a separate file.
 
Once you know what bindings are, the next concept one step up the WCF abstraction level is an endpoint. A WCF endpoint consists of an address, a binding, a "contract", and local implementation details of the endpoint.