When performing Web application UI testing using Windows PowerShell, as I described in my last blog entry, the trickiest task is to determine when the Web application under test is fully loaded into Internet Explorer. You can't realistically use a crude approach such as:

 

$ie = new-object -com "InternetExplorer.Application"
[System.Threading.Thread]::Sleep(5000)

 

because there's really no way to predict how long the app will take to load, or even if the app will load at all. There are several alternatives you can use. One is to use the InternetExplorer DocumentComplete event. This approach is a bit tricky and I personally prefer to avoid using event techniques when performing script-based automation. Another approach is the use the InternetExplorer readyState property. This approach is great in principle but there are just enough minor annoyances with the technique for me to avoid it in general. A third approach, and the one I use most often, is to use a delay loop where the loop will exit if a user control reference is accessible (meaning not null). The loop condition must also exit if the number of delays exceeds some threshold value so that you avoid an infinite loop situation. I am currently writing up the details of this approach for an MSDN Magazine "Test Run" column and I'll be submitting it to the new MSDN Editor-in-Chief, Howard Dierking, within a week or so. 

The use of ASP.NET Web applications continues to grow, and testing Web applications is an increasingly common task for software developers, testers, and managers. Manual testing remains the keystone of application testing but automated testing has five key advantages, which I like to summarize by the acronym SAPES: compared to manual testing, automated tested has better speed, better accuracy, better precision, better efficiency, and better skill-building characteristics. In an Agile-type development environment, traditional automated testing is difficult because of the difficulty and time required t create the automation -- in many cases the test automation is obsolete or irrelevant by the time it's ready to be used. In many situations, the use of Windows PowerShell, the new Microsoft replacement for the cmd.exe shell and .BAT files, changes all that. You can quickly create what I've coined "Ultra Lightweight Software Test Automation" (ULSTA) using PowerShell. For example, the screenshot below (running on Windows Vista) was produced by the following code, which took only about 30 minutes to create.

 

 

If you haven't experimented with PowerShell yet, I encourage you to check it out at http://www.microsoft.com/powershell. 

Software test automation has five key advantages over manual software testing and these advantages are summarized by the acronym SAPES: speed, accuracy, precision, efficiency, and skill-building. One of the four fundamental types of software test automation is Web Application UI automation. (The other three fundamental types are API/unit/module testing, Windows application UI testing, and HTTP request-response testing). If you write custom Web application UI automation (as opposed to using a framework written by someone else), you can either use a high level JavaScript approach to manipulate the browser DOM, or you can use low level calls into Win32 functions contained in the SHDocVw.dll interop assembly. The low level approach works fine on Windows XP but there are some unresolved (by me anyway) issues using the approach on Vista. A key part of the technique involves the test harness launching an instance of Internet Explorer and then attaching to that process. For example, the code below works fine on XP but invariably barfs on Vista. I'll update this blog if I figure what's going on.

 

Recently I posted a blog entry titled, "The Declining Prestige of Software Testing" that generated a ton of e-mail replies. The messages generally fell into one of two Boolean-style categories: 1.) I'm completely wrong, 2.) I'm completely right. Regardless, the replies were very vigorous so I seem to have touched on something that quite a few software engineers care about. For the sake of argument, let's suppose that the perception of the prestige of software testing as a discipline at Microsoft and other companies is in fact declining. What are some possible actions the software testing community can take? The answer to this depends on the root causes of the decline in prestige. So we need to make yet more assumptions; in particular suppose that software testing is uncovering fewer critical bugs because 1.) software developers are doing more testing on their own, 2.) developing software with C#, Java, and frameworks such as RoR is inherently less tricky than with older technologies such as C++ and ATL, and 3.) software developers are using increasingly sophisticated code check-in and regression tools. If all of these assumptions are correct then in order to enhance the prestige of software testing, testers must make some significant additions and changes to the software testing paradigm. One idea that comes to mind is that testing may become more valuable by integrating more closely with the software development process. This integration could take many forms. For example, instead of testing code after the code has been checked into a version control system and built, testers could start decorating source code with pre-conditions, post-conditions, and state invariants (something developers in general absolutely hate to do), using either a commenting approach or an attribute approach. This would enable semi-automatic test case generation (in theory at least) and greatly enhance test coverage. Microsoft Research has a very interesting Spec# project (see http://research.microsoft.com/specsharp/) which is looking at this type of approach.
 

Because I am a Contributing Editor for Microsoft's MSDN Magazine, I regularly survey managers, developers, testers, and all job categories for that matter, at Microsoft and other companies, on a wide range of topics. An interesting trend I have heard over and over again recently has to do with the declining prestige of software testing at Microsoft and other companies. Before you jump to any conclusions (shades of "Office Space"), let me explain. Among the many people I've talked to, there is a common consensus that software test engineers at Microsoft (the job title is "SDET" which stands for Software Design/Development Engineer in Test) today are, as a group, significantly more technically skilled and have better fundamental testing knowledge than test engineers as a group a few years ago. But in spite of this, I have heard literally hundreds of comments from non-testers (primarily developers and program managers) which state in effect that they do not hold software testing as a discipline in as high regard as a few years ago. How can this be? When probed, I usually hear comments that go something like this: "In the old (meaning C++) days, software testing was critical because the code was trickier (pointers and so on) and developers did relatively little testing (because there just wasn't time). The test team found regularly found many critical bugs. Now, developers are testing much more of their own code, so there are fewer high-priority bugs to begin with. Additionally, developers are using increasingly sophisticated code check-in and regression testing tools which further prevent serious bugs from entering a build/iteration. This leads to a situation where software testing is becoming marginalized in importance, simply because there are fewer critical bugs." Well I'm not sure whether I agree with this or not, but I am sure that this perception (or perhaps reality) of the declining prestige of software testing is strong and growing at Microsoft and other companies, at least among the people I've talked to. I don't have any immediate answers to this problem (if in fact it's a problem at all) except to suggest that software testing as a whole (whatever the heck that means!) should closely examine itself and ask if it's time for some radical changes in how we approach software testing. I am a huge fan of Microsoft technologies, Microsoft as a company, and software testing as a discipline, and think that this declining-prestige question should be examined to make all three areas stronger.