James McCaffrey

I have been looking at Silverlight lately and have been investigating how to create UI test automation for Silverlight applications. As far as I can tell so far, it looks like there are two main approaches. First, you can use the Microsoft UI Automation (MUIA) library to create C#, shell-based test automation. A second approach is to use JavaScript-to-Silverlight interoperability and create browser-based test automation. I haven't tried ether approach but I'm fairly confident I could work out the details. A third possibility is to somehow use Silverlight technology to create UI test automation for a Silverlight application. I'm not sure how this would work, or if the technique is even possible, but the idea sounds interesting.

The .NET Framework version 4.0 supports a new programming language feature called variant methods. Let me cut to the chase and state I think variant methods are an example of solutions in search of a problem, but they're fascinating anyway. So what are contravariant and covariant methods, and how do you test them? In order to test variant methods you need to understand what they are. In short, MSDN Library documentation states that contravariance is the ability to use a less derived type, and covariance is the ability to use a more derived type than that originally specified. My initial reaction was pretty much WFH (what the heck). I couldn't entirely follow the MSDN examples so I played around with some code myself. After some experimentation, I believe that the scenario where contravariance and covariance are most likely to be used is when you 1.) have a generic interface, and also 2.) have a base class which implements the generic interface, and also 3.) a second class which is derived from / inherits from the base class. For example, suppose you have some generic interface IConsoleDisplayable<T>:

 

public interface IConsoleDisplayable<T>
{
  . . .
}

and a class Person which implements IConsoleDisplayable<>, and also a class Employee which is derived from class Person. Without contravariance and covariance the following code does not work:

 

IConsoleDisplayable<Person> p = new Person("Smith", "Chris"); // OK
. . .
IConsoleDisplayable<Employee> e = p; // error

 

and requires an explicit cast like this:

 

IConsoleDisplayable<Employee> e = (IConsoleDisplayable<Employee>)p; // OK

 

But if you declare the generic interface as contravariant by adding the "in" keyword like this:

 

public interface IConsoleDisplayable<in T>
{
  . . .
}

 

then the code above (without an explicit cast) compiles. Well this is just plain psycho on a variety of levels and seems like the C# language team had way too much time on their hands. Just because you can create some new language feature doesn't necessarily mean you should. (Of course, there's no reason why application or system developers have to use variant methods). In my opinion, the advantage gained by eliminating an explicit cast is not worth the additional complexity of a contravariant interface. But it's an interesting language concept for sure.

 

The new F# programming language allows you to write curried functions. I am not a fan of using curried functions, at least from my point of view as a software tester. By the way, the term "curried" comes from Haskell Curry, a mathematician. Consider this normal F# code:

 

printfn "\nBegin curried function demo\n"
let s1 = "Hello"
let s2 = "world"

 

let prependString p s =
  let result = p + s
  result

 

let r1 = prependString "//" s1
let r2 = prependString "//" s2
printfn "r1 and r2 are %s %s \n" r1 r2

 

The result would be, as you'd expect "r1 and r2 are //Hello //World". Now here's a way to use curried functions:

 

let curriedPrependSlashes =
  prependString "//"

 

let r3 = curriedPrependSlashes s1
let r4 = curriedPrependSlashes s2
printfn "r3 and r4 are %s %s \n" r3 r4

 

The result would be identical, "r3 and r4 are //Hello //World". From a usage point of view curried functions allow you to reduce the number of arguments passed to a function when the function is called. I don't like this from a testing point of view because currying usually involves an extra call to a non-curried helper function, which makes testing a bit more opaque so to speak. In other words, the call:

 

let r1 = prependString "//" s1

 

is more clear to me as a tester than the call:

 

let r3 = curriedPrependSlashes s1

 

But, this is really more opinion than technical and some people whose opinions I respect think curried functions are great things.

 

Fault injection is the process of manipulating a software system under test so that an error is deliberately generated. For example, suppose some SUT has a method Compute() which may throw an exception under certain conditions. In one form of fault injection you modify the source code of the SUT and recompile so that any time Compute() is called by some other method, say Main(), an exception is thrown. The ideas behind fault injection are to test how well the SUT deals with exceptions, and increase code coverage. Fault injection is rather difficult in many situations because you have to either modify the SUT source code and recompile, modify the SUT binaries, or use some mechanism which modifies the behavior of the SUT at runtime. TestApi is a relatively new and growing collection of C# code which can be used to perform software testing tasks on .NET applications, libraries, and services. TestApi is hosted by the CodePlex Microsoft open source project. Here's a snippet of what fault injection might look like:

 

FaultRule r1 = new FaultRule("MySUT.Compute(int, int)");
r1.Condition =  BuiltInConditions.TriggerIfCalledBy("Program.Main(string[])");
r1.Fault =  BuiltInFaults. ThrowExceptionFault(new Exception("Fatal"));
FaultSession session = new FaultSession(new FaultRule[] { r1 });
session.Launch(D:\\MySUT.exe");

 

There are many other ways to perform fault injection testing but the TestApi approach is worth checking out. I am working with some of the developers of TestApi to put together a research paper for presentation at the 7th International Conference on Information Technology New Generations. See http://www.vteOnline.com/ITNG2010/ for details.

I am the chair of the software testing track of the 2010 7th International Information Technology New Generations conference (see http://www.vteOnline.com/ITNG2010/ ). I am working with the authors of a paper that describes a neat software test automation API set called TestApi. In that paper the authors and I have proposed a taxonomy of different approaches to test case automation. In our model we suggest six levels of abstraction that vary across five key characteristics. The six basic structures of test we propose are: custom code, tool, library, API set, framework, and integrated system. The five key distinguishing characteristics are the extent to which the automation code is reusable, how specific or general the type of target automation tasks are, the extent to which the code is documented, the type of licensing associated with the structure which mediates the extent to which the automation code can be shared, and the extent to which the automation code is designed to allow integration with other code such as logging code, management and monitoring code, and other test automation. As with any taxonomy or set of definitions, no real knowledge is generated; the purpose is to establish a baseline and common set of terminology for more effective communication.