I am a Contributing Editor for MSDN Magazine and write a monthly column about software testing. As part of my background research I talk to as many software test managers as I can. One common theme in discussions with test managers is a pitfall I'll call "the test architecture error". In simple terms, it is easy for software testers to lose sight of their primary goal: analyze a software system in order to find bugs so those bugs can be fixed and improve the quality, reliability, and performance of the target system. According to many of the managers I've talked to, some software test engineers spend far too much time designing their test effort and not nearly enough time actually testing the target software system. I know I've been guilty of this in the past. Sometimes you just have to sit down and simply test the heck out of a system and not worry about architecting a test framework of some sort. A closely related scenario occurs with test automation. It is easy to get seduced into spending weeks creating some test automation only to have your automation be rendered obsolete before it can be used. One reason this can happen is related to how the job performance of software test engineers is sometimes evaluated by inexperienced managers. By creating test automation, a tester has something tangible and often impressive to demonstrate to show progress, even if the automation is not all that useful. It is much harder to demonstrate progress with a list of test cases that may in fact be far more useful than slick test automation. My point is that test automation and test architecture are useful in many situations but may not always the best way to test a system, especially an application which will ultimately be used by actual human beings. Basically, I think that if you focus on doing those test activities that best improve the quality of your target system based on all the complicated factors of your particular development environment -- maybe manual testing, designing frameworks, writing automation, or whatever -- and carefully explain to your management exactly why you are doing what you are doing, the value of your work will be appreciated.

Cryptology is the study of cryptography (schemes for encrypting data) and cryptanalysis (schemes for breaking codes). The field is very complex and testing cryptological software requires a lot of advanced knowledge. One of the things I've was looking at recently is the topic of nonlinear Boolean functions. A Boolean function accepts any number of variables, each of which can have value of only 0 or 1, and returns a single 0 or 1 value. For example f = x1 + x2x3 means "x1 or (x2 and x3). Boolean functions can be represented in a surprising number of ways, including a truth table, an equation in Algebraic normal form, and many others. Boolean functions are a cryptographic primitive -- a fundamental construct -- used for several purposes. It turns out that Boolean functions used in cryptology should have certain properties to make the systems in which they are used less vulnerable to cryptanalysis. One such property is nonlinearity, which is the Hamming distance between a function of n variables, and all possible linear functions of n variables. Whew! Like I mentioned, cryptology is very complex.

Over the past few days I have been looking at browser based test harnesses with JavaScript to test Web applications. Specifically, I have created proof-of-concept Internet Explorer harnesses to perform HTTP request-response testing, and to perform UI testing. In both situations however I used hard-coded test case input. Over the weekend I took a look at reading external test case data, and writing test case results, from a browser-based harness. Basically there are two approaches available with Internet Explorer. First, you can read or write information on the test host machine. Second, you can read or write information on the Web application server machine. To read or write test data on the test host machine, you can use code along the lines of:

 

var result = document.getElementById('TextBox4');
var fso = new ActiveXObject('Scripting.FileSystemObject');
var infile = fso.OpenTextFile("C:\\testCases.txt", 1);
// etc.

 

If this code is located and executed on the test host, it will look for file testcases.txt on the test host.

To read or write test data on the Web server machine you can create a short ASP script which contains similar code, and then invoke the script from the test host along the lines of:

 

<form method='post' action='readData.asp' name='theForm'>
 <input type='submit' />
</form>

 

combined with:

 

var f = document.getElementById('theForm');
f.submit();

 

If this code is located and executed on the Web server, it will look for file testcases.txt on the server. There are many alternatives to browser based test harnesses, and I still don't have a solid grasp of the pros and cons involved, but the approach seems like it may be useful in certain testing scenarios.

The jQuery library is an Open Source collection of JavaScript functions that Web application developers can use to easily create neat client-side effects such as fade-in and menus. But the jQuery library can be used write lightweight test automation too. I have been looking at using jQuery to perform lightweight HTTP request-response testing. See the screenshot below for a proof-of-concept. The screenshot shows that I have a test harness in the form of an HTML page. That page references the jQuery library and contains a script that fires off an HTTP request, fetches the HTTP response, and examines the response for an expected value to produce a test case pass/fail result. This technique might be useful in situations where you want to use an HTML-based test harness rather than a shell-based or Windows Form-based test harness. Additionally, investigating test automation that uses new technologies such as jQuery helps testers stay in sync with developers who are using the new technologies.