| James's profileJames McCaffreyBlogLists |  Tools  Help |
|
November 26 Microsoft's LocStudioLocStudio is an internal Microsoft tool which is used to help "localize" (convert from English to some other language) a software product. Let me explain. If you create software and you want to market your product in another country (such as Japan, China, Germany, or even, heavens forbid, France), the simplest approach is to create base English builds (or "iterations" if you are using Agile software development terminology) and then create separate builds of the target language. This approach is simple and works reasonably well for relatively small software products with one target language but does not scale very well for very large software products -- the localized build is typically slightly out of sync with the English build. Additionally, if you have many target languages, then creating separate build tress for each becomes very messy. Microsoft's approach is to create software in such a way that it can easily be localized. For example, instead of hard-coding all strings such as "Enter Product ID" directly into the source code, all strings are parameterized, and they can then be supplied appropriate values in the target language. LocStudio is an internal tool that helps manage this process. I'd love to show you a screen shot of LocStudio, but because LocStudio is an internal tool I can't do so. There are quite a few commercial software localization tool products that are similar to LocStudio. The image at the bottom of this blog entry shows one called Lingobit. Learning LocStudio is relatively easy so if you ever find yourself interviewing for a job at Microsoft and are asked if you know LocStudio, you can say, "No, but I now what LocStudio is and I'm confident I can learn it quickly." November 20 Black Box vs. White Box TestingIn a very small production environment where developers test their own code, "system discovery" may not be an issue. As the size of a development effort increases, however, the discovery process becomes more difficult. The most common approach is for you to read traditional written specification documents that describe the SUT. In theory at least, every system has a set of documents, usually written by senior developers, managers, or architects, that completely and precisely describes the SUT. In reality, of course, such specification documents are often out-of-date, incomplete, or even nonexistent. Regardless, examining traditional specification documents is an important way to determine how to create meaningful test cases.
You can examine the source code of the SUT to gain insights on how to test your system, although in some cases, this may not be possible for security or legal reasons. Even when source code examination is possible, reviewing the source code for a complex SUT can be enormously time consuming. When you have access to system source code while developing test cases, the situation is sometimes called white box or clear box testing. When you do not have access to source code, the situation is sometimes called black box testing. When you have partial access to system source code, for example, the signatures of methods but not the body of the method, the situation is sometimes called gray box testing. These labels are some of the most overused but least-useful terms in software testing. However, the principles behind these labels are important. You cannot test every possible input to a system (see Chapter 10 for discussions of this idea), so the more you know about your SUT, the better your test cases will be. Although there has been much research in the area of automatic test case generation, currently test case development is still for the most part a human activity where experience and intuition play a big role. Mutation TestingMutation Testing is a technique that software engineers can use to measure the effectiveness of their overall testing effort. Suppose a team of testers has created many individual tests; let's call the collection of tests the test suite. In mutation testing, the original system/program under test is mutated to create a faulty version called a mutant. The mutant program is tested using the suite of individual test cases. The test suite should produce new test case failures; if no new failures appear then that probably means the test suite does not exercise the code path containing the mutated code, which means the test suite does not fully test the system/program. You can then create new test cases that do exercise the mutant code which may reveal bugs. Notice that the term "mutation testing" is somewhat misleading -- in mutation testing you do not actually test the system under test, you measure the effectiveness of a collection of tests.
The screenshot below shows an example of mutation testing in action. I am analyzing a library named LibUnderTest.dll. It is a demo library containing a single Max() method. In the first part of mutation analysis, the DLL under test is tested using an existing test suite. In this case the test suite is named TestHarness.exe. Again, the test suite is just a demo to keep things small and the ideas clear. The test suite has only 10 test cases for simplicity. Notice that all 10 test cases pass. After establishing baseline test results, the DLL under test is examined for potential mutations. I disassemble the DLL under test and examine the resulting .asm file for branching instructions. In this example the DLL under test has three "ble" ("branch on less than or equal") branching instructions. The mutation analysis then enters a main processing loop. On each pass through the loop, the original DLL is mutated, creating a new mutant DLL. The new mutant DLL is built and then the test suite is run against the mutant. In this example, the first mutant did not produce any test failures. This is not good and a message warning you that your test suite probably misses the code path containing the mutated code is printed. The second and third mutants do produce test case failures (which is a good thing) and contribute to the "beta power" of the test suite. The mutation analysis in the screenshot finishes by printing the relative mutation "alpha power" of 0.67 -- which is just the proportion of mutations which produce test case failures. November 09 Unit Testing with PowerShell, Part IILet's continue looking at using PowerShell for very lightweight unit testing in a .NET environment. In order to perform unit testing, the first technique you must know how to perform is calling the method under test from PowerShell -- and I showed how to do that in my last blog entry. Another requirement for unit testing is the ability to iterate through a text file -- assuming you have your test case data stored that way. Suppose I have file TestCases.txt:
3:0:0:4:5.00
2:0:0:2:2.83 etc.
Each line represents one test case. For example, the first line means input point 1 is (3,0), point 2 is (0,4), and the expected result is 5.00.
I can use PowerShell to read through such a file like this:
# readFile.ps1
$file = get-content 'TestCases.txt'
foreach ($line in $file) { $tokens = $line.split(':') $x1 = $tokens[0] $y1 = $tokens[1] $x2 = $tokens[2] $y2 = $tokens[3] $expected = $tokens[4] # process extracted values here } # end script
The script should be fairly self explanatory. I use the get-content PowerShell cmdlet to read the entire contents of the test case file into an interna object. I use the foreach control structure to explicitly iterate through each line. And I use the intrinsic split() method to parse each line. Very easy and very powerful. November 05 Unit Testing with PowerShell, Part II've been looking closely at Windows PowerShell, the new Microsoft shell and scripting language. The more I look, the better I like PowerShell. Because PowerShell can tap into the .NET Framework, you can write lightweight unit test automation with PowerShell. Let me show you what I mean in a series of short blog entries. If you want to use PowerShell to perform a unit test, that is, test a method or property of a class, you must be able to call the method or property. Suppose you have a classic example of a Point class implemented in C# like this:
namespace MyPointLib
{ public class Point { private int x; private int y; public Point(int x, int y)
{ this.x = x; this.y = y; } public int X { get { return this.x; } set { this.x = value; } } public int Y { get { return this.y; } set { this.y = value; } } public double Distance(Point P) { return Math.Sqrt((Math.Pow(((double)(this.x - P.x)), 2)) + (Math.Pow(((double)(this.y - P.y)), 2))); } } // class Point } // ns MyPointLib The rather ugly code in the Distance() method returns the geometric distance between two points. Notice that Distance() is an instance method so the first point object is implied with the "this" keyword. Let's first look at how to call the Distance() method:
# callDistance.ps1
[Reflection.Assembly]::LoadFile('C:\MyLibs\MyPointLib.dll') | out-null
$p1 = new-object MyPointLib.Point(3,0)
$p2 = new-object MyPointLib.Point(0,4) [double] $dist = $p1.Distance($p2) $s = "{0:f2}" -f $dist write-output $s # end script I assume that my Point class is contained inside a MyPointLib.dll file. The first step is to load the .NET assembly containing the DLL by using the static LoadFile() method of the System.Reflection.Assmbly namespace. The LoadFile() method requires a fully qualified path. The pipe to "out-null" suppresses messages from LoadFile(). Next I use the new-object PowerShell cmdlet to instantiate two Point objects. Now I can call the Distance() method under test. Notice I use the ability of PowerShell to supply a .NET-based data type to the return value, but I could have left out the [double] qualifier if I wanted. I transfer my return value into a string formatted to two decimals. I finish by printing the formatted result, in this example 5.00, to the console. |
|
|
