The .NET Platform
Development Tools
COM & COM+
Data Access
Web Development
XML Technologies
Windows Servers
Wireless & Mobile
Security issues
Design & Process
Career Development
Analysis & Comment
Disposable Objects
We found this book useful!
You are not logged in: login here to access all areas.
Visual Studio Team Edition for Software Testers
The Testers edition of VSTS is designed for use by the testers on your development team, although some of the tools are also available in the Developers edition. Ian Murphy tests them out. (Copy now revised for Beta 2!)
Author: Ian Murphy
Last updated: May 2005
NOTE: This article is based on Beta 2 of the software, so the features described may differ from those of the final release. There is a general misconception that developers do little testing of their own software. Instead software testing is seen as the preserve of Quality Assurance (QA) teams, or even worse, the end user. With Visual Studio Team Edition for Software Testers, Microsoft has extended the test tools from the Developers edition to provide continuity, and then added further tools specifically aimed at the Test role. Unit Testing Click on the OK button and the test project appears in Solution Explorer. The test project contains a test class with all the code necessary to test the selected methods. Open the class and you will see the variables which need to be set clearly commented. Enter some appopriate values, then save and rebuild the entire project. You execute tests from the Test Manager or from the Test View. You can run single tests or several tests in one go from this list. Managing tests Code coverage To get more information you click on a single entry which takes you straight to the relevant code. Lines that were executed are highlighted in green, while code highlighted in red was ignored. Web testing Now it is simply a case of finding and browsing your website until you are done, at which point you can stop recording. You can pause the recording at any point, perform actions you don’t want recorded (such as entering information into a secure page), and then resume. Results are stored in an XML file and can be easily edited later to remove unwanted pages or delete errors. Load Tests 
Like other software vendors who have gone for a ‘unified’ development environment, Microsoft accepts that QA cannot be done in isolation from development. As a result, whenever errors are detected in the code, the testers can create a bug report that includes the offending code. Such reports can be stored in the Team Foundation Server that underpins Visual Studio Team System (VSTS), ready to be analysed using the Business Intelligence (BI) tools inside SQL Server 2005. Not only does this allow you to create an auditable review, it also provide the ability to examine the performance of an individual developer or team.
In many ways this is the same approach as that taken by Rational, Borland and other software vendors. However the BI tools allow Microsoft to add something that the other vendors don't at present have.
Many companies have significant investement in software test tools from vendors such as Mercury Interactive, Rational and Compuware, and in the associated training. Although there are plenty of tools built in to the Testers edition, Microsoft has been actively courting such vendors to ensure their tools can integrate closely with the Testers edition, and indeed Beta 2 comes with extensive Application Programming Interface (API) descriptions.
At the time of writing we have little information as to who will have their tools ‘plugged-in’. However, Microsoft is suggesting that more than 250 ISVs are currently porting tools to VSTS, with between 30 and 50 planning to announce or show their products at the forthcoming TechEd conferences.
Spreading the test effort over both the Developer and Tester roles provides a solid handover and common ground between the two roles. Unit testing and code coverage are supported in both editions, while Load Testing and Test Case Management are controlled from the Testers edition. Additionally, Microsoft has released a separate Load Test engine which, like Team Foundation Server, needs to be licensed separately.
The first and most basic testing element is the Unit Test. You create a Unit Test from the Generate Unit Tests page, where you can select the method on which you wish to conduct the test and the language that the test project will use. 
Once a test is finished the results are displayed in the Test Results window. You get a single line entry for each test telling you if it failed, passed or proved inconclusive. Double clicking on a test displays a result page showing you what happened, while right-clicking brings up a menu that allows you to do a range of other tasks. If you had Code Coverage enabled, for example, you can jump straight from the test to the source code. Alternatively, you can create a Performance Session allowing you to use this test to gather more detailed information.
If a test shows up a problem with your source code you will want to make changes and then retest. With many tools this means opening the source code, finding the problem, fixing it, rebuilding and then re-running the test. However with the Testers edition you don’t have to rebuild as Microsoft has arranged for this to be done automatically. While this doesn’t really save any time, it does reduce the number of steps.
Whoever discovers the problem can immediately create a new Work Item. This is where the integration within VSTS comes together, as the bug report can include a lot of information that is automatically generated by the system. The report can be assigned to a particular person and allocated a priority, state, and action that needs to be taken. The output from the test is attached as a file, while the details section holds the name of the test that was run, where it can be found, and a unique error code. This considerably facilitates communication between the tester and the developer.
You can also run manual tests by right-clicking on the project and selecting New Test. This opens the Add New Test window displaying possible templates, some of which use a document template while others are created from a wizard. If you choose to use the Word Format then you will need to have Microsoft Word installed on your computer. Alternatively you can choose the Text Format option and open a text file inside Visual Studio 2005.
While creating tests automatically might seem a significant saving on time, often a manual test can help you better understand the conditions under which a test may fail. You could create a manual test as part of your documentation, so allowing an inexperienced tester to run a series of tests rather than use a script. Manual tests could also be created by sitting with users and capturing typical operations.
Another alternative is to run tests from the command line using the MSTest program. The output from MSTest can be redirected into an XML file for import later.
To do proper granular testing of applications you need to create small focused tests. If VSTS is to achieve the goal of helping developers produce better quality code, then we will need to see a lot of testing of individual components, as well as the more traditional large scale tests of entire systems. This requires that tests be organised effectively, which is where Test Manager comes in.
Test Manager allows you to create your own test lists, and allocate each list a name, description and location. Once created you can simply drag and drop existing tests into your new list, select any number of tests within the list and have them executed in your preferred sequence. You can’t actually drag and drop tests into a different order; instead you assign a number to each test that determines execution sequence.
Code Coverage allows you to instrument your code so you can get much more granular information on what is happening when you run your tests, and is supported by both the Developers and the Testers editions. You select which tests to run in the usual way, except now you ensure that the Code Coverage box is checked. Once the tests have run you will see a new Code Coverage tab alongside Error List and Test Results. This opens the Code Coverage window listing the elements that were called when the test was run. 
Test classes can also be monitored through Code Coverage to check which tests have executed, and which lines have actually been used. Testing is often done in an automated environment these days because it is too complex to assess manually. This technique can be used to provide a comprehensive test pass/fail document.
With the Testers edition, Microsoft has provided tools to conduct proper functional tests on a web site. There are a number of players already in this market but this is a new venture for Microsoft. Nevertheless, these new tools come with good tutorials and a sensible approach.
The start point for any Web test is to create a recorded test, which simply records the user’s keystrokes and the URLs of the pages visited. The simplest way to do this is to start a new Test Project and create a new test using the Web Test option. This opens Internet Explorer with the Web Test Recorder (WTR) toolbar enabled. 
Once you have created a recorded test it needs to be converted into a Coded Web Test, which you do by simply selecting the test and clicking the Generate Code button. The result is either a Visual Basic or C# project, and allows you to customise the test further. For example you could create validation rules that allow you to check for an error message, so that you can track down what is causing it.
Finally, you can use your Web Tests as the basis for Load Tests. This allows you to test the behaviour of the Web site under load, and also gives you a set of tests that can be used whenever problems are reported. With Web designers, ISPs and internal IT departments having to conform to Service Level Agreements (SLAs) such tests can form part of the delivery documents.
A Load Test enables you to run a Web Test in a more realistic environment, simulating the presence of multiple users all accessing the site at the same time.
You start by right-clicking on the project node and selecting Add New Load Test. The wizard then walks you through the number of users you want to simulate, how many simultaneous connections to manage and how frequently to add new users.
There are also more subtle parameters such as ‘user think time’. One of the problems of automating a test is that it then behaves in an unrealistic fashion. For example, people don’t usually click through 50 pages as quickly as possible. They will pause and read the page, even if only to find out where to click next. You need to think about how much time your test should pause on each page.
You can also choose the browser version used for the test. This is a huge issue for website testing. Too many websites are tested using the same browser version and the same add-ins as the developer. While Microsoft has provided support for a number of different browsers, there doesn’t seem to be too much information on how to add browsers such as Opera, for example. This may appear later.
In a large company, users will be sharing bandwidth. If you have users who may want to access the site through a dial-up connection, or from a mobile device, then you can make the test more realistic by restricting bandwidth.
Finally you get to decide when the tests will be run, and then you save and exit the wizard. Running a load test is then simply a case of clicking the Green button at the top of the Load Test screen.
Click here for our Privacy Statement. Copyright © Matt Publishing. All rights reserved. No part of this site may be reproduced without the prior consent of the copyright holder.
Introducing Visual Studio Team System Visual Studio Team Edition for Software Architects
