8 Determining a Pass/Fail Result When the Stored Procedure Under Test Does Not Return a Value
Tải bản đầy đủ - 0trang
6633c09.qxd
4/3/06
2:00 PM
Page 257
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
select @actual = checksum_agg(checksum(*)) from dbEmployees.dbo.tblEmployees
if (@actual = @expected)
print 'Pass'
else
print 'FAIL'
If the stored procedure does not return a value, then it must perform some action, such as
deleting data from a table. To test such stored procedures, you need to compare actual and
expected values of the object acted upon by the stored procedure. This situation is very similar
to testing stored procedures that return a SQL rowset object.
Comments
Many stored procedures affect an underlying data table. Obvious examples include stored
procedures that use an INSERT, DELETE, or UPDATE statement. When testing such stored procedures, you must be sure to reset the state of the underlying data tables to some known state
before each call in the test harness. For example, suppose you are testing a stored procedure
usp_DeleteEmployee() defined as
create procedure usp_DeleteEmployee
@empID char(3)
as
delete from tblEmployees where @empID = empID
return @@rowcount
go
If the code in your test harness resembled
declare @input char(3)
declare @actualRows int
-- main test loop
-- read test case data into @caseID, @input, @expectedRows
exec @actualRows = dbEmployees.dbo.usp_DeleteEmployee @input
-- determine pass/fail
-- store or display test case result
-- end main loop
then each iteration through the main test loop would be testing against a different state of the
database, which would make determining an expected value very difficult. You need to reset
the database state before each test harness call to the stored procedure under test:
declare @input char(3)
declare @actualRows int
-- main test loop
-- read test case data into @caseID, @input, @expectedRows
truncate table dbEmployees.dbo.tblEmployees
www.it-ebooks.info
257
6633c09.qxd
258
4/3/06
2:00 PM
Page 258
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
insert into dbEmployees.dbo.tblEmployees
values('001', 'Adams', '06/15/1998')
insert into dbEmployees.dbo.tblEmployees
values('e22','Baker', '06/15/2001')
-- etc.
exec @actualRows = dbEmployees.dbo.usp_DeleteEmployee @input
-- determine pass/fail
-- store or display test case result
-- end main loop
In most situations, resetting the state of the database under test requires many statements
that can make your test harness script very long. So, a good approach is to write an auxiliary
stored procedure in your test harness script to handle the task of resetting the database state
before each call to the stored procedure under test:
if exists (select * from sysobjects where name='tap_Reset')
drop procedure tap_Reset
go
create procedure tap_Reset
as
truncate table dbEmployees.dbo.tblEmployees
insert into dbEmployees.dbo.tblEmployees
values('e11','Adams', '06/15/1998')
insert into dbEmployees.dbo.tblEmployees
values('e22','Baker', '06/15/2001')
insert into dbEmployees.dbo.tblEmployees
values('e33','Young', '06/15/1998')
insert into dbEmployees.dbo.tblEmployees
values('e44','Zetta', '06/15/2001')
-- other data would be inserted too
go
Here you create a utility test automation procedure that deletes all the rows in table
tblEmployees and then repopulates with rich test bed data. Your script can then call the stored
procedure inside the main test loop just before each call to the stored procedure under test.
An alternative to hard-coding the INSERT statements that populate the target table is to use the
BULK INSERT statement in conjunction with an external data store, as described in Section 9.3.
Do not misinterpret this discussion to mean that you should always reset the database
under test to some initial state. You must also perform test scenarios that manipulate system
state through several changes. For example, a test scenario could insert five data rows, delete
one of the new rows and one of the original rows, then insert three new rows, and delete a row.
Each state of the database could be examined for correctness to determine an overall scenario
pass/fail result.
www.it-ebooks.info
6633c09.qxd
4/3/06
2:00 PM
Page 259
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
9.9 Example Program: SQLspTest
The scripts in this section combine several of the techniques in this chapter to create a
lightweight T-SQL test harness system. There are three scripts: script makeDbEmployees.sql,
which creates the underlying test bed and the stored procedure under test; script
makeDbTestCasesAndResults.sql, which creates test case data and result storage; and
script SQLspTest.sql, which is the actual test harness. The stored procedure under test is
usp_HiredAfter(), which accepts a datetime input argument and returns a SQL rowset of
those employees in tblEmployees whose date of hire is after the input argument. When
run, the output will be that shown in Figure 9-1 in the introduction section of this chapter.
Listing 9-1 shows the script that creates the underlying database and the stored procedure
under test.
Listing 9-1. Script to Create Test Bed Database and Stored Procedure Under Test
-- ===========================================================
-- makeDbEmployees.sql
use master
go
if exists (select * from sysdatabases where name='dbEmployees')
drop database dbEmployees
go
if exists (select * from sysxlogins where name = 'employeesLogin')
exec sp_droplogin 'employeesLogin'
go
create database dbEmployees
go
use dbEmployees
go
create table tblEmployees
(
empID char(3) primary key,
empLast varchar(35) not null,
empDOH datetime not null,
)
go
-- this is dev data, not test case data
insert into tblEmployees values('e11','Adams', '06/15/1998')
insert into tblEmployees values('e22','Baker', '06/15/2001')
go
www.it-ebooks.info
259
6633c09.qxd
260
4/3/06
2:00 PM
Page 260
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
exec sp_addlogin 'employeesLogin', 'secret'
go
exec sp_grantdbaccess 'employeesLogin'
go
create procedure usp_HiredAfter
@dt datetime
as
select * from tblEmployees where empDOH > @dt
go
grant execute on usp_HiredAfter to employeesLogin
go
-- end script
Listing 9-2 shows the script that creates a test case data and test result store.
Listing 9-2. Script to Create Test Case Data and Test Results Stores
-- ===========================================================
-- makeDbTestCasesAndResults.sql
use master
go
if exists (select * from sysdatabases where name='dbTestCasesAndResults')
drop database dbTestCasesAndResults
go
if exists (select * from sysxlogins where name = 'testLogin')
exec sp_droplogin 'testLogin'
go
create database dbTestCasesAndResults
go
use dbTestCasesAndResults
go
create table tblTestCases
(
caseID char(4) primary key,
input datetime not null,
expectedChecksum int not null
)
go
www.it-ebooks.info
6633c09.qxd
4/3/06
2:00 PM
Page 261
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
-- this is the test case data for usp_HiredAfter using a checksum expected
-- value approach
-- can also read from a text file using BCP, DTS, or a C# program
insert into tblTestCases values('0001','01/01/1998', 1042032)
insert into tblTestCases values('0002','01/01/1998', 9999999) -- deliberate error
insert into tblTestCases values('0003','01/01/2000', 25527856)
insert into tblTestCases values('0004','01/01/2006', 0)
go
create table tblResults
(
caseID char(4) not null,
result char(4) null,
whenRun datetime not null
)
go
exec sp_addlogin 'testLogin', 'secret'
go
exec sp_grantdbaccess 'testLogin'
go
grant select, insert, delete, update on tblTestCases to testLogin
go
grant select, insert, delete, update on tblResults to testLogin
go
-- end script
Listing 9-3 is the test harness script.
Listing 9-3. The Test Automation Harness Script
-- ===========================================================
-- SQLspTest.sql
-- test dbEmployees..usp_HiredAfter
-- reads test case data and writes results
-- to dbTestCasesAndResults
set nocount on
www.it-ebooks.info
261
6633c09.qxd
262
4/3/06
2:00 PM
Page 262
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
if not exists
(select * from master.dbo.sysdatabases where name='dbTestCasesAndResults')
raiserror('Fatal error: dbTestCasesAndResults not found', 16, 1)
go
if exists (select * from sysobjects where name='tap_Reset')
drop procedure tap_Reset
go
create procedure tap_Reset
as
truncate table dbEmployees.dbo.tblEmployees
insert into dbEmployees.dbo.tblEmployees
values('e11','Adams', '06/15/1998')
insert into dbEmployees.dbo.tblEmployees
values('e22','Baker', '06/15/2001')
insert into dbEmployees.dbo.tblEmployees
values('e33','Young', '06/15/1998')
insert into dbEmployees.dbo.tblEmployees
values('e44','Zetta', '06/15/2001')
-- other data would be inserted too
go
-- prepare dbEmployees with rich data
exec tap_Reset
go
declare tCursor cursor fast_forward
for select caseID, input, expectedChecksum
from dbTestCasesAndResults.dbo.tblTestCases
order by caseID
declare
declare
declare
declare
@caseID char(4), @input datetime, @expectedChecksum int
@whenRun datetime
@resultMsg varchar(80)
@actualChecksum int
create table #resultRowset -- for checksum technique
(
empID char(3) primary key,
empLast varchar(35) not null,
empDOH datetime not null,
)
www.it-ebooks.info
6633c09.qxd
4/3/06
2:00 PM
Page 263
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
set @whenRun = getdate()
print
print
print
print
'Stored procedure under test = usp_HiredAfter'
' '
'CaseID Input
Expected Actual Result'
'==============================================='
open tCursor
fetch next
from tCursor
into @caseID, @input, @expectedChecksum
while @@fetch_status = 0
begin
exec tap_Reset -- reset test bed data
truncate table #resultRowset -- empty out the result rowset
insert #resultRowset (empID, empLast, empDOH) -- call sp under test
exec dbEmployees.dbo.usp_HiredAfter @input
if (@@rowcount = 0)
set @actualChecksum = 0
else
select @actualChecksum = checksum_agg(binary_checksum(*)) from #resultRowset
if (@actualChecksum = @expectedChecksum)
begin
set @resultMsg = @caseID + '
' + cast(@input as varchar(11)) +
' ' + cast(@expectedChecksum as varchar(20)) + ' ' +
cast(@actualChecksum as varchar(20)) + ' Pass'
print @resultMsg
insert into dbTestCasesAndResults.dbo.tblResults values(@caseID, 'Pass',
@whenRun)
end
else
begin
set @resultMsg = @caseID + '
' + cast(@input as varchar(11)) +
' ' + cast(@expectedChecksum as varchar(20)) + ' ' +
cast(@actualChecksum as varchar(20)) + ' FAIL'
print @resultMsg
insert into dbTestCasesAndResults.dbo.tblResults values(@caseID, 'FAIL',
@whenRun)
end
www.it-ebooks.info
263
6633c09.qxd
264
4/3/06
2:00 PM
Page 264
CHAPTER 9 ■ SQL STORED PROCEDURE TESTING
fetch next
from tCursor
into @caseID, @input, @expectedChecksum
end
close tCursor
deallocate tCursor
drop table #resultRowset
-- end script
www.it-ebooks.info
6633c10.qxd
4/3/06
1:55 PM
Page 265
CHAPTER
10
■■■
Combinations and
Permutations
10.0 Introduction
Combinations and permutations are fundamental concepts in software testing, and the ability
to programmatically generate and manipulate them is an essential test automation skill. An
arbitrary combination is a subset of k items selected from a larger set of n items, where order
does not matter. For example, if you have the 5 items
{ "ant", "bug", cat", "dog", "elk" }
then the 10 possible combinations of size 3 are
{
{
{
{
{
{
{
{
{
{
"ant",
"ant",
"ant",
"ant",
"ant",
"ant",
"bug",
"bug",
"bug",
"cat",
"bug",
"bug",
"bug",
"cat",
"cat",
"dog",
"cat",
"cat",
"dog",
"dog",
"cat"
"dog"
"elk"
"dog"
"elk"
"elk"
"dog"
"elk"
"elk"
"elk"
}
}
}
}
}
}
}
}
}
}
You can imagine that these could be test case inputs to a method that accepts three string
arguments. Notice that { "cat", "bug", "dog" } is not listed because it is considered the same
as { "bug", "cat", "dog" }. A mathematical combination is a generalization of this idea of subsets. Instead of being a subset of arbitrary items, a mathematical combination of order (n, k) is a
subset of size k of the integers from 0 up to n-1. So the 10 elements of a mathematical combination of 5 items taken 3 at a time are
{
{
{
{
{
{
0,
0,
0,
0,
0,
0,
1,
1,
1,
2,
2,
3,
2
3
4
3
4
4
}
}
}
}
}
}
265
www.it-ebooks.info
6633c10.qxd
266
4/3/06
1:55 PM
Page 266
CHAPTER 10 ■ COMBINATIONS AND PERMUTATIONS
{
{
{
{
1,
1,
1,
2,
2,
2,
3,
3,
3
4
4
4
}
}
}
}
In this example, the elements of the combination are listed in lexicographical order (also
called lexicographic order or dictionary order). For mathematical combinations, this means
that the elements, if interpreted as integers, are listed in increasing order. For example, if n = 5
and k = 3, the first element is { 0, 1, 2 } and the next element is { 0, 1, 3 } because “12”
comes before “13”. Notice, too, that the atoms (individual integers) of a combination element
also appear in increasing order so there is a kind of dual orderedness to a lexicographical combination. With a lexicographical combination of order (n, k), the identity element is defined to
be the first element: { 0, 1, 2, . . . n-k }.
The function that calculates the total number of combinations for given n and k values is a
very important function when dealing with combinations. For instance, the previous two examples demonstrate that the total number of combinations of 5 items taken 3 at a time is 10. This
helper function is often called Choose. So, you can write Choose(5,3) = 10.
Closely related to combinations are permutations. An arbitrary permutation is one of the
possible arrangements of a set of n items. For example, if you have the three items
{ "Adam", "Barb", "Carl" }
then, the six permutations of these items are
{
{
{
{
{
{
"Adam",
"Adam",
"Barb",
"Barb",
"Carl",
"Carl",
"Barb",
"Carl",
"Adam",
"Carl",
"Adam",
"Barb",
"Carl"
"Barb"
"Carl"
"Adam"
"Barb"
"Adam"
}
}
}
}
}
}
Notice that unlike combinations, permutations take order into account by definition. A
mathematical permutation is a generalization of this idea of rearrangements. Instead of being a
rearrangement of arbitrary items, a mathematical permutation of order n is a rearrangement of
the integers from 0 up to n-1. So the six elements of a mathematical permutation of order 3 are
{
{
{
{
{
{
0,
0,
1,
1,
2,
2,
1,
2,
0,
2,
0,
1,
2
1
2
0
1
0
}
}
}
}
}
}
Permutations can be lexicographical as in this example—notice that if the permutation
elements were interpreted as integers, you would have { 12, 21, 102, 120, 201, 210 }. The
total number of permutations of order n is given by n factorial, often denoted by n! or Factorial(n). So in the preceding two examples, because we are dealing with n = 3, the total number
of permutations is 3! = 3 * 2 * 1 = 6.
www.it-ebooks.info
6633c10.qxd
4/3/06
1:55 PM
Page 267
CHAPTER 10 ■ COMBINATIONS AND PERMUTATIONS
Combinations and permutations occur in many aspects of software testing. For example,
suppose you had a program with a UI that has three drop-down controls. You need to analyze
how many different combinations and permutations of user inputs there are so you can design
your test cases. Or suppose you are testing a program designed for multiple hardware configurations. You need to analyze the different combinations and permutations of the configurations
so you can plan your test effort.
You can write combination and permutation methods that work directly on type string.
But a more flexible approach is to write methods that work on integers and then map these
mathematical combination and permutation methods to string arrays.
10.1 Creating a Mathematical Combination Object
Problem
You want to create an object to represent a mathematical combination.
Design
Use an object-oriented design to create a Combination class with an array of type long to hold a
combination element, and long values n and k to hold the total number of integers and subset
size, respectively.
Solution
public class Combination
{
private long n = 0;
private long k = 0;
private long[] data = null;
public Combination(long n, long k)
{
if (n < 0 || k < 0)
throw new Exception("Negative argument in constructor");
this.n = n;
this.k = k;
this.data = new long[k];
for (long i = 0; i < k; ++i)
this.data[i] = i;
}
}
Comments
A mathematical combination lends itself nicely to implementation as a class. Because a
mathematical combination represents a subset of k items selected from a set of integers from
0 through n-1, you need to store those values as well as an array to hold the combination
www.it-ebooks.info
267