Regression testing

Regression testing

Regression testing is any type of software testing which seeks to uncover software regressions. Such regressions occur whenever software functionality that was previously working correctly stops working as intended. Typically regressions occur as an unintended consequence of program changes.

"Common methods" of regression testing include re-running previously run tests and checking whether previously fixed faults have re-emerged.

Background

Experience has shown that as software is developed, this kind of reemergence of faults is quite common. Sometimes it occurs because a fix gets lost through poor revision control practices (or simple human error in revision control), but often a fix for a problem will be "fragile" in that it fixes the problem in the narrow case where it was first observed but not in more general cases which may arise over the lifetime of the software. Finally, it has often been the case that when some feature is redesigned, the same mistakes will be made in the redesign that were made in the original implementation of the feature.

Therefore, in most software development situations it is considered good practice that when a bug is located and fixed, a test that exposes the bug is recorded and regularly retested after subsequent changes to the program. Although this may be done through manual testing procedures using programming techniques, it is often done using automated testing tools. Such a test suite contains software tools that allow the testing environment to execute all the regression test cases automatically; some projects even set up automated systems to automatically re-run all regression tests at specified intervals and report any failures (which could imply a regression or an out-of-date test). Common strategies are to run such a system after every successful compile (for small projects), every night, or once a week. Those strategies can be automated by an external tool, such as BuildBot.

Regression testing is an integral part of the extreme programming software development method. In this method, design documents are replaced by extensive, repeatable, and automated testing of the entire software package at every stage in the software development cycle.

Regression test generation

Effective regression tests generate sufficient code execution coverage to exercise all meaningful code branches. Therefore, software testing is a combinatorial problem. However, in practice many combinations are unreachable so the problem size is greatly reduced. Every boolean decision statement requires at least two tests: one with an outcome of "true" and one with an outcome of "false". As a result, for every line of code written, programmers often need 3 to 5 lines of test code. [cite web|last=Cramblitt|first=Bob|author-link=Bob Cramblitt| title=Alberto Savoia sings the praises of software testing|date=2007-09-20| accessdate=2007-11-29|url=http://searchsoftwarequality.techtarget.com/originalContent/0,289142,sid92_gci1273161,00.html]

Traditionally, in the corporate world, regression testing has been performed by a software quality assurance team after the development team has completed work. However, defects found at this stage are the most costly to fix. This problem is being addressed by the rise of developer testing. Although developers have always written test cases as part of the development cycle, these test cases have generally been either functional tests or unit tests that verify only intended outcomes. Developer testing compels a developer to focus on unit testing and to include both positive and negative test cases. [cite web|last=Dudney|first=Bill|author-link=Bill Dudney|title=Developer Testing Is 'In': An interview with Alberto Savoia and Kent Beck|date=2004-12-08| accessdate=2007-11-29|url=http://www.sys-con.com/read/47359.htm]

When regression test generation is supported by a sustainable process for ensuring that test case failures are reviewed daily and addressed immediately, the end result is a regression test suite that evolves with the application, and becomes more robust and more intelligent each day. [cite web|last=daVeiga|first=Nada|author-link=Nada daVeiga| title=Change Code Without Fear: Utilize a regression safety net|date=2008-02-06| accessdate=2008-02-08|url=http://www.ddj.com/development-tools/206105233] If such a process is not implemented and ingrained into the team's workflow, the application may evolve out of sync with the generated test suite — increasing false positives and reducing the effectiveness of the test suite.

Uses

Regression testing can be used not only for testing the "correctness" of a program, but it is also often used to track the quality of its output. For instance in the design of a compiler, regression testing should track the code size, simulation time and compilation time of the test suite cases.

Quotes

*"Also as a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice, such "regression testing" must indeed approximate this theoretical idea, and it is very costly." -- Fred Brooks, The Mythical Man Month (p 122)

ee also

*Quality control
*Software development process
*Software testing
*Unit testing
*Smoke testing

Notes

References

* Nada daVeiga: [http://www.ddj.com/development-tools/206105233;jsessionid=2HN1TRYZ4JGVAQSNDLRSKH0CJUNN2JVN Change Code Without Fear: Utilize a Regression Safety Net] DDJ, February, 2008
* Adam Kolawa: [http://www.stpmag.com/issues/stp-2008-01.pdf#page=37 Want to Automate Regression Testing? Get Development and QA in Sync] STP, January, 2008
* Adam Kolawa: [http://www.wrox.com/WileyCDA/Section/id-291252.html Regression Testing] , Programmer to Programmer.
* Ramzi A. Haraty: [http://www.amazon.com/dp/B0008EX5T6 Regression testing of database applications] , Journal of Database Management, April 1, 2002, Volume 13, Issue 2.
* Sal Mangano: [http://www.xml.com/pub/a/2002/12/04/xsltcookbook.html Using XSLT to Assist Regression Testing] , xml.com
* Elfriede Dustin: [http://safari.oreilly.com/0201794292/ch08lev1sec4 Automate Regression Tests When Feasible] , Automated Testing: Selected Best Practices, Safari Books Online.

*cite web
last = Savoia
first = Alberto
authorlink = Alberto Savoia
title = Beautiful Tests
work = Beautiful Code
publisher = O'Reilly
year = 2007
url = http://www.junitfactory.com/articles/beautiful-code/BeautifulTests.pdf
accessdate = 2007-11-29


Wikimedia Foundation. 2010.

Игры ⚽ Поможем сделать НИР

Look at other dictionaries:

  • regression testing —    A methodology that supports the continued testing of a program as features are revised or added and compares the results of those tests against the expected results …   IT glossary of terms, acronyms and abbreviations

  • Non-regression testing — (NRT) is an approach of software testing. The purpose of non regression testing is to verify that, after introducing or updating a given software application, previous software functions have not been compromised. This is aimed at preventing… …   Wikipedia

  • Regression — could refer to:* Regression (psychology), a defensive reaction to some unaccepted impulses * Past life regression, (psychology) a process claiming to retrieve memories of previous lives * Software regression, (software engineering) the appearance …   Wikipedia

  • regression test system —    A tool that automates the organization and execution of tests and systematically stores the results. Using the concepts of regression testing, it runs the tests and compares the results against expected results …   IT glossary of terms, acronyms and abbreviations

  • Regression analysis — In statistics, regression analysis is a collective name for techniques for the modeling and analysis of numerical data consisting of values of a dependent variable (response variable) and of one or more independent variables (explanatory… …   Wikipedia

  • Regression toward the mean — In statistics, regression toward the mean (also known as regression to the mean) is the phenomenon that if a variable is extreme on its first measurement, it will tend to be closer to the average on a second measurement, and a fact that may… …   Wikipedia

  • Regression discontinuity design — In statistics, econometrics, epidemiology and related disciplines, a regression discontinuity design (RDD) is a design that elicits the causal effects of interventions by exploiting a given exogenous threshold determining assignment to treatment …   Wikipedia

  • Testing hypotheses suggested by the data — In statistics, hypotheses suggested by the data must be tested differently from hypotheses formed independently of the data.How to do it wrongFor example, suppose fifty different researchers, unaware of each other s work, run clinical trials to… …   Wikipedia

  • Software testing — is an empirical investigation conducted to provide stakeholders with information about the quality of the product or service under test [ [http://www.kaner.com/pdfs/ETatQAI.pdf Exploratory Testing] , Cem Kaner, Florida Institute of Technology,… …   Wikipedia

  • Unit testing — In computer programming, unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an… …   Wikipedia

Share the article and excerpts

Direct link
… Do a right-click on the link above
and select “Copy Link”