Test Execution

Executing a formal test or informal test cases to confirm the business requirements and to identify the defects is called Test Execution

Test execution is the process of executing the code and comparing the expected and actual results.

Test case. A collection of one or more test steps designed to exercise a small number of related test

conditions.

Test step. A short action, such as entering a page of data, that produces a desired test condition.

Test condition. Some interesting situation in which we want to place the system under test for

purposes of looking for invalid behavior, response, and/or output.

Test suite. A collection of related test cases.

Following factors are to be considered for a test execution process:

  • Based on a risk, select a subset of test suite to be executed for this cycle.
  • Assign the test cases in each test suite to testers for execution.
  • Execute tests, report bugs, and capture test status continuously.
  • Resolve blocking issues as they arise.
  • Report status, adjust assignments, and reconsider plans and priorities daily.
  • Report test cycle findings and status.

Error,Defect,Failure

ERROR

in the application which is created. A programmer while designing and building the software can make mistakes or error. These mistakes or errors, A mistake made by programmer

Defect

When actual result deviates from the expected result while testing a software application or product then it results into a defect. Hence, any deviation from the specification mentioned in the product functional specification document is a defect. In different organizations it’s called differently like bug, issue, incidents or problem.

Failure

When the result of the software application or product does not meet with the end user expectations or the software requirements then it results into a Bug or Defect. These defects or bugs occur because of an error in logic or in coding which results into the failure or unpredicted or unanticipated results.

 

 

Requirement traceability matrix

Requirements traceability matrix is a table that shows the many to many relationship between requirements and test cases.  It is used to track the requirements and to check the current project requirements are met.

Requirement Traceability Matrix

Requirement Traceability Matrix or RTM captures all requirements proposed by the client or development team and their traceability in a single document delivered at the conclusion of the life-cycle.

In other words, it is a document that maps and traces user requirement with test cases. The main purpose of Requirement Traceability Matrix is to see that all test cases are covered so that no functionality should miss while testing.

Requirement Traceability Matrix – Parameters include

  • Requirement ID
  • Risks
  • Requirement Type and Description
  • Trace to design specification
  • Unit test cases
  • Integration test cases
  • System test cases
  • User acceptance test cases
  • Trace to test script

 

What is Test data

Test data is the data that is used in tests of a software system.In order to test a software application you need to enter some data for testing most of the features. Any such specifically identified data which is used in tests is known as test data.

You can have test data in excel sheet which can be entered manually while executing test cases or it can be read automatically from files (XML, Flat Files, Database etc.) by automation tools. Some test data is used to confirm the expected result, i.e. When test data is entered the expected result should come and some test data is used to verify the software behavior to invalid input data.

Test data is generated by testers or by automation tools which support testing. Most of the times in regression testing the test data is re-used, it is always a good practice to verify the test data before re-using it in any kind of test.

 

Test Case

Test Scenarios

Test Scenario – A Scenario is any functionality that can be tested. It is also called Test Condition or Test Possibility.

Test Scenario is ‘What to be tested’

Test scenario is a combination of test cases which defines what is to be tested on an application/feature, or simply test scenarios are the series of test cases. Suppose you are testing a login form. Then, the test scenario would simply be a single sentence i.e “Ensure the working of login form”. Test scenarios mainly focus on the functionality and not the input datas.This document includes test conditions, where as test cases are the step by step procedure to be followed to meet this condition.

Test Design

Testplan VS Test strategy

Test Plan Test Strategy
A test plan for software project can be defined as a document that defines the scope, objective, approach and emphasis on a software testing effort  Test strategy is a set of guidelines that explains test design and determines how testing needs to be done
 Components of Test plan include- Test plan id, features to be tested, test techniques, testing tasks, features pass or fail criteria, test deliverables, responsibilities, and schedule, etc.  Components of Test strategy includes- objectives and scope, documentation formats, test processes, team reporting structure, client communication strategy, etc.
 Test plan is carried out by a testing manager or lead that describes how to test, when to test, who will test and what to test  A test strategy is carried out by the project manager. It says what type of technique to follow and which module to test
 Test plan narrates about the specification  Test strategy narrates about the general approaches
 Test plan can change  Test strategy cannot be changed
 Test planning is done to determine possible issues and dependencies in order to identify the risks.  It is a long-term plan of action.You can abstract information that is not project specific and put it into test approach
 A test plan exists individually  In smaller project, test strategy is often found as a section of a test plan
 It is defined at project level  It is set at organization level and can be used by multiple projects

Test plan

A Software Test Plan is a document describing the testing scope and activities. It is the basis for formally testing any software/product in a project.

ISTQB Definition

  • test plan: A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice,and any risks requiring contingency planning. It is a record of the test planning process.

IEEE 829 test plan structure

  • IEEE 829-2008, also known as the 829 Standard for Software Test Documentation, is an IEEE standard that specifies the form of a set of documents for use in defined stages of software testing, each stage potentially producing its own separate type of document.

    COMPONENTS OF THE TEST PLAN DOCUMENT

  • Test Plan id
  • Introduction
  • Test items
  • Features to be tested
  • Features not to be tested
  • Test techniques
  • Testing tasks
  • Suspension criteria
  • Features pass or fail criteria
  • Test environment (Entry criteria, Exit criteria)
  • Test deliverables
  • Staff and training needs
  • Responsibilities
  • Schedule

 

 

Test Strategy

A Test Strategy document is a high level document and normally developed by project manager. This document defines “Software Testing Approach” to achieve testing objectives. The Test Strategy is normally derived from the Business Requirement Specification document.

The Test Strategy document is a static document meaning that it is not updated too often. It sets the standards for testing processes and activities and other documents such as the Test Plan draws its contents from those standards set in the Test Strategy Document.

Some companies include the “Test Approach” or “Strategy” inside the Test Plan, which is fine and it is usually the case for small projects. However, for larger projects, there is one Test Strategy document and different number of Test Plans for each phase or level of testing.

COMPONENTS OF THE TEST STRATEGY DOCUMENT

  • Scope and Objectives
  • Business issues
  • Roles and responsibilities
  • Communication and status reporting
  • Test deliverables
  • Industry standards to follow
  • Test automation and tools
  • Testing measurements and metrices
  • Risks and mitigation
  • Defect reporting and tracking
  • Change and configuration management
  • Training plan