Prioritize the Sequence of Test Cases and Increase the DRE Rate by Implementing Orthogonal Array Testing Strategy


 Software testing is an emerging technology which is use to increase the rate of error detection as early as possible in the software testing life cycle process. Test case prioritization technique plays a crucial role in organizing the test cases in sequencing order both ascending and descending such that test cases having high priority or high severity are planned to get executed doing a proper risk-based analysis. This prioritization technique effectively addresses two important organization constraints namely “Time” and “Budget”, also improve the quality of service. The proposed work is all about how effectively we can sequence the application modules for testing during Test plan phase using fuzzy logic and how to write optimized test cases efficiently design during Orthogonal Array Test Strategy (OATS) in Test design phase of testing life Cycle.


Introduction
Software Testing isn't done all in all, it focuses on various test stages -Test Plan, Test Analyze, Test Design, Test Execution, Defect Reporting/Tracking and so on., Writing test cases based on speci cations are the building blocks of testing that mainly occurs in test design stage. Test case experimental sequencing or organizing the test cases is a procedure of requesting the experiments as per business requests to accomplish a speci c objective. On the off chance a similar imperfection is recognized in plan or coding phase of the product improvement life cycle, it will expand the Cost of Quality (COQ). COQ includes three different costs: A. Prevention costs B. Appraisal costs C. Rework costs. Prevention costs are the costs that are spent in organizations to prevent the defects. Example: Training costs. Appraisal costs are the cost incurred for appraising defects. Example: A tester identifying critical defects will get paid for his/her efforts. Rework costs are the costs spent in backtracking and reworking on any already completed work. Example: Requirement defects identi ed during testing phase must get reworked again from the scratch. Any project that targets on ensuring quality mainly focuses on minimizing the cost of quality (COQ).
Testers usually have a different mind-set from that of the developers who coded the application. Since people can hardly nd their own mistakes, an independent team is recommended who can able to test the product quality in a cost-effective way. The higher the independence, the more it can contribute to nding defects, focus on the testing efforts, and provide an independent view on the product.
Software testing helps in the real world to reduce cost and time, to maintain reputation, to meet contractual or legal requirements. Phases of software testing can guarantee that the application meets all the execution related measurements, for example, reaction time, accessibility, and limit necessities.
Any algorithm that targets on test case prioritization needs to take care of these two parameters for successful completion of the project ensuring "quality" minimizing quality costs.
There is a technique widely used in the industry called Boundary value analysis (BVA) to prioritize the test cases for testing. Here extreme boundary values are considered while designing the tests. A boundary is the place where the system's behavior is expected to change. A boundary can also be de ned as a limit where a valid value turns into an invalid value. In BVA, boundaries between equivalence partitions are tested. BVA is also known as 'Range Checking'. The boundary values for the most part incorporate greatest, least, only inside/outside, common esteems, and blunder esteems. The Test Lead is the person who designs it when the testing courses of events are characterized. Equivalence Partitioning and BVA are related to each other and can be used together at all levels of testing. Using the BVA technique will allow you to be more successful in your testing over equivalence partitioning, only in that defects are more likely to occur at boundary points between partitions. This technique believes that more errors are prone to occur as clusters in bounding edges. This validates one of the 7 principles of testing -"Defect Clustering" or "80-20 rule". This technique believes the principle that 80% of defects are revealed in 20% of the overall code. This technique is suited for projects where we have multiple test partitioned identi ed with different expected outcomes.
Decision table is another popular technique used for test case optimization which targets at system level considering causes (inputs) and effects (outputs). This is also otherwise called as "Cause-effect Graphing". This technique is suitable for problems that has huge number of inputs (causes), on combining provides huge number of different outputs (effects). This technique is used to solve real world business problems that have varying combinations of test inputs which provides different set of test outcomes. The fundamental preferred standpoint of this system is that it guarantees all basic test mixes are secured staying away from redundancies and miss-outs. Decision tables are also a very good method to deal with a combination of inputs producing different results. The Decision Table method is very signi cant because, with this method we can test multiple combinations. The number of possible combinations is given by 2 ^ n ('n' being the number of inputs). We can choose a rich subset of all possible combinations using this method. The test lead is the one who plans it when the testing timelines are de ned. Decision Table testing is a technique that assists in breaking down the complex logic by recording business rules based on a set of test conditions. The Decision Table technique helps to test business rules against various system conditions (user inputs) and corresponding actions (system outputs). This technique analyses two parameters: Conditions: Represent process or application inputs.
Actions: Represent what should be done or what should happen for each set of conditions. The steps testers use to build a decision table includes, identify conditions and their values, listing all possible actions, identifying all condition combinations, identifying all actions for each condition combination to create business rules and link the rule to the corresponding actions. Tabular format is the best way to represent complex business rules and analyze them for test design.
The next technique mainly used in organizations for effective test designing is State transition testing. It is used to determine the invalid system transitions. This testing technique is used in the application design. State Transition testing technique represents the system behavior as nite number of states. This technique is useful when analysis and system testing are dependent on previous and current behavior, previous and current states. The test lead prepares it when the testing timelines are de ned. It chooses the starting point, where we begin examining an application or system. Understand all the states that an object or user can be in. Identify the transitions (events, conditions, actions) that are applicable to each state. Build a table or diagram to describe the states transitions. The limitation with the State Transition diagram is that it does not allow users to completely describe a complex or a big process with many branches due to lack of space. It is a fact that the usability of over complicated State Transition diagrams degrades signi cantly. State Transition tables must be utilized to list all blends of the states, conditions, and activities, that assistance to thoroughly dissect the framework advances.
Utilizing use case-based testing is a discovery test that outlines system in which TCERs are intended to execute client situations that are characterized by the base and elective streams of a utilization case. Utilize Cases are client situated portrayals of the application's usefulness. A Use Case portrays an arrangement of activities that an application performs for a speci c Actor. Utilize Cases are intense technique for portraying necessities. They frame reason for Software Design, Test cases, User documentation and GUI outline. By having an arrow connecting the Actor and the Use Case, the Tester knows the actions or features the Actor can perform on the system under test.
Orthogonal Array Test Strategy (OATS) is a discovery test advancement system which utilizes a factual method for recognizing the improved test suite. This technique is well suited when the number of test inputs is relatively huge and where exhaustive testing goes ineffective, testing all possible combinations. Say for example, if the requirement is to test a train ticket booked through IRCTC mobile app, the factors (input combinations) needs to be tested are "the number of passengers travelling", "ticket number", "PNR number", "seats allocated", "train number", "train name" etc., Considering each factor identi ed can have different values. Say for example: the factor "seats allocated" can have multiple values if the numbers of passengers travelling are more. This type of problems will have huge challenge identifying the number of factors and for each factor identifying the number of levels. Such problems that can have factors and levels are what we call as a "combinatorial" problem. OATS technique is mainly suited for combinatorial problems. Identifying all possible factors and levels and writing effective test cases for every possible combination is extremely challenging and impractical to achieve.
Orthogonal Array Test Strategy is a test optimization technique that concentrates on identifying and prioritizing the test combinations which ensures maximum test coverage identifying major defects with minimum number of test combinations. OATS is targeted on identi ed different modes of defects that occur in real life projects. A defect that is occurred because of a single statistical input is called as a "Single mode" defect. A defect that is occurred combining two statistical inputs is called as "Double mode" defects. Like the same way, there can be triple mode, multi-mode and so on. Universal researches have proved that maximum of real-life projects fail because of double mode combinations of defects.

Existing Work
Numerous works relating to experiment prioritization has been proposed and executed by numerous analysts before. A portion of the couple of essential work has been referred in this segment. Existing approaches are used [1], [2] derived dependency structures among various test cases in a test suite is performed manually that are consumes more time. Hai dry's procedure [3] is utilized for discovering absolute number of wards for each experiment utilizing Dependency Structure Prioritization volume isn't work legitimately. Existing approaches [6] that uses surmised dependence structures among different examinations [12] of test cases are performed manually that exhausts greater chance to test the action by an analyzer and furthermore tests the system time.
Dan Hao, Lu Zhang et al., [16] actualized the perfect ideal experiment prioritization method, which plans the execution request of experiments in view of their identi ed shortcomings utilizing ideal scope-based experiment prioritization as a number straight programming (ILP) issue yet not in more prior stage keeping in mind the end goal to deliver the outcome in streamlined way. Already Alessandro Marchetto et al., [21] proposed method that goes for both early nding de ciencies and diminishing the execution cost of experiments by applying a metric-based way to deal with consequently distinguish basic and blame inclined bits of programming relics, in this way getting to be ready to give them more signi cance amid experiment prioritization however in the outline period of testing not in arranging stage. Hence, my proposed work an effective test sequencing methodology which identi es the best sequence and worst sequence of module identi cations for testing using dependency structure matrix during the planning phase of testing life cycle. This overcomes all the existing research techniques [23,25] as my propose our work optimizing test quality during test plan itself, identifying the best and worst sequence of modules for testing followed by optimizing the test suites identi ed during test design phase customizing OATS techniques optimizing single, double and triple mode test combinations.

Proposed Work
As like software development life cycle, we do have software testing life cycle which core software testers in software project will work on. It includes the following stages: Requirements, Test Plan, Test Design, Test Execution and Defect reporting & Tracking. In Requirements stage, testers will analyze the requirements ensure whether it is exible enough to achieve in terms of budget, technology, practicality of requirements. A thorough feasibility study is required to take the testing process to the next stage. Generally during this stage, testers lack clarity on the requirements, dependency on all requirements, mainly on how to follow testing in a sequence -which modules of application to test rst, test next and test last for early detection of defects. This has become a huge challenge for testers working in Test Plan phase to decide on the ow of testing, plan budget and schedule timelines in a testing project.
The proposed idea of test sequencing using dependency structure matrix provides a methodology for testing using fuzzy logic to decide on the best sequence of testing called as "As early as possible sequencing" and the worst sequence of testing called as "As late as possible sequencing" which is referred in short as AEAP and ALAP respectively. This test sequencing of modules identi cation using fuzzy logic identifying the intermediate modules which are partially true or partially false is very much appropriate during Test plan phase rather than Test design phase as other proven techniques exist shows in g.1. Implementing effective test sequencing using Fuzzy logic mechanism provides a valuable exibility for reasoning. The main advantage of using fuzzy logic is that it resembles human reasoning in many ways picking the right modules for test sequencing deriving 2 main fuzzy sets AEAP and ALAP. In our fuzzy sets -AEAP and ALAP, every module identi ed as sequence are elements in the fuzzy set. So, consider our fuzzy set as "As early as Possible" sequence set (AEAP), the members of the set will be the modules -Login, Bank Account, Credit cards etc., considered in sequence. Modules considered for testing in the rst release R1, will form the fuzzy set of AEAP and ALAP and each module considered for release 1 will be assigned a membership value. Fuzzy logic systems AEAP and ALAP are generally governed by membership functions. Using fuzzy sets has an edge over classical sets, as it allows partial membership and it contains elements of varying degrees of membership. This enables to sequence the modules for testing more effectively and e ciently so that considering the sub-modules, external modules holding partial membership to a greater extent in AEAP and ALAP sequence. In test sequencing the modules for testing gives a better view to the testers to plan effectively and e ciently the ow of subsequent testing phases which in turn helps the project minimizing defects, rework and able to deliver the project well ahead of scheduled time.
After deciding the best sequence or worst sequence considering the risks, availability and other factors, moving to test design phase and writing quality test cases for the identi ed test sequence is a time killing process among the entire testing life cycle. Our goal is to identify a suitable test optimization technique for creating effective test cases which identi es maximum defects with minimal efforts. The entire architecture as shown in g.1

Test Module Sequencing In Test Plan Phase
My proposed work consists of a real time banking project which has the following core modules like Login, Loans, Bank Account, Credit cards, Currency conversion, Fund transfer, Database and Account summary. The ow graph of the same is shown in g.2.
Totally 8 core modules to get tested both individually which includes of Unit Testing and as a whole testing the interactions and the dependencies of one module over the other modules. Generally, in a typical testing project once the testers receive the modules for testing, there is no de ned way so far as on how to start testing or which module to start testing rst and move on till all functionalities are covered. It is not always necessary that the rst module should get tested rst, in our case Login must get tested rst. What if, the rst module is still under development or under design where testing team doesn't have a better clarity to start testing it? There are situations where we will get only the design document that de nes the modules, ow of execution, but the actual modules are still in development phase with the developers. Test sequencing is an approach that tells the testing team in which order we need to test the modules to save time, budget, man-power etc., Almost in all projects till today, test sequencing is decided individually by the project folks considering the availability of the modules, resources, ease of use etc., There is no standard way of deciding the best approach of Test sequencing and worst approach of Test sequencing of modules.
My research proposes a standard way of deciding the best possible sequence for Testing and the worst possible sequence for testing the modules using Fuzzy sets. For deciding this, the dependency matrix is important which has the information about relationship of one module over the other. The proposed concept takes the dependency information of modules in a excel table called as "Dependency matrix" as mentioned below in the gure below. Dependency matrix acts as an input here. Dependency matrix for the mentioned banking project is as below table.1. Table.1.Matrix representation of dependency structure.
The grey celled "1"s is self-dependency and the other "1"s are the dependencies of modules based on the requirements shared by the customer based on the ow graph. Once input these details, our proposed concept on test sequencing using fuzzy logic identi es the intermediate modules which are partially true and partially false forming the fuzzy sets. All the information in the identi ed fuzzy sets are discrete and they represent the degree of truth in the fuzzy logic system. Here we deal with partial true and partial false levels; we identify membership levels for each module assigned to level. These membership values depict the degree of truth in the identi ed fuzzy systems. The 2 major fuzzy systems -"As early as possible sequencing (AEAP)" and "As late as possible sequencing (ALAP)" forms the base of our test sequencing way of testing the modules considering the risks, assumptions and other factors that impact the testing process in subsequent stages of testing life cycle.
The "As early as possible sequencing" or "best sequencing" of bank modules represented in fuzzy systems is as below in Fig:3 The sequence structure of modules placed for possible sequencing in levels identi ed (1-4) as shown in g.4.

Test Case Optimization In Test Design Phase
Test case optimization is a technique using which we can minimize the number of test cases planned for testing ensuring maximum coverage with minimal testing effort. It will not ensure 100% test coverage. It satis es one of the 7 testing principle -"Exhaustive Testing is not possible". Orthogonal Array Test Strategy has given a combinatorial problem which consists of "n" factors and "m" levels, OATS technique will ensure all double mode combinations of inputs are tested 100%. Any tools/techniques which follow OATS principle should follow "Double-mode combinations are tested 100% in the optimized suite of test cases.
My proposed work analyzed Combinatorial Problem in Real life for a retail supermarket project billing possibility. A customer who purchases products in this supermarket must answer the following questions during billing at the counter. The factor "Product category" indicates the type of purchased item. Meaning assumptions include 2 levels -Loose (item given in loose based on buyer's need) and Packed (packaged item/sealed item). The next factor is "Discount". Is price cut or discount applicable for the item purchased? It accepts 2 levels as answers -Yes and No. The third factor is "Tax". What kind of tax is applicable for the buying item? Possible answers can be the identi ed levels like: VAT, Duty, Sales etc., the last factor can be "Mode of Payment". What is the possible mode of payments accepted in the supermarket for the item purchased? The applicable levels could be: Cash, Credit Payment, Debit Payment, Google Pay etc.

Results And Outputs
Factors & Levels Abstraction: The following table.2 represents the factors name and level for the retail marketing project. Once factors and levels are identi ed, selecting the precedence factor is quite important. At times, customer may prefer certain factors of top priority and need those factors to appear a greater number of times in the optimized test suite. Precedence factor selection decides the level chosen as priorities, will appear a greater number of times in the optimized output.

Optimized Outputs with Custom Precedence Factors
The sample output below shows the optimized output generated for "Custom" article category with greater precedence. We can able to see more test case combinations have "Custom" category in it.  When we try to generate a pair-wise combinatorial output using any OATS, we will get a combinatorial optimized test suite shown in table.6. Table.6.Combinotorial optimized test suite.
The tool reasonably considers the factors and levels provided as statistical inputs for the combinatorial output generation, rather than considering the real meaning of the inputs provided. In the above analysis, the combination with Passport -No and Visa -Yes is irrelevant. It's illogical to create a test case like Passport -No, Visa -Yes, Location -Onsite. There is no question of Visa, if an employee doesn't hold a passport. Such a combination is called as "Infeasible Combination" We can well in advance avoid generating such an infeasible combination by providing the combination as a infeasible input before generating the test optimization suite.
Infeasible Combination Input: "Passport -No, Visa -Yes, Generate no test case" Considering the infeasible input, the optimized output is as below table.7.

Comparison Analysis And Discussions
The following are implications that we want to show from our usual way of testing applications against implementing my proposed work. Fig.7 represents the total no of test cases written for the given requirements. Only it needs half have from existing system. Also it shows the increased rate of DRE against existing system.
We see a decent raise in the Defect Removal E ciency(DRE) rate which we usually use to analyze the test effectiveness . In future my work can be extended with any other real time projects analysis and increase the DRE rate further.

Declarations Ethical Compliance
Not applicable

Con icts of Interest
There is no con ict of interest

Funding
There is no funding Figure 1 Architecture diagram of Test Sequencing and Test case optimization   Comparison chart for proposed system.