4.1. Multi-Criteria Decision Making (MCDM)
Multicriteria decision support methods are decision support tools developed around the 1960s[8]. MCDM is a discipline of operations research, which deals with decision problems in the presence of a certain number of criteria [23]. It is used to make a comparative judgment between heterogeneous projects or measures.
There are three main families of methods: methods based on multi-attribute utility theory, over-classification methods and interactive methods.
4.1.1. WSM (Weighted Scoring Method)
- Each evaluation criterion is associated with a weight which represents its importance.
- Each criterion-candidate pair is associated with a score which represents the candidate's ability to fulfill the criterion.
- The candidate with the highest total score is considered the best.
- The most common WSM formula uses addition as an aggregation function:
The major problems of this technique are
- the difficulty of defining a set of relevant evaluation criteria and then assigning them the right weights.
- the evaluation of local scores and the assignment of weights remain manual, which makes this technique tedious when faced with a large number of criteria and candidates.
4.1.2. AHP : Analytic Hierarchy Process
Compared to WSM, AHP helps describe need in an organized way. Firstly, it is necessary to define the main objective to be reached in order to make a decision on the selection of a candidate. The objective is decomposed into a hierarchical tree of criteria and sub-criteria, the leaves are the candidates to be evaluated. AHP allows to define and prioritize the various evaluation criteria with precision.
4.1.3. MAGIQ: Multi-Attribute Global Inference of Quality
MAGIQ was originally developed to validate method results (AHP). Although MAGIQ has not been subjected to extensive research, the technique has proven to be very useful in practice. The MAGIQ technique uses the ROC concept (Rank Order Centroids). Ranking centroids are a way to convert ranks (1st, 2nd, 3rd) into scores or weights which are numeric values. If n is the number of attributes, the weight of the attribute K is:
4.2. Software component selection processes
Several efforts have been made during the last decade to model the selection process.
There is no commonly accepted method for components selection [28]. Nevertheless, the existing selection processes have in common a number of phases which are presented in [18] known as the General Components Selection Process (GCS) which is described as follows:
1. Step 1: The Functional and non-functional Requirements Specification Process: Define the evaluation criteria according to the system requirements.
2. Step 2: Software Component research Processs :
Find components from repositories.
3. Step 3: the filtering process:
Filter search results based on requirements. This allows the definition of a list of promising candidate components that need to be evaluated in more detail.
4. Step 4: Evaluation Process:
Evaluate the candidate components of the shortlist to select the best candidate.
4.2. 1. OTSO (Off-The-Shelf-Option)
This process is developed by J. Kontio. Figure 2 shows the main phases of the OTSO process [14] as well as the parameters and data used.
- Phase N ° 1: Definition of evaluation criteria The first step consists in defining the evaluation criteria according to four parameters related to the application in which the candidate will be integrated: The first parameter (Requirement specification) relates to the specification of the needs of the application, this includes in particular the functional requirements. The second parameter (Design specification) concerns the specification of the overall architecture of the application. The third (Organizational characteristics) and the fourth parameter (Project plan) are the organizational and project constraints related to its development.
- Phase N ° 2: Search , the search in the selection process consists of identifying the potential.
- Phase N ° 3: Screening consists in filtering the most relevant candidates by means of basic criteria and certain information about these candidates (COTS Sources).
- Phase N ° 4: Evaluation, consists in evaluating in detail each candidate by means of more precise criteria.
- Phase N ° 5: Analysis of results The final phase consists in analyzing the results of these evaluations according to the previously defined criteria.
A major disadvantage of the OTSO process is that it does not propose anything to specify the needs. Moreover, it is mainly the functional needs of the application that are taken into account, to the detriment of non-functional needs.
4.2.2. PORE (Procurement-Oriented Requirements Engineering)
Maiden and Ncube have developed the PORE method. Basically, PORE defines three types of needs: main functional needs, secondary functional needs, and: non-functional needs. In order to satisfy these three types of needs, PORE uses an iterative process, decomposed into generic processes:
- Identify Product: identifies potential candidates through market research or expertise.
- Acquire Information: This process involves obtaining information about the customer requirements and the product requirements. The acquisition of information is done through use cases and knowledge engineering techniques.
- Acquired Information Analysis: consists in analyzing the information acquired to produce data that can be used in the following process.
- Decision Making: The purpose of this process is to determine whether the candidates meet the needs or not, using multi-criteria decision-making (MCDM) techniques.
- Product selection: involves rejecting candidates that have been assessed as unsatisfactory by the previous process.
Once the selection is made, two choices are possible:
- Either the operation is recommenced by returning to the first or second process to detail the needs and reduce the list of candidates,
- Either we stop the selection when we consider that the query is satisfied. Each process can be repeated several times in case the information is insufficient.
The PORE process has some disadvantages, because it is sometimes difficult to know how to stop the iteration [16]. And in [17], the authors find it unclear how to use the needs in the evaluation process, as well as how to eliminate candidates
4.2.3. CRE (COTS-based Requirements Engineering)
CRE [17] is an approach similar to PORE, it has four iterative phases:
- Phase N ° 1: Identification: Define objectives based on: user needs, application architecture, project objectives and limitations, component availability, and organization infrastructure.
- Phase N ° 2: Description: In this phase, the evaluation criteria must be elaborated in detail, emphasizing the role of NFR (Non Functional Requirements). The CRE method uses the NFR Framework for representation and analysis of non-functional requirements. This Framework is a process-oriented approach where NFRs are explicitly represented as objectives to be achieved. These objectives will then be broken down into sub-objectives using a tree structure and logical operators connecting the parent node with its children.
- Phase N ° 3: Evaluation: The decision to select a particular product COTS is based on the estimated cost compared to the analysis of the benefits of each COTS alternative. The CRE process suggests the use of cost models such as COnstructive COTS (COCOTS).
4.2.4. SCEP (Software Component Evaluation Process) :
In [25,26] two evaluation processes are proposed, the first concerns the component quality evaluation and the second the assembly evaluation. The evaluation process is based on two functions :
- function of representation: β
Ei β ejj=1,...,n means that the element Ei is presented by the set (e1, e2, e3,…,en).
Two cases can be distinguished:
- ∀ci ∈ C [1] , ∀aj ∈ A [2] / ci β aj ⇒each criterion cj is represented by a set of atributes.
- ∀fj ∈ F [3] , ∀ci ∈ C / fj β cl ⇒each factor fj is represented by a set of criteria
- Evaluation function: £
If Ei β ej/j=1,...,n ⇒ V(Ei) =£ (V(ej))
If the element Ei is presented by the set of elements ej/j=1,...,n the quality value of Ei: V(Ei) can be calculated based on the quality value of the elements ej: V(ej).
Process 1: SCEP (Software Component Evaluation Process )
SCEP contains three main steps:
- Quality characteristics classification: in this step, the user must classify the quality characteristics (factors) according to his needs.
- Weightings calculation: in this step, the rows are converted into weighting using ROC concept.
- Evaluation: the evaluation processes is based on the relation β and the function
Process 2: SCAEP (Software Component Assembly Evaluation Process)
This process is based on component quality evaluation process. It comprises four steps:
- Consists in fixing the desired quality value.
- Construction of the application by assembling the suitable software components: to determine all possible compositions (all candidate systems Sn/n = 1,…,g, g is the number of possible compositions).
- Evaluate the provided quality (PQ) by each system then compare it with the desired one.
5. Comparative study
This part represents a comparative study of four software component selection processes . The following table (Table 2) shows if the selection process does not cover or provides partial or total coverage of the different phases of General Components Selection ( GCS) process :
GCS steps
Process
|
Functional Requirements Specification
|
Non-Functional Requirements Specification
|
Filter search results
|
Evaluation
|
PORE
|
Total
|
/
|
Total
|
Total
|
OTSO
|
Total
|
/
|
Total
|
Total
|
CRE
|
Total
|
/
|
Total
|
Total
|
SCEP
|
Total
|
Total/ partial
|
Total
|
Total
|
Table 2. Coverage of GCS main phases .
The comparison shows that OTSO, PORE , CRE processes do not sufficiently deal with non-functional needs.
The SCEP Process allows a specification of functional and non-functional needs using SCM (Software Components MetaData) [26] which provides the information necessary for the selection and evaluation of components. SCM is designed into two parts:
- The first concerns the functional aspect: the developer must give a description of all the services provided and required, specifying key words.
- The second part covers the non-functional aspect.
Another characteristic common to all these methods is the parallelism between the specification of requirements and the selection and evaluation of products. Moreover, these processes are very often iterative. As requirements are collected, some candidates are rejected, new candidates appear and the analysis of these new candidates in turn reveals new requirements. This characteristic of parallelism and iteration is very important and even essential to any method of selecting software components.
Components are evaluated by a set of criteria that represent system requirements, goals, and constraints. Kontio et al [29] suggest defining the evaluation criteria hierarchically, where a set of abstract goals are progressively refined, based on factors such as software application requirements, application architecture, and existing product capabilities.
Three strategies can be followed to evaluate software components [30] :
1. Progressive filtering : this method requires running GCS process iteratively until a small number of components identified from which one or more can be selected for integration into the system. The process starts with a large set of software components, and then gradually defines discriminating criteria by successive iterations, the "less suitable" products are eliminated.
2. Puzzle assembly : implies that a software components based system requires the assembly of various components like pieces of a puzzle.
This strategy considering the requirements of each product while simultaneously remembering the requirements of the other products in the puzzle.
3. Keystone identification: starts by identifying a key requirement, then searches for products that meet that fundamental requirement. This allows the possibility of eliminating a large set of candidates that do not meet the key requirement.
5.1. Comparing software components selection process
This section presents a survey and comparison of four software component selection processes: Table 3 compares these approaches in terms of the following criteria:
- GCS: Conformance to the GCS method (cf. Table 1 )
- EVAL: Evaluation strategy used.
- PF: Progressive filtering ,
- KS: Keystone ,
- PZ : Puzzle assembly.
- SNG: single selection.
- MLT : multiple selection.
Selecting a single software components focuses more on functional requirements than the non-functional ones and does not address the interoperability problem .
Usually, there is no single component satisfying all the defined requirements. Hence, it is necessary to have adequate methodologies for multiple software selection.
- ADAPT: Adaptability of the process to different contexts.
- TOOL: Disponibilité d'un support d'outils pour faciliter l’application de la démarche
criterion
Process
|
GCS
|
EVAL
|
SNG
|
MLT
|
ADAPT
|
TOOL
|
PORE
|
~
|
PF
|
✓
|
X
|
X
|
✓
|
OTSO
|
~
|
PF
|
✓
|
X
|
X
|
X
|
CRE
|
~
|
PF
|
✓
|
X
|
X
|
X
|
SCEP/ SCAEP
|
✓
|
PF
|
✓
|
✓
|
✓
|
~
|
✓ : fully satisfies the criterion
X : does not satisfy the criterion
~ : partially the criterion
|
Tableau 3 : Comparing software components selection process