1. Introduction
Quality is essentially one of the factors that allows the organizations of Technologies and Information Systems (TIS) to become competitive in current environment. For decades, quality has become the main strategic aim of many organizations because its survival depends on the quality of services provided to internal and external users.
Before proceeding and in order to avoid confusions, we will use the term "TIS" as synonymous of "IT" or "TIC" based on the definition of the ISO / IEC 38500 standard (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008), which defines this as: "the necessary resources to acquire, process, store and disseminate information".
According to (Pérez Juste, López Rupérez, Peralta Ortiz, & Municio Fernández, 2014), quality is defined as " … quality, as a complex reality, must be understood as an inclusive harmonization of these different elements: effectiveness achieving a service, product or excellent object, through efficient processes, that are satisfactory to the direct and indirect recipients, and to the staff of the organization responsible for achieving it”.
In a definitional sense, the scientific and cultural revolution of the organizations in the twenty-first century demands a new paradigm, which is called "inclusive culture" (Pérez Juste, López Rupérez, Peralta Ortiz, & Municio Fernández, 2014), because it belongs to a new conception of the world and life. The inclusive culture includes historical aspects of quality in order to search for a better solution to the problems of the organizations in all of their aspects. The quality stops to focus on the good or on the product and becomes a requirement of the entire organization through its leaders. The role of the management and the employee involvement are the essential factors of change.
In the case of leadership that is understood as a quality criterion, it has been written about the influence of this aspect not only in general, but also in specific and exhaustive of the subject. Among them: Nwabueza (Nwabueze, 2011), Bäcktröm (Bäcktröm, Wiklund, & Ingelsson, 2011), Doeleman (Doeleman, Have, & Ahaus, 2012), Ooi (Ooi, 2012) and González (González Rosas, Carrión García, & Acosta Uribe, 2014) have made studies to analyze it. The leadership has been translated into models of excellence and quality for the evaluation of organizations through sub-criteria, dimensions and components (Benavides Velasco & Quintana García, 2003) (Martínez-Vilanova y Martínez, 2008) (Fundación Iberoamericana para la Gestión de la Calidad [FUNDIBEQ], 2015). Similarly, it has been translated into the principles of ISO / IEC 38500 (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008) standard for the evaluation of the Government of Technology and Information Systems (GoTIS).
The organization directing, evaluating and controling regardless of its quality criterias. The use of TIS should also be directed and controlled, under the support of a GoTIS value-based leadership. In this regard, the guidance provided by the ISO / IEC 38500 (Toomey, 2009) (Fernández Martínez & Llorens Largo, 2012) (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008) standard is appropriate. However, there are related works demonstrating the need for innovation from the perspective of quality assessment in the GoTIS (Sánchez Peña, 2013) (Merchán & Rodriguez, Towards Value-Based Information Technology Leadership Excellence, 2016).
According to directing perspective, there are quality criteria that are covered by a full understanding of the principles of the standard. However, these are not an aim in which the standard is focused. Consequently, this allows reviewing the concepts of "GoTIS quality" and specially the leadership concepts proposed in other studies. This also allows proposing in our context a set of sub-criteria, dimensions and components that guide the evaluation processes of the quality of leadership seen as a criterion. The new model is validated through a research survey (Pfleeger & Kitchenham, 2001), that reflects the opinion of technology and information systems leaders.
This paper has four more sections. The second section provides the background about assessment models from the conceptual point of view, emphasizing leadership. The third section, GoTIS value-based assessment model of leadership is proposed. The fourth section, the proposal is validated. Finally, the conclusions and future works are stated in fifth section.
2. Assessment models
Some evaluation models are described in the context of quality:
Model MBNQA (BNQP - Baldrige National Quality Program, 2015): Is responsible for evaluating and improving the performance through 7 basic criteria: leadership, strategic planning, customer and market focus, information and analysis, the focus of human resources, process management and business results. The program provides a total score of 1000 points in the final evaluation. The model defines leadership as the rank that guides and keeps the organization. It includes the government system, the ethical and social responsibilities, and key communities.
EFQM (European Foundation for Quality Management [EFQM], 2013): It is responsible for improving the business performance by understanding the strengths and weaknesses that need to be improved through the assessment of 5 enablers or "criteria" such as: leadership, strategy, people, suppliers and resources and, processes, products and services; and 4 agents of results that are: customers, people, society and key. The model defines leadership as the model for its value and ethics, being a person who inspires reliability. It looks at the future and makes things happen. Flexible people allow the organization to anticipate and react in time to ensure its success.
Ibero-american Model (Fundación Iberoamericana para la Gestión de la Calidad [FUNDIBEQ], 2015): The model is aimed to the public administration. It consists of 5 facilitators criteria and 4 criteria results. Each criterion has its respective sub-criteria. The facilitators criteria are: Leadership and government style, strategy, people development, resources and partners, and processes and customers-citizens. The criteria results are: Client-citizens, people development, society and globals. All of them assign a total score of 1000 points in the final evaluation. This model defines leadership as the process that directing and guiding the activities of the members of a group influencing on the process to route their efforts towards achieving a goal or specific goals.
There are some works of TIS that have used these assessment models with relevant purposes and that are now taken into account. Such is the case of Municio (Municio & Red Universitaria de Evaluación de la Calidad RUEC, 1998) who led to the creation of the high education quality assessment model with the definition of variables based on TIS. Sanchez (Sánchez Peña, 2013) created two models to assess the quality of GoTIS based on ITIL and COBIT; Merchán (Merchán & Rodríguez, Análisis de los modelos de Gobierno de Tecnologías de la Información y sus relaciones con el Modelo de Excelencia Iberoamericano, 2015) analyzed the relationship between the ibero-american model and ISO/IEC 38500.
3. Proposed quality model
If you do not have an evaluation model and works as described above, it would be an arduous task to determine the GoTIS leadership as a quality criterion. Thus, based on this information, a critical analysis of the criterion of leadership is performed and a description of the sub-criteria, dimensions and components of quality assessment is proposed. This proposal received suggestions from three experts on government. Finally, the operational validation of the proposal took place by a group of twenty-seven leaders of TIS that are on leadership positions and / or Chief Information Officers public and private organizations in Ecuador.
In addition, this work is not about looking adoptions nor adaptations to current dysfunctions. What is sought is an approach to assessing the quality of GoTIS leadership for the local environment in the following terms (Merchán & Rodriguez, Towards Value-Based Information Technology Leadership Excellence, 2016):
1) Identification of basic quality principles on which the proposal is based: A first principle of leadership is in the framework of actions, the systematic support and the commitment of the senior management to quality through the process of GoTIS decision making. In the process of governance, the strategy, mission and vision are formulated, the objectives are expressed, the plans and projects are approved, the resources are organized, the technologies are selected, and the processes and results are evaluated. The leader who supports quality does it consciously and coherently. This creates a culture in which every action and decision is at the service of quality. The evaluation measure is the second leadership principle, which translates the criteria value into control elements that can be evaluated in terms of their level of compliance.
2) Evaluation model: Figure 1 shows the structure of the proposed evaluation model, which includes two quality sub-criteria: Culture of value-based excellence, value-based TIS-Enterprise Architecture; and eleven dimensions (which will be detailed later in this section).
Fig. 1:Structure
of the evaluation model
Each sub-criteria includes characteristics of TIS organization considered necessary to get good results through the leadership criteria and the defined governance style. The content of the sub-criteria meets the quality principles described in the previous subsection, and captures the essence of the reviewed principles of quality models. Both sub-criteria are defined in terms of 11 assigned dimensions.
The hierarchy of the evaluation model to guide the classification of sub-criteria, dimensions and evidence is appreciated in Figure 2.
Fig. 2:Hierarchy
of the evaluation model
The previous model defines the Value at the top of the hierarchy acting as a facilitator that is related directly to the effects of the business. Leadership and its effects are consistent with the perception of value that the business will have. Likewise, the sub-criteria influence on the definition of the dimensions, and these influence on the evidence. It is important to state that the evaluation model provides feedback for quality criteria. The dimensions are designed to measure the achievement of each of the sub-criteria. The dimensions have the characteristic to define the most significant aspects of the sub-criteria. Finally, the evidences, representing quantitative or qualitative data, facilitate the value judgment of the accomplishment of the dimensions on GoTIS team. Table 1 describes the components that make up the leadership criterion.
Table 1
The components that make up leadership
Each sub-criterion is defined by dimensions that represent general aspects of what is included in the model. These in turn are evaluated by five components that make up the evidences: Importance, Diffusion, Utilization, Monitoring and Perceived value. Tables 2 and 3 show the dimensions and effects of the fifth component: Perceived value.
Table 2
Topics to be assessed in the culture of excellence
Table 3
Topics to be assessed in TIS-Enterprise Architecture
3) Method of measurement and assessment: The quality is introduced following a systematic hierarchical process for each sub-criterion. This process allows to get orderly, systematically and consistently the subjective elements of the evaluators and then incorporate quantitative elements estimating the value. The general rating scale shown on Table 4 consists of three hierarchical and independent levels of evaluation that have to be analyzed properly.
Table 4
General rating scale
The first level measures the implementation of GoTIS quality system. The measurement of the implementation is performed on the dimensions that define each sub-criterion. Each dimension is measured in terms of two Implementation components: Importance and Diffusion. The second level measures the application of GoTIS quality system. The Application measurement is performed on the dimensions that define each sub-criterion. Each dimension is measured in terms of two components of Application: Utilization and Monitoring. The third level includes the measurement of GoTIS quality system effects according to the perceived value by the shareholders, executives and/or employees. Finally, the score assigned to each level corresponds to a criterion of importance that it represents.
Each dimension is assessed in terms of its components and each component is valued according to a Likert rating scale with five scores as it is shown on Table 5.
Table 5
Scoring
The concepts of successions and series (Allen R., 2008) are used to develop equations that allow to calculate the components, dimensions, sub-criterion and criterion of the model.
Equations (1) and (2) allow to evaluate the Importance I[i] and Diffusion D[i], respectively; of the i-th dimension to a set of k evaluators.
Where:
score (1) and (2) represent the value assigned by the assessor to the component according to the rating scale.
The level of Implementation of the sub-criterion
IMSC
of the i dimensions set is measured by (1) and (2) as shown in (3).
Where:
Weight represents the highest value of the scoring.
Equations (4) and (5) allow to measure the Use U; and Monitoring S, respectively of the i-th dimension for a set of k evaluators.
Where:
score (4) and (5) represent the value assigned by the assessor to the component according to scoring.
The Application level of the
APSC
sub-criterion of the i dimensions set is measured through (4) and (5), as shown in (6).
Where:
weight of (6) represents the highest value of the scoring.
Equation (7) allows to measure the Perceived Value V[i] of the i-th dimension to a set of k evaluators.
Where:
score (7) represents the value assigned by the assessor to the component according to the scoring.
The level of Effect of the
EFSC
sub-criterion of the i dimensions set is measured through (7) as shown in (8).
Where:
weight of (8) represents the highest value of the scoring.
Equation (9) determines the total measure of a sub-criterion SC.
Where:
IMSC, APSC and EFSC represent the implementation measures, application and effect of the respectively sub-criterion; and, PIM, PAP and PEF represent the weighting values assigned to the implementation level, application and effect respectively.
Equation (10) determines the total measure of the CR criterion and therefore of GoTIS leadership quality system.
Where:
n represents the number of sub-criteria associated with
CR
criterion.
The total measurement of CR criteria will be evaluated according to quality parameters, which were also defined by TIS leading experts. Table 6 shows the level of leadership quality by percentage. This table shows a range of proportional measure between the number of levels that are estimated to have the same importance.
Table 6
Level of leadership quality
The evaluation process should be supported in the use of questionnaires and/or interviews, tables and graphs for each sub-criterion, as part of the work of the evaluation team. The use of tools that support the process to replace the data of leaders avoiding the possible influence of an evaluator and that can influence the process is not excluded.
Finally, the self-evaluation process follows three steps: Awareness, Planning, Implementation, Learned Lessons and Final Report.
4. Validation of the proposed quality model
The validation of the model featured the participation of 27 leading managers of TIS properly chosen by the nature of the research project. A questionnaire of 15 questions was delivered: 4 related to demographic data and 11 related to leadership criteria data.
To collect the opinion of the experienced people, the Likert score was used in the answers on a scale of 1 to 5. Where:
1 - COMPLETE INSIGNIFICANCE
2 - MODERATE INSIGNIFICANCE
3 - INDIFFERENCE
4 - MODERATE IMPORTANCE
5 - COMPLETE IMPORTANCE
The demographics data of the selected TIS leaders are analyzed on Table 7.
Table 7
Demographic data of surveys
Analyzing the demographic results it can be seen that the percentage of leaders based on the primary sector to which they belong is higher in services by 48.1%. This shows that the services sector excels in the use of TIS. The other fact is the gender whose majority participation is by the male gender with 77.8% compared with 22.2% of the female gender. This difference may be circumstantial although it is possible that the male gender actually has greater access to technology for other aspects. The studies of the leaders reflect that most have master's level with 74.1%. Finally, there is the age of the leaders where mostly of them are found in the group of 36-45 years old by 59.2% of the participants.
Meanwhile, the support for the creation of the evaluation model through the importance of the dimensions that define the sub-criteria is evidenced by noting that the percentage of importance (Moderate + Complete) given to each of the dimensions exceeds 90% except for the dimension 2.a (Figure 3 and Figure 4).
Figure 3 ilustrates the importance that the leaders give to the dimensions that define the sub-criterion of culture of excellence, where it is observed that the dimensions with "complete importance" are 1.a and 1.b. The active involvement of the leaders of TIS dominates in these dimensions and in the definitions of key strategic objectives generators of value in the organization. Otherwise, stands the percentage of "complete importance" in the other dimensions.
Fig. 3:Level
of importance of the excellence culture sub-criterion
Figure 4 presents the importance that leaders provide to the dimensions that define TIS-enterprise architecture sub-criterion where it is noted that the dimensions with "completa importancia" are the 2.c and 2.d. The monitoring and control in the delivering value in TIS organization predominates on these dimensions. On the other hand, it should be noted the variety in the level of importance that the leaders provide to the 2.a dimension. According to the expert´s opinion, the definition of GoTIS by the senior executives would have let various interpretations; for example, the participation of the CEO and shareholders that are not exactly part of the senior executives. In any case it is a matter of GoTIS experience. In conclusion, IT stands the percentage of "complete importance" in all dimensions.
Fig. 4:Level
of importance of TIS-Enterprise architecture sub-criterion
With Alpha (α) of Cronbach (Cronbach, September 1951) it is determined the reliability or consistency of the results obtained in the comprehensive opinion survey. For this we proceeded with the analysis using the method of variance of the items of the questionnaire, obtaining a value of 0.885. This result is considered optimal or high reliability in view of the fact that it is close to 1 (Aragón & Sánchez, 2012), interpreted as efficiency in the test.
5. Conclusions and Future Works
The work presented in this paper was supported on related works; therefore, the premise that was used on this proposal was based on using leadership and government style as one of the quality criteria that generates effectiveness and competitiveness to an organization of TIS.
The methodology used on this paper is based on critical analysis of leadership that emphasize the assessment models and related works, which leads to the definition of two sub-criteria, eleven dimensions and five components that received suggestions from experts. Finally, the validation of the proposal took place by a group of twenty-seven leaders of TIS that are on leadership positions and/or on high-level on public and private organizations in Ecuador.
The main reflections of the work are:
The proposal highlights the leadership and government style as a criterion of quality management with directing perspective, as it is the conception of models of excellence that supports them.
The criterion of leadership is known for its commitment to culture of excellence, whose strategic development is supported in TIS-enterprise architecture.
The quality of leadership is given by a set of sub-criteria and dimensions. The dimensions are evaluated by components that stand out on the importance of TIS investigated leaders.
In general, it stands out the "complete importance" that the leaders provide to the dimensions 1.a and 1.b, with 96.3% and 81.5%, respectively. This aspect is explained in the active involvement of the leaders of TIS and in the definitions of the key strategic objectives that generate value in the organization.
Dimensions: 1.c, 1.d and 1.e; stand out by the average percentage of 97% of "complete importance" and "moderate importance”, confirming the active participation of the leaders in GoTIS commitments.
The 2.a dimension brings up attention because of the "divided" opinions; nevertheless, it highlights the most important with 77%. This aspect may be due to the diversity of GoTIS experiences in Ecuador.
Dimensions: 2.b, 2.c, 2.d, 2.e and 2.f show an interesting opinion on where stands out the "complete importance" of the behavior that must have the leaders through the decision making for both control and GoTIS monitoring team.
From the point of view of the results, it is possible to evaluate the leadership not only seeing the satisfaction effect, as it is done, but also seeing the perceived value effect.
The assessment process allows getting in an orderly, systematically and consistently manner the subjective elements of the evaluators and then incorporate quantitative elements in the estimation of the value.
The work highlights the importance of expanding the set of criteria that will lead to a comprehensive solution characterized by the existence and value delivery which meet the national need and the public administration policy issued by the national government. For this reason, the next step of this research focuses on the advancement of a more complete and easy definition, based on this methodology of measurement and assessment from the perspective of direction and/or GoTIS, which will be discussed in future works.
