A Validity and Reliability Study of the Basic Electronics Skills Self-Efficacy Scale ( BESS )

The aim of this study is to improve a measurement tool to evaluate the self-efficacy of ElectricalElectronics Engineering students through their basic electronics skills. The sample group is composed of 124 Electrical-Electronics engineering students. The validity of the scale is analyzed with two different methods through factor analysis and distinctiveness. To evaluate the how much each item that exists in the scale can measure the factor they belong to, item total factor correlations and corrected correlations are calculated on the data. According to the acquired values, each item and each factor in the scale are found to serve to the run-of the scale and the aim of scaling the desired facility, in a meaningful level. Moreover, analyzing the t value related to the differences between the groups of top 27% and bottom 27%, the item distinctiveness’s are researched and it is detected that the distinctiveness of both of the run-of the scale and each one of the items is high level; in other words, it is detected that each item is distinctive in the desired level. The internal consistency coefficients of the scale is calculated using two congruent halves correlations, Cronbach Alpha, Sperman-Brown formula and Guttmann split-half reliability formula. Consequently, it is concluded that the scale is a reliable and valid scale and this scale can be used to determine the basic electronics skills of the Electrical-Electronics Engineering students through their selfperceptions.


INTRODUCTION
The main aim of engineering education is to develop the designing skills and to solve the design problems.Engineering, in general, can be considered as a road that starts with analysis, goes through synthesis by finding solutions to the needs of the society [1].It is a stubborn fact that we are related to the scientific and technological innovations in the whole of our daily life.Engineering is the key component of the technologic society and inventiveness [2].In this context, many countries feel the need of reviewing their system of education grounding on Science, Technology, Engineering and Mathematics (STEM) quartet [2].FeTeMM, is an educational approach that focuses on integrating the skills and knowledge about science, technology, mathematics and engineering areas with an engineering design oriented education, and aims to make the students obtain the skills of interdisciplinary collaboration, systematic thinking, being open to communication, belonging the ethical values, researching, producing, creativity and solving the problems in the most appropriate manner [3][4][5].STEM, in general, can be defined as a complementary educational approach, which emphasizes that the skills that constitute the basic foundation score of the technology education should be earned together [6].Therefore, it is possible to state that STEM constitutes the foundation of the engineering education [7].
It is possible to say that depending on the need for technology, the fact that engineering is strategically very important, causes the engineering education need to be more well-qualified and accreditation subjects remain on the agenda.In this frame, curriculum development and alternative learning-teacher studies continues increasingly.STEM is one of the important products of this process.Today's engineering graduates are responsible for solving the problems of a tomorrow in a world that develops fast and faces difficulties that are much more critical than it was in all the times [8].Therefore it is inevitable that they need to have several high-rank skills.Correspondingly, it is possible to state that there is an important demand for engineering in the societies.
Different educational approaches can be observed in engineering; in addition to this, it is inevitable for engineering education to change and develop for catching the novelties of the time.Together with the respective alteration and developments, engineering profile and therefore the effect of engineers on the society also changes [9].Just like the fact that the alignment of the engineering education with the novelties of the time effects the outcomes of this occupation; it is possible to say that the attitudes of engineering candidates towards engineering education and occupation effects the outcomes of this occupation through academic success and performance in the occupation in a similar manner.When considered in this frame, it does not see possible that the engineering profile decked out with only theoretical knowledge can fulfill the advanced technology needs that is being lived today.Therefore, an engineering profile that possesses engineering skills alongside with fundamental theoretical knowledge is required.Within this scope, it is possible to say that nurturing engineers that performs enough application has laboratory and studio experience is necessary.Nowadays, as the accreditation institutions noticed this necessity too, they emphasize the engineering sufficiency frequently.For example, according to [10] "the output of the program should involve all of the components that defines the knowledge, skills and behavior that the students should earn until they graduate and are necessary for achieving the program's educational goal after the graduation; and should be defined in a way that it involves the 11 qualities that [10] specifies."Moreover, MÜDEK [10] embraces the program outputs as an important criterion in the accreditation process.While evaluating the program out-PAPER A VALIDITY AND RELIABILITY STUDY OF THE BASIC ELECTRONICS SKILLS SELF-EFFICACY SCALE (BESS) comes "Engineering programs should prove that the students on the graduation stage possess the program outcomes."criterion is addressed.Several different methods can be used to designate how much the program outcomes are fulfilled, or in other words, how much the graduate engineers possess the skills that the department they are graduated from aims to bring in.One of these methods is the determination of how sufficient the graduates or students see themselves in terms of these outcomes.In another saying we can express it as scaling of the selfefficacy of the graduates or students about the skills that they should have earned.Self-efficacy is defined to be the belief about achievement of a specific work.It is expressed that if this belief is attempted to the behavior about the work or not effects the continuity of this behavior, motivation about the behavior and consequently the performance [11].It is claimed that the strong belief about the capabilities of the person increases the endeavor and continuity [12].
When literature is scanned, self-efficacy perception scales that are for the scaling of the sufficiency in the fundamental engineering area of the graduates or engineering candidates that are continuing their education in different statuses are not encountered.In this study, development of a scale that evaluates the self-efficacy of Electrical-Electronics Engineering students in terms of basic electronics skills is aimed.

A. Sample
Continuing 124 students of 2nd 3rd and 4th year in Amasya University, Faculty of Technology, Electrical-Electronics Engineering students constitute the study group for this research.The distribution of the study group according to the sex and grade is summarized in Table 1:

B. Development process of the scale
In the development process of the scale, firstly, overall effectiveness and curriculums of some of the faculty of technology Electrical-Electronics Engineering departments in Turkey are examined [13][14][15][16].Along the frame of this examination of curriculum, the basic electronics skills that the Electrical-Electronics students are expected to hold before branching, in general, are revealed.Then, these skills revealed are reviewed by 4 experts of their areas, 2 associate professors and 2 assistant professors in Electrical-Electronics Engineering, and are given the final form.The skills obtained in this way, lastly, are transformed into self-efficacy expressions.At the end of this process, an item pool of 27 items including beginner, intermediate and advanced level skills.Across the items that are constituted, five degree choices to specify the student's level of skills are placed.These choices are arranged and scored as "(1) Totally insufficient", "(2) Not adequate", (3) Intermediate", "(4) Adequate" and " (5) Totally sufficient".The scale given the final form is applied to the study group.Help of a lecturer for each class that are included in the scope of the study group is desired and applications are done under their supervision.The data collected are uploaded to the programs SPSS 15.00 and AMOS 16 in order to be analyzed statistically in terms of validity and reliability.

C. Data analysis
In order to determine the structure validity of the scale, and to size and determine the factor load of the items on the scale; principal components analysis is applied.While deciding the convenience of the data to the factor analysis, Kaiser-Meyer-Olkin (KMO) coefficient and the results of Bartlett test are used.In order for a datum to form factor structure, KMO coefficient should have a value bigger than 0.6 and Bartlett s test should give meaningful results [17].For the meanings of the factors to be interpreted better, Varimax rotation is done [18].This rotation is one of the rotations that show the differences between the factors the best and most preferred.In the detection of the number of factors, eigenvalue is applied as 1.00 [19].To immobilize how much the scale evaluates the similar attitudes of the items, items and the relation between the factor scores of the items (item-factor correlation) are found by calculations.Besides that, in order to immobilize how much each one of the items included in the scale scope effects the distinction of the students in terms of the level of attitudes; being looked at the scale scores, it is checked how meaningful is the distinction between the item scores of top and bottom 27% groups.The items with low factor load or a factor that disperses to multiple items or has a low item-total correlations coefficient are excluded from the scale.After this processes, a scale consisting of 19 items is prepared.After the exploratory factor analysis is done, confirmatory factor analysis is also done.The factor structure consisting of 19 items and 2 factors is verified.Korkmaz [20] expresses that, confirmatory factor analysis is a structural equation model that deals with the relations between the scaling techniques of hidden variables and observed measurements.The calculations applied in order to determine the reliability are internal consistency and test-retest calculations.This is applied as segmented test analysis, "one of the methods that are applied in the determination of internal consistency coefficient reliability" [21].While this study is being done, cronbach !internal consistency coefficient is calculated.Moreover, scale is redone on 33 students with 6 weeks break using test-retest method and the consistency level of the scale is researched calculating the relation between the two applications.

A. Findings regarding the validity of the scale
The structural validity along the frame of the validity of Basic Electronics Skills Self-Efficacy Scale (BESS), itemtotal correlations and item distinctiveness's are analyzed and the findings are presented below: Findings regarding the exploratory factor analysis: In order to test the validity of BESS, Kaiser-Meyer-Oklin (KMO) and Bartlett tests are applied on the data, KMO= 0,887; and Bartlett test value came out to be "2= 1983,964; sd=351 (p=0,000).Pursuant to these values, it is understood that factor analysis can be done on the scale with 27 items.In the first stage, principal components iJEP -Volume 6, Issue 4, 2016 PAPER A VALIDITY AND RELIABILITY STUDY OF THE BASIC ELECTRONICS SKILLS SELF-EFFICACY SCALE (BESS) analysis is done in order to determine if the scale is one dimensional or not.Later on, Varimax rotation method is used according to the principal components.Accordingly, it is found that items with item load less than 0.4 does not exist.As is, it is determined that the scale consists of 6 factors with eigenvalue bigger than 1 and the total variance is found to be 66.78.As is known, a scale is supposed to evaluate the behavior it is aimed to scale with the least number of items and the least number of factors.Accordingly, the declared variance is found sufficient to be 30% for the scales with one factor, and 40% for the scales with multiple factors [22,23].Therefore, the structure of the scale is decreased to two factors and the analyses are repeated.After this process, 3 items with factor load less than 0.40 and 5 items that are spread through multiple factors are removed from the scale.However, as removing it may damage to item content validity, one item is not removed from the scale although its factor load is 0.380.It is seen that the remaining 19 items are gathered under two factors.With the last form the KMO value of the scale with 19 items is determined as 0,871; it's Bartlett values are found to be "2=1308,963 p<0,001.Factor loads of the 19 un-rotated items remaining in the scale are found to be lying in the range of 0.380 and 0.678; in return to this, the rotated forms of these loads after varimax rotation method are seen to be in between 0.5696 and 0.800.After the operations done, the results about the item loads with factor eigenvalues and the rate of variance explanation amounts are presented in Table II.As seen in Table II, "F1: Intermediate Level" factor involves 10 items and factor loads change in between 0.788 and 0.569.The eigenvalue of this factor in the scale is 7.633; it's contribution to the general variance is 26.906%."F2: Basic Level" factor contains 9 items.The factor loads of the items lie in between 0.800 and 0.621.The eigenvalue of the factor in the scale is 2.384; it's contribution to the general variance is 25.814%.
Findings regarding confirmatory factor analysis: confirmatory factor analysis is applied to the scale which is found to consist of 3 factors in the result of exploratory factor analysis, in order to validate the factor structures.As the result of the confirmatory factor analysis that is done using likelihood technique with no restrictions, goodness of fit values are found to be "2/DF= 1.537, p<.001, RMSEA= 0.046, S-RMR= 0.00, GFI= 0.901, AGFI= 0.899 and IFI= 0.89.According to these values, goodness of fit values of "2/DF, RMSEA and S-RMR occurs to be perfect, the rest of the goodness of fit values come out to show acceptable alignment.With another saying, this model reveals that the factors are confirmed by the data.The factorial model of the scale and the t value related to factor-item relation is given in the Fig. 1.Item factor total and corrected correlations: In this part, calculating the correlation between the scores obtained from each item in the factors with item-total correlation method and corrected item correlation method and the scores obtained from the factors; levels of serving the purpose for each item are tested.The item-factor correlations obtained for each item is given in Table III.
As seen in Table III, item test correlation coefficients are between 0.602 and 0.815 for the first factor; between 0.680 and 0.786 for the second factor.Each item is in a meaningful and positive relationship with the factor (p<0,001).Hereunder, it is possible to say that each item serves to the purpose of the factor.
Item discrimination: The discrimination powers of each item in the scale are calculated.Through this aim, firstly the raw scores obtained from each item are binned from the bigger to smaller, then top and bottom 27% groups consisting of top and bottom sub-groups with 34 students each, are designated.The t-test values for the independent groups are calculated through the total points in the groups.t values related to the distinctiveness powers and the findings related to the meaningfulness level are presented in Table IV.
In Table IV, it is seen that the independent sample t test values related to 19 item, factors and total score in the scale change between 6.361 and 9.656.For the run-of scale, t value is found to be 318.745.The levels of each difference identified are meaningful (p<0,001).Accordingly, it can be stated that the scale and the items in the scale, all have high distinctiveness levels.

B. Findings regarding the reliability of the scale
In order to calculate the reliability of the scale, internal consistency and stability analyses are applied on the data.The operations done and findings are presented below: Internal consistency level: The reliability analysis of the scale as a whole and regarding the factors is done using Cronbach Alpha reliability coefficient, two congruent halves correlation value, Sperman-Brown formula and Guttmann split-half reliability formula.The resulting reliability analysis values about the scale as a whole and about each factor are summarized in Table V.
As seen in Table V; the two congruent halves correlation of the scale is 0.880; Sperman Brown reliability coefficient is 0.878; Guttmann Split-Half value is 0.753; Cronbach's Alpha reliability coefficient is found to be 0.9158.On the other hand, the congruent halves correlations regarding the factors are 0.863 and 0.846; Sperman Brown values are 0.907 and 0.765; Guttmann Split-Half values are 0.902 and 0.861; Cronbach's Alpha values are seen as 0.8989 and 0.883.Pursuant thereto, both factor and the scale in general are capable of making consistent evaluations.
Stability level: The stability level of the scale is determined using test-retest method.The last form of the scale with 19 items is re-applied to the 33 students who attended the first application, 6 weeks later.The scores obtained these two applications are analyzed regarding both the scale and the factors separately.Thus, the ability of making stable measurements of the scale, the items in the scale, the factors are tested and the findings are summarized in Table VI.
The correlation coefficient, that are obtained by testretest method, of each item scaled in Table VI are seen to be changing between 0.391 and 0.719 and each relation are seen to be meaningful and positive.The correlation coefficients of the factors that are obtained by the testretest method are lying in between 0.614 and 0.609; the correlation related to the total score is 0.712; and each relation are seen to be meaningful and positive.Therefore, it can be said that the scale can make stable measurements.

IV. DISCUSSION
In this study, in the aim of measuring the basic electronics skills of Electrical-Electronics engineering students through their self-effacement, a scale is developed.The scale is developed taking the Classical Test Theory is selected as the baseline.In the Classical Test Theory, the results obtained from the group that the test is applied on are assessed as a whole.The power and distinctiveness indices of the items can be estimated after all the data of the group is obtained.While an assessment is being done, the points are studied as a whole, not seperately for each item [24].In the Classical Test Theory, reliability can be attained as a result of assessment of the result of the test applied to the whole group.Differently from this, in the Item Response Theory, the aim is to estimate the characteristics of the students through the items.The important part of the theory is the correct scaling of each of the items in the trial stage.According to Hambelton, Swaminathan and Rogers [25]; some limitedness such as personal test development, identification of item biasness, weighting the distractors and equation of tests to each other can be resolved with the help of Item Response Theory.
It can be said that in the Item Test Theory one of the very fundamental movement points is personalized test preparation and application [26].However, this model has some limitedness.For example, with this approach the reliability may come out to be very low and meaningless in the item and skill estimations.In this theorem, the estimations done to the models chosen.With the increasing number of the parameters in the model, the difficulties in estimations and calculations occur.Another disadvantage is the single dimensionality.Similar to this study, traditionally in the identification of the dimension, factor analysis techniques are being imposed.However, Item Response Test approaches to the scale directly as only one-dimensional.Although the Item Response Theorem is a powerful model to explain the covered characteristic that lies under the performance of the person, usage of a one-dimensional model in the multiple dimensional test data for which the one-dimensionality assumption is not fulfilled will result with validity problems in the estimations of skills and items, and will rise important problems in the model-data harmony [27].On the other hand, the applications predicated on this Item Response Theorem are very difficult as the applications differ between the students [26].Since, the scale that is developed in addition to these scales is multi-dimensional and does not carry an aim of defining the personal skills; after all it aims to gather general information about the group, Classical Test Theorem is preferred instead of Item Response Theorem.
This scale is a BESS five degreed likert type scale and it consists of 19 items that can be gathered under 2 factors.Factors are named regarding to the general characteristics of the items under them.Along this frame, when the skill sets are analyzed, the baseline level skills stand under Factor 2 and intermediate level skills stands under Factor 1. Baseline level skills involve the fundamental courses that the students take in the 1st year and the fundamental applications they performed in the 2nd year; intermediate level skills involve the more complicated skills that the students acquire in the 3rd year of their education.However, as the advanced level skills are special skills that are acquired by branching in the 4th year, they are not included in this scale.Since the students take courses directed to the branches such as signals, antenna, image processing, medical electronics and coding, advanced level skills cannot be defined in a standard manner.
The validity of the scale is analyzed in two different methods via factor analysis and distinctiveness characteristics.To evaluate the how much each item that exists in the scale can measure the factor they belong to, item total factor correlations and corrected correlations are calculated on the data.According to the acquired values, each item and each factor in the scale are found to serve to the run-of the scale and the aim of scaling the desired facility, in a meaningful level.In addition to this, analyzing the t value related to the differences between the groups of top 27% and bottom 27%, the item distinctiveness's are researched and it is detected that the distinctiveness of both of the run-of the scale and each one of the items is high level; in other words, it is detected that each item is distinctive in the desired level.The internal consistency coefficients of the scales calculated using two congruent halves correlations, Cronbach Alpha, Sperman-Brown formula and Guttmann split-half reliability formula.Regarding these values calculated, it is determined that the scale is able to make reliable evaluations.In order to define the level of stableness with time for the scale, testretest method is applied to the data obtained from the applications done with a 6-weeks space.Test-retest method is applied on both the factors of the scale and to each of the items and it is found that each of the items and factors are able to make stable scaling in terms of stableness with time.

V. CONCLUSION
Consequently, it can be said that BESS is a valid and reliable scale that can be used in the scaling of basic electronics skills of Electrical-Electronics students through their self-efficacies.In the literature, a reliable and valid measurement instrument that aims to scale these skills of the students is not encountered.Therefore, this scale can be thought of making huge contribution to the literature.However the validity and reliability study of the scale is restricted in 124 Electrical-Electronics Engineering students.For this scale to be used in different education stages, repeating the validity and reliability studies can be suggested.

Figure 1 .
Figure 1.Correlation diagram of the scale (t values)

TABLE II .
FACTOR ANALYSIS RESULTS OF THE SCALE AS PER FACTORS