Effectiveness of the Partial Implementation of a Cloud-Based Knowledge Management System

—Nowadays cloud computing technologies are cost-effective services enabling to generate the learning quality. The goal of this research is to define the borderline of the effectiveness and limitation of the partial implementation of Knowledge Management System based on cloud computing technologies. In view of this, the research in the form of knowledge testing as well as objective and subjective assessment of the learning quality within a wide sampling of 396 students in two independent reference groups was conducted. One of the groups has used traditional methods of the training conducted in classrooms by applying e-learning, and the other one has used the Knowledge Management System based on cloud services under the most budgetary option. As a result, a range of certain differences in the quality of training of two groups was found out and the latter must be used for a further study. According to the results of all tests related to the quality of training, in the group, that has used Knowledge Management Systems and cloud computing technologies, the students have demonstrated results above average in various tests than in the group where these technologies have not been used. The results allow defining specific advanced features of Knowledge Management Systems with the application of cloud computing technologies in the education.


Introduction
The rapid development of information technologies has changed the educational environment faster than expected [1].The education is to satisfy needs of the society and business for specialists who are ready to study throughout their lives, always, anywhere and in any conditions [2].The total penetration of social networks into the society has affected the change in habits in the areas of communication, training, access to knowledge and perception of knowledge [3].These changes often negatively influence learning abilities and striving to acquire knowledge.The development of information transfer and storage technologies, mobile technologies has led to the possibility of education, less dependent on face-to-face communication with the teacher [4].
Knowledge Management Systems, which are obligatory grounded on technology infrastructure, are in their context an effective technology phenomenon for current need in permanent accumulation, processing and transfer of new knowledge.The education system as well as small and medium-sized businesses and the public administration have tackled with the challenges referred to the cost of technical solutions for creating such infrastructure.Their solution also refers to the application of cloud technologies.The process of launching technical solutions takes much time during which the educational process and the process of generating knowledge have been continuing to develop.Therefore, there is an urgent need to determine the borderlines of effectiveness and the opportunity to apply partial, modular solutions of Knowledge Management Systems that would be based on the minimum, most widely available architecture.
Cloud computing technologies can be defined as a set of virtualization services, providing modularity, technical and software independence of the end user from the system resources that he/she uses [5,6].The use of these services enables the dramatic reduction in costs on maintenance of the technical infrastructure (servers, data transmission lines, uninterruptible power supply devices), the purchase and maintenance of the software, the costs on supporting of specialists in computing systems' maintenance, the costs on solving issues and fixing violations in the engineering and software infrastructure [7].
The need for the use of cloud services in education is due to the presence of three basic issues of higher education: 1. Limited funds within constantly increasing requirements for the quality of training and the volume of knowledge transferred 2. The formation of knowledge based on continuous training and the skill of selfimprovement 3. Social changes in the practice of communication and information processing [8] The common scheme of the training in classrooms was disrupted by the emergence of a large number of electronic devices that have greatly simplified the access to the information.This process caused the appearance of two significant phenomena: elearning and distance learning.Further, the process of decreasing the quantity of training sessions conducted in a specific place and at the definite time was becoming more viable.The advent of the Internet and generally available mobile devices led to the development of Knowledge Management Systems model [9].Learning media, which originally looked like tape recorders, special modules or TVs are currently being developed by using Internet media and smart devices such as computers, laptops or smartphones, which e-learning is often associated with.In fact, the e-learning is changing towards a digital format from a conventional learning format.One uses Internet technologies and smart devices, so that e-learning is possible anywhere without space and time limits [10].The next logical step was the presentation of adaptive Knowledge Management Systems in the e-learning [11].E-learning systems are much more subject towards individualization and creation of the infrastructure that automatically adapts to the learning style specific for a particular person [12].
In the process of the implementation of Knowledge Management Systems based on cloud computing technologies many models of their architecture have been developed and proposed [13,14].All of them have universal features grounded on the homogeneity of the fundamental architecture of cloud computing, although there are some differences in certain models.Some researchers have estimated the impact of cloud-based services technologies and the use of Knowledge Management Systems among students from higher educational institutions and numerous options of the efficient launch of cloud-based technology programs by universities in many countries [8,15,16].All these studies describe the final state of some ideal system that must be implemented by a higher educational institution, or they describe the process of solving intermediate cases that have arisen during the implementation of such system.However, the studies do not describe the effectiveness of Knowledge Management Systems at each individual stage and the growth of their effectiveness or lack thereof.The studies of the intermediate effectiveness of the implementation process are necessary as the latter can be carried out for years.
Therefore, some authors highlight that issues related to the implementation of Knowledge Management Systems and the application of cloud computing technologies for them are similar for education system and small/medium business [17][18][19].Public authorities tackle with the same in their essence issues [20].In the work of Sultan [21] the issues linked to the implementation of cloud computing technologies for knowledge management and e-learning are fully the same for higher educational institution and small/medium business.Solving issues as for the minimum needed effectiveness of a functioning system in one area can be applied to the other one.
Due to the fact that in practice many educational institutions have issues with financing of long-term development projects, the authors understand that the full knowledge management structure on a developed technical base is not instantly implemented.Thus, most educational institutions function in a way of implementing only part of the modules of such system, and there is a question: "How effective is the application of such an "unfinished" system?" The goal of the research is to verify the effectiveness of the use of the Knowledge Management System in the educational environment of the higher educational institution, limited by minimum opportunities and based on publicly available free cloud-based services.As a rule, this is exactly what the first step of implementing such systems means.
The hypothesis of the research is the assumption that even the minimal structure of such system based on cloud computing technologies can lead to a relevant increase in the quality of training, which will be noticeable and can be fixed by using tests' system, which is applied in the already existing pedagogical system and existing practice of knowledge testing.

Materials and Methods
To conduct the research, a group of 396 students from the Russian State Vocational Pedagogical University has been chosen, who study the same subject, in two equal reference groups of 198 students each.Some studied subjects during knowledge processing cycle close to the regular training in a classroom, the second groupin an adaptive knowledge management environment based on cloud services.Both groups worked with the same group of teachers and under the same training program.Group 1 used a simpler basic approach to e-learningthe usual classroom training with personal presence, but with access to teaching materials and working with the teacher using electronic devices (smartphone, laptop, remote computers, computers in classrooms).Group 2 worked in the environment with more complete integration with cloud tools (and, accordingly, with a higher level of the use of an adaptive Knowledge Management System).
The research was conducted for 2 months in the middle of the semester to eliminate psychological effects from training students for exams at the end of the semester or gradually starting and re-adapting to the training at the beginning of the semester after returning from holidays.The authors studied how the training material was conceived as a whole as well as the interest to learning and the ability to independently process educational material.The most important factor was the ability of students to strive independently for better mastering of knowledge, their accumulation, structuring and application, increasing their motivation for learning.
In particular, mixed groups of 1-3 year students of the same specialty took part in the testing without differentiation in academic performance over previous periods of training.The Group 1 and Group 2 were trained with the same program and under the participation of the same teachers.For testing of the knowledge one separate discipline was used, not a group of disciplines, in order to exclude psychological factors of students' attitudes towards different disciplines and different qualification of teachers.
The testing is divided into two significant parts.The first part is subject to determine the quality of assimilation of previous knowledge of each of the groups of students.Three testing methods are involved that are constantly used in university practice and are well known to students: 1. Knowledge testing with a limited number of answer options (out of which the one is right) for each of the questions.In the test, 20 questions were used on the material mastered during the research.This test supposes the check of keeping in memory and assimilation of a clearly defined set of information.2. Testing the depth of acquiring a recommended content (educational and scientific literature, research articles, educational videos and audio recordings in the Internet).
It was carried out by the method of formulating the answer to key questions within mastered educational content.The test has included 30 questions, to which participants had to give a detailed written answer.The purpose of the test is to check the quality of students' independent work outside the audience, the level of understanding and assimilation of the educational content.
3. Testing the acquisition level of mastered knowledge.It was conducted in the form of an interview with 5 independent teachers who were previously not familiar with these groups of students and are experts in the discipline under study.Each teacher has assessed each student upon a 10-point scale, the average value was derived using the arithmetic value of five ratings.During the interview, the ability of students to formulate adequate, reasoned judgments in the field of knowledge acquired during the test's performance was checked.The testing time did not exceed 10 minutes to exclude stress factors and emotional fatigue of the students, and taking into account the relatively small amount of material that was supposed to be acquired during the research time.
The second part of the testing aims at defining the objective and subjective involvement of students in the learning process, their personal attitude to this process.For a relatively objective assessment of involvement, the authors used the research of the quality of students' work on the written assignments that they received in the learning process.The results of the research split the students of both groups based on: ─ Who completed all the tasks during the research period on time and adequately ─ Those who made mistakes, which were then corrected after the comments of the teacher ─ Those who completed part of the tasks and did not complete some of the tasks, whose works were commented and their re-examination was postponed, and the time for re-passing the task was far from the time of research execution ─ Those who did not complete at least some part of the tasks As the objective of this research has no reference to pedagogy, the study of the work quality of students or teachers, the level of their performance or reasons of different characteristics of acquired material were not important for the authors.The research is aimed at demonstrating the quality of students' work, their timeliness and effectiveness with or without using technical capacities of a cloud-based Knowledge Management System.
A subjective assessment of involvement in training was carried out by the method of a written survey, in which respondents from both groups were asked to rate three factors on a 3-point scale.Researchers tried to detect a biased assessment, in which the rational component would be minimized.Therefore, a three-step scale was proposed with the simplest evaluation approach: "very good", "good enough", "bad".
1.The "usefulness" of the teaching method that students used.It has been explained to respondents that "usefulness" means how they evaluate the level of the effectiveness of the applied training method for their future life in general, study and professional growth.2. Subjective "quality of knowledge acquisition".In this case, students had to give an assessment of how well they acquired the training material during the study.This element of the research, in comparison with objective estimates of performance, also allows evaluating the level of adequacy of students' self-esteem.
iJET -Vol.15, No. 13, 2020 3. "The satisfaction with the learning process".The respondents were asked to indicate whether they were interested in the learning, whether they wanted to pass the training and complete training tasks.However, it was pointed out that there was nothing wrong with a negative assessment according to this criterion.Often there is a situation when the student studies well, but he/she makes considerable efforts and is not happy with the training, overcomes himself/herself, acts without enthusiasm or pleasure.In this case, a subjective satisfaction during the educational process was precisely estimated.
The following technologies were used: ─ Google Drive cloud-based services group as a service for storage, exchanging and ranking of educational information, and also of independent works and testing tasks performed by students ─ Google Docs as a service for accessing applications without any reference to a device or a specific place ─ specialized groups in social networks Facebook and VKontakte for separating educational materials and comments, distribution of links to the content necessary for the training ─ mobile and platform versions of Facebook Messenger, Viber, WhatsApp applications as quick exchange services with messages and files No special program works were carried out to create more deeply integrated systems of interaction between students and the teacher or students among themselves.Key characteristics of the work of the Group 1 were: ─ Use of regular training sessions in classrooms with face-to-face contact with the teacher, execution of tasks and participation in seminars with the involvement of the teacher ─ Individual study with online materials, which are placed in network attached storage by teachers and in accordance with their recommendations ─ Opportunity to use mobile hardware devices during workshops and carrying out lessons with the involvement of the teacher Conceptual differences were in the Group 2 that used "cloud computing technologies": ─ Opportunity to execute a task directly in the network, using a laptop or a smartphone ─ Opportunity to familiarize instantly with completed works on the similar item of other students ─ Opportunity of written request to the teacher 24/7 with minimum term of providing feedback ─ Opportunity of general document's amendment ─ Opportunity to apply training materials found independently upon the consent of the teacher (the agreement passed very quickly) ─ Opportunity to exchange opinions, ideas and questions in a single communication environment at any time ─ Opportunity of instant amendment during working process ─ Presence of "alarm clocks"regular reminder about the time of the execution of written works and preparation to seminars The authors considered the presence of the specific protocol with data handling of all types, which was brought into notice to all participants of the Group 2 and which was in a successive order maintained for the period of the entire research.The specific protocol is critical for effective functioning of cloud-based technologies in education.
The protocol contained rules for placing, indexing and access to all types of files created in the general location of files, as well as rules for communication between all participants.For example, the rules for response time to different types of messages, comments or requests for help.Before the start of the research, a consensus was reached between all participants in the Group 2 and teachers and the approved protocol met all their requirements and expectations.Maintenance of the protocol was the subjective need and responsibility of all participants, and was carried out by each of them personally.At the deeper software integration of cloud computing services within an adaptive Knowledge Management System, the implementation of a data exchange protocol seems to be performed by integrated software through special mobile applications that can be downloaded by training participants to their mobile and other electronic devices.
The obvious restrictions of the research were the quantitatively limited group of those who were tested, the study of one discipline but not a group of disciplines.Such restrictions were imposed in order to exclude the imposition of any other random factors, such as the level of complexity of the disciplines for perception, the quality of work of certain teachers and the quality of the applied training programs.However, the completeness of the research requires verifying its results for a group of disciplines that intersect in the learning process.
The implementation of control and use of network resources, particularly mobile systems in relation to ensuring the communication quality and monitoring of tasks' execution is especially difficult.The control of the learning process outside a classroom is the most difficult, especially when it comes to free selection of the training material.The students can fulfill individual tasks for each other, perform them together, but master the material in full.On the one hand, it will contribute to the increase of the quality of material's acquisition for the group as a whole, but, on the other hand, it also hampers the quality of material's acquisition for each participant of the group from individual development point of view.Thus, ranking methods for access to information and methods for defining the identity of the owner of the information and its creator are incorporated into the Knowledge Management System.
The one of the most important identified restrictions that has been pointed out is a higher workload of the teacher when using the Knowledge Management System.With a further increase in the workload the use of specific software is becoming in a demand, including on mobile devices and automation of many teacher's functions, in particular, automation of verification of completed tasks in many cases.There is a requirement for a better infrastructure, which would allow the teacher to easily create personalityoriented tasks that suppose the identification and study of the individual style of iJET -Vol.15, No. 13, 2020 students' learning.Formulating a personal learning style is the most difficult part of the Knowledge Management System, which is harder to implement and can only be implemented with the help of Big Data technologies [22].

Results and Discussion
The analysis of statistical studies showed that within all types of testing results of knowledge acquisition the Group 2 demonstrated the best performance than the Group 1.In addition, in the Group 2, self-estimation of the learning quality and the satisfaction level with the learning process were much higher than in the Group 1.

Objective results of learning assessment
Figure 1 shows that both groups have no unsatisfactory results; there is as well the minimum number (14.65% in the Group 1 and 7.07% in the Group 2 of the total number of participants) of those with the lowest results.In the Group 2, there are almost two times less such participants than in the Group 1.The best test results in Group 1 were demonstrated by 32.83% of participants, while in the Group 2 -43.43% of participants.

Fig. 1. Test's results on facts' knowledge
Results of the interview with teachers (Fig. 2) showed that in the Group 1 the lowest mark was 3 on a 10-point scale, while in the Group 2 -4.Pursuant to the results of the test, 86.87% of students received the highest grades (from 6 to 10 scores) in the Group 1, and 95.96% of students in the Group 2. At the same time, 37.88% of students received three highest grades (from 7 to 10 scores) in the Group 1, and 62.63% of students in the Group 2, which is almost twice as much.

Fig. 2. Results of the interview with teachers
The test results for the knowledge of the recommended training content are given in Fig. 3. Participants of Group 2 gave a greater number of right answers (namely, 69.19% in a conditional group "25-30").While in the Group 1, there are much more those who have given right answers in two other conditional groups "20-30" and "15-20" -37.37% and 9.6%, respectively.

Fig. 3. Test's results on key answers for the acquisition of literature
The authors pay attention to the precision of obtained results in both groups.There were no fundamentally different results in relation to the learning quality that might be referred to the too short period for conducting the research.The authors highlight a iJET -Vol.15, No. 13, 2020 range of relevant trends, which over a longer period can ensure faster development of the learning quality in the Group 2.
There is the complete disappearance or sharp reduction in the number of the lowest grades in the Group 2, which affirms that students' audience is fully covered by teacher's control and the presence of a higher level of control over the learning process.In this case, there is practically no lagging behind in training those, who are unable or unwilling to cope with the learning tasks.Tighter communication and knowledge sharing in the group leads to the fact that even those who master a little of the material get enough of it to increase their knowledge and get a higher grade.
There is a steady trend, in all tests, for advancing the number of the highest grades in the Group 2. This may indicate that those who are interested in learning by using cloud-computing technologies are able to more quickly and more fully master the learning material, to achieve new additional knowledge.
In addition, in the Group 2, there is a tremendous change in the quality of independent application of the material under study and in the ability to make independent judgments based on it.In the authors' opinion this ability is a key one in training of a future professional who is capable to implement the education paradigm required by Industry 4.0 for the lifelong learning [23].
The objective assessment of control tasks results (Fig. 4) showed the expected prevailing number of students, whose works were corrected during the training process (49.50% in the Group 1 and 81.31% in the Group 2).The change in the quantity of participants who have succeeded in fulfilling all or the majority of their tasks slightly differs in both groups.However, in the Group 2, there were no uncompleted tasks and a number of delayed tasks was also significantly lower than in the Group 1 (39.39% for the Group 1 and 14.14% for the Group 2).

Subjective assessment of the learning process and results
The Group 2 students assessed the learning process as the most useful (49.50% in the Group 2 compared to 14.65% in the Group 1) (Fig. 5).There were no crucial inconsistencies in the estimation of the usefulness as a whole.Group 1 participants (19.19%) and only 11.11% of the Group 2 participants defined the learning process as "useless".The grade "useful enough" as an average one prevailed as expected.Most students of the Group 2 (67.68% against 34.34% in the Group 1) subjectively assessed the quality of acquired knowledge (Fig. 6) as "high".Considering subjective estimates, the average mark "qualitative enough" in both groups prevailed, but such average grade was given mostly by the Group 1 (44.95%), while in the Group 2 this grade was given by rather less number of students (24.24%)at the expense of prevailing grade "high quality".The assessment of the criterion "satisfaction with learning" is especially significant for understanding the impact of Knowledge Management Systems and cloud-based services on the students' motivation (Fig. 7).The data obtained herein are unambiguous.In the Group 2, which used such technologies, there was no grade "not nice", and the highest mark "very nice" was given by 78.79% of the students.In the Group 1, there are the following expected estimates, namely 23.23% "very nice" and 29.29% "not nice".A result of the grade "pleasant enough" for the Group 1 is 47.48%, as expected almost two times bigger compared to Group 2. The objective assessment of the interest to learning in the Group 2 unambiguously indicates the greater efficiency of using a cloud-based Knowledge Management System.The authors observe that there are no uncompleted tasks in the Group 2 concurrently with a sharp increase in the number of completed tasks.In fact, the results of the Group 2 on this test can be interpreted as full involvement in training; the difference in the performance of works is dictated by the abilities and personal circumstances of the students.The subjective assessment also shows the increase in the adequacy of self-esteem of students from the Group 2: students have assessed the quality of their knowledge acquisition as average, which almost exactly coincides with objective test results in the first part of the research.In the Group 1 such a phenomenon is not observed.
The second important factor is very high rates of satisfaction with learning in the Group 2. In the authors' opinion, the divergence of the subjective assessment of training usefulness in two groups cannot be interpreted unambiguously.In the Group 2, there is an obvious shift in the usefulness estimation in a positive direction, but in the Group 1, much smaller number of students have rated learning as "very useful".It is possible that the feeling of usefulness of knowledge and lectures increases with the growth in communication in the group and the collective work on the task, however, this aspect of the research requires a more thorough follow-up study.
The authors did not manage to find out studies dedicated to the issue of the work.In most cases researchers analyze the specific implementation of the cases related to the use of cloud technologies in these or those higher educational institutions [24], means of solving issues, which have arisen during the process of such adaptation [24].In addition, most researchers propose various forms of studying the peculiarities of cloudbased architecture [25][26][27] and improvement of the structure of Knowledge Management Systems [6,10,28,29]; including for certain knowledge areas [30,31] or successful aspects of its implementation in universities of some countries [28,32,33], as well as networks' management of cloud-based services.Arpachi [34] analyzed the increase of interest to knowledge systems upon the use of cloud computing technologies and the growth of interest to the education herewith among students.
In the authors' opinion, the studies related to the impact of cloud computing technologies on the psychological state of students and the relationship of the psychological state with the learning results represent a special interest.Yu et al. [35] analyzed the possibility of using monitoring systems of the psychological state of students, integrated into the Knowledge Management System of the university, for early detection of depression cases, which are growing every year.Kurelović et al. [28] have highlighted the importance of cloud computing technologies as regards the realization of students' expectations and needs.Gunadham and Kuacharoen [36] analyzed the issues of ensuring access security, information protection of Knowledge Management Systems based on cloud computing services and raised up the issues of ranking access to the information in such systems.Almost all the researchers pointed out the issues of information protection and access security that are among the main vulnerabilities and risks of cloud computing technologies.
Sadler-Smith and Smith [37] described a strategy for personalizing a cloud-based knowledge management process.Due to their research, the authors can make reasonable conclusion that the creation of a flexible individual style of learning for each student is the next expected step in the development of educational technologies.The use of Big data technology and specialized mobile software integrated into the Knowledge Management System to collect data on the learning style that is specific for each student are mandatory for the implementation of this step.The current research fully confirms this conclusion.The authors highlight that there are no such works aimed at the full implementation of the system in higher educational institutions, which allows regarding the current work as very relevant.
Thus, there are practically no studies on the quality of work of the above-mentioned services and on the assessment of the impact of various components of such systems on business processes and on learning processes.It is not always possible to immediately implement the entire Knowledge Management System, besides, the complexities of its implementation and aspects for improving its functioning are generally revealed only during long-term maintenance.It remains to analyze the features of such systems that can be applied with minimum damage to its effective functioning.This issue is significant for all countries and is equally important both for higher education and for small and medium-sized businesses around the world, because these areas face similar problems [38,39].Hence, the current research is exactly aimed at identifying the features of the Knowledge Management System, which is also specific to most of small and medium-sized enterprises.There are common features between higher educational institutions and small/medium-sized business [5].

Conclusion
Cloud-based Knowledge Management Systems ensure significant benefits for the organization of the learning process as well as for its effectiveness.The authors consider the hypothesis, which states that even minimum implementation of a more adaptive Knowledge Management System based on cloud services provides relevant positive changes in the quality of education.In fact, the implementation of such systems at the early implementation stages requires no financial costs but only operational ones.The early implementation stage of the Knowledge Management System needs a mandatory in-depth analysis of accessible data exchange protocol and their application between end-users.All other elements of the system and services are available for a free use and on the sufficient quality level.In this case of relevance is the application of standard, open-sourced, free services Google Docs, Google Disk, Facebook, messengers Viber, WhatsApp, Facebook Messenger.The latter are available on computers, smartphones and laptops within 3G connection or Internet access via GPRS protocol rendered by mobile providers.The global trend of the involvement of people in regular use of mobile applications can be applied for improving the learning quality and increasing the interest to the learning process.In addition, the use of cloud computing technologies becomes relevant for implementing the principle of lifelong learning.

Fig. 4 .
Fig. 4. Objective assessment of the performance of control tasks

Fig. 6 .
Fig. 6.Subjective assessment of the criterion "quality of knowledge acquisition"