Rule-based Interactive Learning Application Model on How to Play Music Instruments

This research aims to develop an interactive learning application on how to play instruments of orchestra music. The object of this study was Gamelan, a well-known traditional music orchestra from Java, Indonesia. This study proposed a model of gamelan orchestra learning application using a rulebased method to support independent learning process. The model supports a virtual instructor that can assist the application user in learning to play gamelan instruments, and can also function as music accompaniment. To evaluate the success of the learning model proposed, the researcher conducted an experiment which involved control group and treatment group. The results of the study showed that the average score of the treatment group members’ skill supported by gamelan learning application was higher than the skill of control group members. Keywords—Interactive learning application, rule-based, gamelan.


Introduction
Computer music is resulted from continuous research and experiments [1] involving computer which plays the composition, performance, or sonic realization, and also gives an important broad role for computer through application related to music [2]. One of computer research topics is virtual music instrument research. This topic has been widely developing and growing in several decades. A virtual music instrument which runs on computer or smart phone platforms should be able to attemptively duplicate the design of its conventional music instrument. This is significant for nonmusician users as they are lack of conceptual and technical musical skills [3]. Moreover, a computer music approach is used to assist adults in learning to play musical instrument, where the adults have some constraints in assimilating new knowledge and less time for doing such activities that do not make visible profit; hence, an imitation of music instrument was designed as similar as possible to the real one [4]. The development of virtual music instruments results in an innovation system which supports music play in a group of difference or similar instruments, such as a virtual music instrument system as proposed in this study [5] that supports multiple users to play virtual percussive music in one input device. Computer music computation can transform conventional music instruments into computer media which gives an advantage in accessing, learning, or playing musical instruments. Smule by Ge Wang is a popular example of virtual music instrument application consisting of such various types of music instruments as piano, guitar and so on. Smule can transform iPhone into creative channel which allows its user to learn and play musical instruments, and even to perform music [6][7][8]. In addition, the development of computer technology makes computer music cheaper. In contrast, traditional music instruments become much more expensive. Accordingly, this makes most of new music instruments sales are now in the form of digital music instruments [9]. Besides, this makes virtual music instruments more popular.
This research aims to develop an interactive visual music instrument application to learn how to play gamelan music instruments. Gamelan is a traditional music orchestra from Java, Indonesia. It is always challenging to learn or play musical instruments that typically played in all types of music orchestra performance. A music orchestra needs an expensive cost and a large room to perform, and it takes years to master such musical instruments. Furthermore, the presence of other players is importantly significant since gamelan cannot be individually played in a performance, and it is difficult to learn playing gamelan without any accompaniment from other players. There might be a difficult situation when some players are late or unable to attend for playing or learning session. Thus, such problems in learning gamelan are interesting to be solved. A computer music approach was chosen to solve the problems in learning to play gamelan instruments. This was conducted by developing a virtual gamelan music instrument system. Moreover, rule-based method, a part of knowledge-based system approach, was used to develop a model of gamelan orchestra learning application.

Research Method
The implementation of a computer music approach to gamelan has been developed in several works of mobile applications, such as e-Gamelan, Gamelan Toetoel, and Smart Virtual Gamelan Orchestra. The application of Smart Virtual Gamelan Orchestra (SVGO) developed by [10] offers more innovative technology than other virtual gamelan applications. The application supports notes sequences database and automatic play feature of the instruments. The feature plays a selected notes-sequence, and transforms it into audio, visual, and animation forms. Then, SVGO was not designed to support learning to play gamelan instruments, despite its innovation. There were virtual players in SVGO. However, there were not any virtual instructors which can support the learning process to play the instruments.
In this research, a model of gamelan orchestra learning application is proposed to support individual in learning to play gamelan music instruments. The model consists of visual, audio, and animation data of the instruments, notes-sequences data, and rules of how to play the instruments. The data as well as the rules were formulated to set visualization, interaction, and evaluation in learning to play gamelan instruments. Furthermore, the model was implemented by designing a gamelan learning system that can run in mobile application platform. A mobile application supports learning without space and times restriction, and this platform can improve the output quality of the users [11]. Thus, varied users' skills should be considered in developing a musical instrument interface. A simple interface may reduce the expressivity of the musical depth [12]. Hence, in order to adapt to the varieties of the users' skills, gamelan learning system was designed to simultaneously follow and adjust the learning progress of a user during his play.  Figure 1 shows a diagram of gamelan orchestra learning application model. The tasks of virtual instructor are to guide a user in learning how to play gamelan instruments. In addition, the virtual instructor was constructed based on gamelan theory, and based on observation on how a musician plays a gamelan instrument. Meanwhile, gamelan theory was used to define rules of how to play gamelan instruments based on the tempo of the music play. Observation part was used to define rules of approximate time, duration to hit a button of the instruments based on the tempo where the hit can be tolerated, not too fast or not too late to play [13]. The virtual instruments were designed using photos and recorded sounds of the instruments. The virtual instructor and virtual instruments were visualized by animation support. During the play, the interaction of a user in hitting instrument's buttons to play a note sequence was simultaneously sent to the system to be evaluated. The evaluation gives a feedback of an output quality of a user's play.
The method used in this study was conducted in four phases. The first phase was knowledge acquisition to achieve necessary knowledge of gamelan music to develop gamelan orchestra learning application. Secondly, this phase involved knowledge representation to produce rules of learning on how to play gamelan instruments. The third phase was system development to implement the proposed model and the method to develop a gamelan orchestra learning application. The last was evaluation phase to measure the success of this research using an experimental group method.

Knowledge Acquisition
Knowledge acquisition was conducted to understand the rules and methods to play gamelan instruments. This phase was conducted by studying theory of gamelan music and interviewing gamelan practitioners. Brief examples of the result of knowledge acquisition were described in this section.
Karawitan art known as gamelan music consists of gamelan as music instruments and gendhing as the song. There are two types of musical scale used in gamelan, slendro and pelog. Musical scale of slendro contains notes 1, 2, 3, 5, 6; while pelog contains notes 1, 2, 3, 4, 5, 6, 7. Each note in these two musical scales has different audio signal frequency. Note 1 in slendro has different sound with note 1 in pelog, and so do the other notes. There is a dot note in both musical scales which represents silent beat. This dot note was transformed into number 0 for it can be treated as number. Gamelan theory and the results of interviewing gamelan experts were explored to build the base knowledge. Moreover, the data of base knowledge base were described in this following Table 1. A group of gamelan music orchestra which uses both slendro and pelog musical scale contains up to 80 instruments and some singers called sindhen (human female singer in gamelan performance) [14]. A song in gamelan music can be in the form of melody with or without vocal [15][16]. A song contains a notes-sequence divided into gatra (bars), of which a gatra consists of four beats. Figure 2 below shows an example of a notes sequence in a gamelan song consisting of 16 gatra (bars).
Tempo is a variable in music which has two parameters. The first parameter is duration to play between two consecutive notes based on gamelan theory, and the second parameter is based on the estimation of the time a human takes to play it. The value of tempo based on gamelan theory was used to set the silent time between two consecutive notes, while the value based on the estimation of a human play was used to define the approximate time for the system or user hitting the buttons of the instruments.
Approximate time is duration for hitting buttons of the instruments based on the tempo in which the hit can be tolerated as not too fast or not too late to play. For example, based on the theory for playing a gamelan instrument, saron and demung, slow tempo takes one second for a silent time between two consecutive notes. Hence, if the previous note is played in second n, the target time of the following note to be played is in second n + one second. A computer system can be designed to accurately play the note as the tempo value, but not for human. Human plays the note based on his intuition. Thus, the play is based on an approximate time.
Approximate time to play gamelan music instruments has been studied by [13]. Moreover, the performance of a gamelan musician in playing an instrument was evaluated using time-frequency analysis. Time difference between hitting a button (execution time) to the target time of the tempo was used to define approximate time. First, Fast Fourier Transform (FFT) method was used to remove noise from audio signal recorded from the musician's play, and then peak detection technique was used to define approximate time by measuring time difference between the execution time to the target time. The result of the approximate time was duration between -328ms to the target time and +246ms from the target time. In addition, this study implemented the approximate time value proposed by [13].

Rules Production
The results of knowledge acquisition were used to define rules for automatic play, learning and evaluation features. Automatic play was designed for music accompaniment in orchestra performance and for user guidance in learning how to play gamelan

System Development
The system of learning how to play gamelan music instruments was designed by implementing the proposed model and method. The system was designed to be a virtual instructor and as music accompaniment as well. Thus, it should be able to automatically play gamelan instruments as an orchestra music group. Based on the knowledge implemented into the system, user interactions in hitting buttons of an instrument can be evaluated. Hence, the system can define the tempo and the visualization guide which are appropriate to the skill level of the user. The virtual gamelan music learning application was designed to run on mobile (cellphone) platform to support learning without time and space restriction.

Data collection
Most of gamelan instruments are percussion instruments, such as kenong, kempul, peking, saron, and demung. These five types of percussion instruments were used in the application. In slendro musical scale containing notes 1, 2, 3, 5, 6; some instruments have five buttons for those notes, and some have six buttons with an additional button after note 6 button for higher note 1. This is also found in pelog musical scale, where some instruments have seven buttons for note 1, 2, 3, 4, 5, 6, 7, and some have eight buttons with one button is for higher note 1. The following Figure 3 shows demung instrument in pelog musical scale. The dataset consisted of notes sequences collection, audio and visual of gamelan instruments, and animation to visualize user interaction. Notes sequences data were collected from www.gamelanbvg.com. Moreover, notes sequences data were transformed into text format with all dot notes were converted into number 0. The data was then embedded into the system. Furthermore, all notes 0 were converted back to the dot notes in the display.
Audio and visual data were collected from gamelan instruments set owned by Universitas Dian Nuswantoro, Indonesia. Each button of gamelan instruments was separately recorded, and then the data were embedded into the application. The audio data sounded according to the button hit by the user. For the visualization, all instruments were photographed, and then the pictures were manipulated using image editor, including animation of user interaction. Each key was manipulated for silent and hit moment. Figure 4 shows the illustration of an instrument animation for a user interaction visualization.

System design
There were two main parts in the system. The first part was to support learning with visualization guide, and the other part was to evaluate the user play. The system works by allowing the user to choose one of musical scales, slendro or pelog, and then to choose an instrument and a notes sequence to be played. The system plays audio, visual and animation based on the selected instruments, and automatically plays other instruments sounds as music accompaniment. Figure 5 shows the workflow diagram of the system with S stands for scores based on the number of correct hits, T stands for total number of notes in the selected notes sequence, B stands for bars of which a bar contains four notes, O stands for notes order to be hit.
The system simultaneously evaluates the user while learning to play an instrument. The evaluation was conducted by calculating the hits on the instrument's button based on the accuracy hit of the target note and the approximate time. The system automatically increases the challenge by increasing the tempo and controlling the visualization guide. There were three types of visualization guide; a note marked in a button to be hit, approximate time visualization, and notes sequence display. The visualization guides were diminished one by one when a user can correctly hit eight consecutive notes. The first part to be diminished was a note mark in a button to be hit, and then approximate time visualization, and finally the notes sequence display. As the opposite, if there is one or more incorrect hit in eight continuous notes, the visualization guide which has been diminished will be redisplayed System design was implemented in gamelan learning Android application. There were five types of gamelan music instruments developed in the application; kenong, kempul, peking, saron, and demung. An experiment involving control group and treatment group has been conducted to evaluate the proposed interactive learning model. The experiment was conducted to measure the effectiveness of the proposed learning model on how to play gamelan music instruments. The model gives some advantages for independent learning process considering that gamelan is a type of orchestra music which requires a group of players to play or to learn the instruments.
Each control and treatment groups contained five members who were second grade university students that never play and learn any type of gamelan music instruments. The members of control and treatment groups had a task to learn gamelan instrument. The five types of the selected instruments were distributed to each member of the groups. Then, the groups were scheduled to join 10 times practices conducted once a week within 1.5 hours duration at every practice session.
Two gamelan musicians with more than 20 years experiences were asked to train these groups and to measure the improvement of each member's performance. In addition, the members of the treatment group were supported by independent learning using the application. They were asked to do daily practice using the application with at least one-hour duration a day.
Likert scale was further used to measure the output of the group members' skill with excellent, very good, good, poor, and very poor grade. The instructors gave score for the learning progress of the members based on the accuracy of the notes and the tempo. During 10 times practices, the performance of the members of treatment group showed that they could learn more quickly than the members of control group. Besides, the average skill score of the members of treatment group reached excellent grade at the fourth practice, while the control group reached this grade at the ninth practice. Most of the members of control group had difficulty in memorizing the notes sequences, and this made them often play the notes not based on the tempo. On the other hand, the members of treatment group can memorize the notes sequences to play as they do daily practice using the application. Furthermore, the interview conducted with the instructors gave an interesting challenge that the members of both groups could not control the hardness level of their hits. This made the harmony of the sounds difficult to achieve as some hit harder than the other players.
In conclusion, it has been proven that the proposed model of rule-based interactive application in learning how to play music instruments can increase the performance of the learners. The application can support the user in learning how to play orchestra music instruments, especially in memorizing notes sequences and playing the note according to the tempo. For further research, it is suggested that the system will be explored to support a percussion virtual instrument application which can control the hardness of the users' hit.