variPiano™: Visualizing Musical Diversity with a Differential Tuning Mobile Interface

—In this paper the framework of Computer Music interfaces is convolved within the context of Human Computer Interaction (HCI) to produce metaphors and paradigms that engulf differential tunings within adaptable mobile device screens. A presentation of the basic concepts involved in the Computer Music world prevails the discussion on the evaluation criteria for Computer Music virtual instruments and interfaces. New criteria are introduced, like knowledge representation of musical data and alternate musical interfaces materialization. A survey of various systems, based on the previously mentioned criteria is committed.


INTRODUCTION
Few people are yet overwhelmed by the advent of mobile computing, which while being characterized by small size and portability may have the crude impact of popularizing a new revival for electronic sound synthesizers, i.e. instruments of awesome dimensions that had revolutionized the global music of the 60's.
Indeed, new species of music had evolved then, based on instruments like the one depicted in Figure 1.
The instrument seen there was used to simulate the advent of new music styles and techniques, an idiom of the 60's, when musicians were able to handle sounds exceeding the range and versatility of conventional instruments.
Through the years, research in Human Computer Interaction (HCI) has clarified that interaction with complex systems does not take place in a merely physical level but in a much broader environmental context: in simple words, a simulating display interface is not enough, if not previously the metaphor and the paradigm of an alternate "world" are not comprehended. In our case the alternate system, is the varying world of global music that deviates from the well perceived Western music. It is true that Western music, the dominant music paradigm, permeates every society; however, ethnic music is inexorable due to the globalization of music streams flowing within the computing "cloud".
Music is a rich communication medium, and Computer Music is the amalgam of interface science and musical praxis forming a dynamic subset of Human Computer Interaction.
There are structural similarities between the job of a music composer and that of a User Interface designer, albeit their objectives may be different: both seek to tame relentless streams of symbolic information. As seen in Figure 2, the production of music involves handling switches, knobs and buttons, and that is where both the composer and the engineer move their creativity beyond limits: in spheres where symbols merely can describe the rich multimedia content [1].
Indeed, when the first synthesized sounds started circulating around, there was difficulty for those using semiotics to describe the new sounds of mixed overtones giggling with harmonic-tone generators of "unnatural" signals, as then the sawtooth wave signals were perceived [2]. And yet, this is now the public hearing context for global pop music, received as the ordinary norm of most middle aged and younger audiences! Beyond that, a new class of musicians appeared, serving the new specimen for synthesized music, as is the case of the well known composers and performers Vangelis, Jean-Michel Jarre, Johannes Schmoelling, and Yanni, to mention some indicatively.
The task accomplished by these well renowned artists is immense: In most cases sound had been used in general purpose interfaces as a primitive object, and its use had been deteriorated at a low level, that of signal-processing approach. Indeed, in the beginnings, the handlers of the first synthesizers dealt with voltage changes that controlled pitch, timbre, attack -decay of tones, vibrato and several other low level aspects of sound ( Figure 2). However, music composition and performance are highly abstract human activities involving a semantic and a symbolic mechanism of human intellectual activity. The aforementioned artists and many other alike managed to tame a new world of devices and demonstrated extraordinary musical talent, opening the road ahead for exploring the possibilities of computer composed and synthesized music within the context of object oriented graphical user interfaces [3]. In this paper, we are dealing with subjects which lie partly within the province of science and partly within the province of art. Yet the boundary of these two disciplines is not always clear. However, the mistily borderland is not perpetrated by the peremptoriness of ongoing multidisciplinary research, but mainly due to our nebulous understanding of how the neurons of the auditory nervous systems respond to frequency tuning [4]. This is the case of complex wave shapes with asymmetric shapes for width modulated effects (Figure 2 revisited).

A. Background musical information
For reasons of facilitating the reader, we use the convention of the acoustic piano interface, which seems to be the global interface that most musicians use to accredit synthetic sounds with the semantics of music: the notes of the well tempered scale.
Not to mesh up with such analyses below the aural substrate, musicians are merely concerned with finding the most satisfying perceptual organizations that are consistent with the musical surface [5]. Most theories for perceptual organizations unfold on two axes: The first axis relies on the likelihood principle of preferring the most probable interpretations, and gives a mathematical approach for harmony. In 1738 Euler attempted an explanation on psychological lines, stating that the human mind delights in law and order, and therefore, it conceives as natural and more pleasant sounds that have frequencies whose ratio may be expressed with the smaller numbers possible. For instance, the frequency ratio of the common chord C E G C! is 4 : 5 : 6 : 8. Helmholtz in 1862 prolonged this theory of consonance and dissonance in terms of beats, and accordingly we have the example of C and G that sound well together, since only a few of their harmonics beat badly This approach has evolved in our era as a statistical interpretation for the harmonic and melodic structure analysis of music.
The second axis copes with the fundamentals of psychology and in particular Gestalt psychology, as far as HCI is concerned. This axis evolves on the notion of grouping, which uses metrics based on the principles of proximity, similarity, symmetry, and continuation.
By these principles rhythm, form and content are put into taxonomy.
These principles are considered weak [6] when it comes to contemporary music anticipation, and this is the point where recent research is focusing on the endless possibilities of expression that new performing media have brought to the "whole mysterious world of sound" [7].
Most importantly, the new hi-tech substrate acts as a technology enabler in music, since it is "also adding an unbelievable variety of new timbres to our musical store, but most important of all, it has freed music from the tempered system, which has prevented music from keeping pace with the other arts and with science" [7]. Western music theory and practice is many times insufficient to explain the uniqueness and the richness of the melodic structures and the metric structures that have emerged in contemporary music scenes (like that of Eurovision) as an amalgam of traditional Eastern music.
In musical mathematics these concepts are expressed as follows: Let a scale S be uniquely described by any list of intervals that amounts to an octave, and the modal group Gs of S as the set of all rotations of those intervals (that is, all permutations obtained by removing the i-st value and placing it at the end). For scale S to partake m notes means it comprises m intervals, i.e. it is an m -atonic scale. The notes in S are the numbers, commencing with zero, obtained by adding each interval in turn.
The key designation implemented in this paper will be given firstly formally and then, informally, by a rapid prototype visualization. In the former case, let S be any m-atonic; then the n-spectrum of that scale is the set of all scales Z of n notes such that exactly one of the following criteria holds: m < n and some z ! G Z contains all the notes of S (1) m > n and some s ! G S contains all the notes of certain scales Z (3) Some examples will highlight the musical consensus for these relations: the most trivial scale is the well tempered scale, coming from the 8-spectrum of an octatonic scale. In terms of music commas it is expressed by the sequence in cents and comprises only the white keys of the piano roll.
iJIM -Volume 9, Issue 3, 2015   6 20 4 12 6 20 4 ] (5) It is the scale for mode Second plagal in Byzantine music, roughly approximating maqam Hijaz of Middle East music; however, if 5-tuplet [ 0 4 12 6 20 ] (6) was considered, then we have an different hearing, both in terms of hearing (it is more soft -chromatic) and note distribution [8]. Fortunately, both subsets stretch to the same number of echomoria, namely 42. However, it is reasonably obvious that these two pentatonic scales do not contain the same musical and pedagogic information.

B. Rapid Prototyping of the Interface
Diving into the musical and acoustic background of microtonism is really unusual to software engineers who need not necessarily plunge into the concepts of polytonality or pandiatonicism to produce an adequate interface.
Indeed, in Eastern music traditions performers come across with scales having various "structures", as is the case of pentatonic scales containing a variable number of notes, from 4 up to 32! What increases that much the number of notes involved is the existence of many alterations between notes, partly due to speech inflexions, as is the case of vocal traditions in Eastern countries, or as a result of string instruments like santur, that indeed include adherent notes between mainstream ones.
Practically, this means that the well known piano interface of seven white and five black keys per octave is altered as shown in Figure 3.
Indeed, musicians refer to the "well tempered clavier" as the prevailing interface for Western music being shaped since the Bach era [9]. For most people this is equivalent to the easily recognizable piano Interface on the left of Figure 3. However, for many people outside the prevailing realm of Western societies, such an interface would be considered as a meager contribution and its contempt would be remedied by the interface on the right.
What makes things worse, is the advent of chromaticism [10], that demands alterations not only in the number of notes per octave, but most importantly, in the intervals between main notes. Thus, a keyboard that could indulge polytonality stretches to an interface like the one seen in Figure 4. Needless to say of course, that the interfaces of Figures  3 and 4 were not fully functional, since they were manipulated with a mouse and lacked direct manipulation. Additionally, there was no way to play simultaneously two, three or more notes.

III. MOBILE DEVICE INTERFACES TAXONOMY AND USER SUPPORT
Although the leader in new technology penetration was desktop computing for more than a generation, neither its metaphor nor its paradigm seemed to overindulge the multifaceted mobile device interfaces [11].
It is well perceived in HCI literature that people use short term memory in conjunction with their long term working memory for processing information. In our research, short-term memory processes the auditory input, whereas working memory rescinds knowledge and elucidation. Therefore, Americans may not be successful in remembering Russian words, Chinese pictograms or Spanish sayings. Even further, most people cannot speak or understand more than handful of languages; when it comes, though, to the conception of an arrangement of sounds as harmonious or dissonant, then the human brain is stimulated to proceed with psychophysical and behavioral masking that increases the sensitivity threshold of our aural perception [4].
Theoretically in medicine, in agreement with the facts elaborated for somaesthetic synapses, the tuning curves of groups of neurons can be biased to elaborate increasing steepness specific within the frequency domain. Further research into the diverse musical cultures throughout the world has deepened our knowledge for universal versus genre-specific aspects of music, and has enabled us to truly evaluate the generality of various modeling strategies [10]. Additionally, the advances in the HCI perspective for increased interaction with performing instruments illuminate the historical fades and theoretical jump cuts when confronting various cultures, promoting musical creativity, expressivity and interaction.
The assessment of HCI elements for Computer Music under the prism of usability aims at the development of new graphical tools, new symbolic languages and finally better user interfaces. Technological advances in these fields challenge for increasing demands on the quality of the user interface and offer the potential to further progress the functionality of Computer Music devices.
Under this prism, HCI becomes more central to the design and development of Computer Music systems investigating functionality that does not exist for the user or functionality that is not virtually usable [11].

A. Existing simulation standards
For reasons of facilitating the reader, we use the neologism Mobile Computer Music interfaces when referring to software running on mobile devices aiming to produce melodic pieces. The instrument used in this case is a computer program, this time not withstanding the hardwired via the MIDI IN and OUT ports usual keyboard. Producing melodic lines is a matter of inspiration and not an arbitrary or disciplined procedure. Therefore, in terms of HCI the touch-pad device used must have functionality and usability features that enable the user to record in symbolic form the music he has conceived [3]. Usually, five criteria are used in order to evaluate the usability of an interface according to the ISO/DIS 9241-11 directive: I. Learnability for using a new system II. Effectiveness, i.e. the extent to which the intended goals of musical synthesis and composition are achieved III. Efficiency, when used by experienced and trained users, i.e. the amount of resources that have to be expended to achieve the intended goals IV. Satisfaction, in the sense of the extent to which the user finds the use of the product acceptable and desirable V. Capability to use the system from users not familiar with its musical categories and predicates after a long time.
Some of these criteria are more procedural than quantitative. For instance, although most Westerners identify music semantics with the Common Music Notation (CMN) standard, a performer from the East would reasonably argue that in Figure 4, where the notion of parametric interfaces is presented, we need transcendence not only in semiotics but also on the concomitant events of the corresponding User Interface.
Nevertheless, in Table I according to [3], [6] and [11] a classification of the musical interfaces has been attempted based on the previously mentioned criteria I, II, …,V.  Table I are tailored to the task of Computer Music simulations, and as a result they are slightly altered [12].
In practice these criteria demonstrate that a fancy piano interface for a mobile device, like the various ones readily available at the Android Play Store, may seem more versatile, but its rigidness prevents from implementing a pentatonic, parametric scale with more than five accidentals.

B. Providing an augmented performance interface
variPiano™ is an exemplar application for Android that incorporates a paradigm allowing the user to create keys of varying sizes, which are provided either at the beginning or later via the apk settings. The vector describing the scale according to relations (1), (2) and (3) is provided by the end user, in a style similar to (4) or (6).
The amount of keys entered is practically unbounded, and the scaling of the keyboard is based on inter-number ratios. The user may also provide scales encompassing part of an octave, as is the case with pentatonicism or tetraphonicism.  Figure 6. Left, inserting a vector representing the accidentals between two notes; right, the user can toggle the visibility state of the tacit orange editing buttons and gain vertical space for the black keys. Also, he can hide interaction, moving to the performing surface, where he can produce melodies performing with the virtual keyboard The tacit points in the variPiano™ snapshot on the right of Figure 4 represent a second level of interaction. Indeed, as it has been demonstrated with the rapid prototypes of Figure 3, the model on the left lacks not only visualization for pentatonic intervallicism, but also it does not appear to signify any accidentals.
Therefore, variPiano™ does not only encompass accidentals by triggering the tacit keys, but, by clicking on a tacit key, an edit button appears allowing the user to input a secondary vector that ignites the insertion of a series of numbers. These numbers signify the arrangement of a number of accidentals between the basic notes the tacit key lies on.
In terms of music theory, this is clearly a microtonal keyboard with in-situ interaction. On the popup that appears, as seen on the left screen of Figure 5, the user is prompted to insert a secondary vector, as space separated numbers, that mathematically denote how the interval between two consecutive notes is split in terms of frequency between one, two, three or even more accidentals! For visualization purposes a vertical array of black keys appears, incorporating the accidentals. Their size and their position are relevant to their distribution: the key on the bottom of the screen is having the lower frequency, closer to the note of the white key on the left, while as we move to the top, higher in frequencies are denoted, closer to the white key frequency on the right of the tacit point.
For visualization purposes, although we use the terminology "white keys" as a delicate hint to the white keys of the piano roll representation for the unique notes of the octave, in the variPiano™ interface consecutive "white keys" appear with shades of gray, for the purpose of visualizing the intervals between the major notes. A key that is wider than its consecutive ones denotes that the interval between these notes is in terms of frequency bigger than the 200 cents that compartmentalize most of the successive white piano keys.
Of course, when the vector for accidentals is inserted, a primordial validation takes place, since the variPiano™ interface is not a prototyping tool, but primarily a performing instrument.
Once a valid input is set, the black keys will be generated and fill in the space between the white and grey keys. The careful observer will note that the tacit symbols in Figure 5 have changed to an editing symbol, pertaining a parametric alteration for accidentals, not only in tonality but also in number, indeed a unique feature of variP-iano™!

IV. CONCLUSION
For the first time a multi touch software interface can depict visually scales alternate to the tempered scales of the Western music literature. Taking into advantage the highly interactive gadgets of Mobile Device Interfaces, it can also parametrically visualize two -or more! accidentals in vertical serialization and create the long awaited by the Computer Music industry software microtonal keyboards.
Its main objective in near future is to promote alternatives for hardware devices that not only cost significantly more but also bear interface shortcomings.

ACKNOWLEDGMENT
The conducted research has been boosted under the auspices of the A! AHEPA clinic on ENT diseases "Hearing Implant Program" at the Aristotle University of Thessaloniki General Hospital. This project provides worldclass treatment in helping patients who have significant hearing impairment, allowing them access to a full-range of educational, social and vocational life experiences. Its main scientific focus is on enhancing oral communication by fine tuning auxiliary aural equipment. A version of variPiano™ will be used for parameterizing rehabilitation.