Development and Evaluation of a Prototype of a Learning Support System for Honorific Forms in Japanese

Most Japanese encounter difficulty using the honorific form. A prototype of a web-based learning support system for Japanese honorific forms was developed in response to the results of a needs assessment survey. Using the results garnered from this study, learning performance and its interaction with learners’ characteristics was examined. Initial results suggests that by using this prototype tool, learners have gained mastery in assessing the situations they are engaged in, thus leading them to use honorific forms more correctly.


INTRODUCTION
Most Japanese have difficulty with communication using the honorific form, which is a formal Japanese speaking and writing style for showing respect to others.For Japanese social communication, particularly in business settings, speaking and writing in the honorific form is often required.This fact has been summarized in the annual survey report regarding Japanese language issues, which was conducted by the Japanese Agency of Cultural Affairs (2007).In the survey, 67.6% of respondents experienced some difficulty with the use of honorific forms, and 78.4% of respondents pointed out the particular difficulty of using honorific forms in certain situations requiring verbal communication.
Some grammar books (Ogata 2007, NIHONGO KENTEI IINKAI 2007) and technology-based learning systems have been developed for the purpose of overcoming this, however.For example, Ogata and Yano (1996) have proposed a learning system, and its effectiveness has already been examined, yet the problem still has still not been fully resolved.This suggests that the simplicity and portability of these systems is not yet sufficiently evolved.
In Japan, most people have cell phones, which are often used to access a wide variety of information anytime, anywhere.This could become a simple and portable way of accessing a web-based prototype learning system that has been developed to help learners better use Japanese honorific forms to communicate in formal situations.The design policy for a learning system which works in this environment has not been established yet.The procedure for developing a support system to learn honorific verb forms needs to be worked out using learners' characteristics from an assessment experiment.
For this procedure, a speaker has to understand the situation where he/she is engaged.It is not easy to recognize each situation because the speaker will usually be involved in a variety of conversational situations.Therefore, a learning support system which indicates the situational relationship between the speaker and the listener may be a useful tool.
In this paper, to examine the possibility of learning Japanese honorific forms using a Web based system, a prototype of a learning support system including tool designs was developed.To determine the effectiveness of these procedures, a simple experimental assessment was conducted.The main purpose of this paper is to summarize preliminary evidence which should be considered for improving the system.

A. Survey of learner's situations
It seems that verb conjugation of honorific forms is the most difficult task of all, so we conducted a survey to test the degree of understanding of respectful and modest verb forms.The subjects were eight university students.The mean percentage correct was 50.5%; these results showed the insufficiency of the subject's performance.According to an analysis of their responses, the following four types of errors need correcting.Type I. Selecting unusual or incorrect verb forms.Type II.Presenting duplicated honorific forms.Type III.Confusing respective and modest forms.Type IV.Failing to choose a correct active verb form while a third party individual is doing something or is active in some way.
The problems which cause the first three types of errors may be resolved when people have the correct information, but study and practice are needed to overcome the fourth type of error.To support this, a prototype of learning system and exercises has been developed as follows.

B. Overview of a learning support system
To determine the possibility of supporting the learning of Japanese honorific forms, a prototype web page with JAVA programs and some illustrations was created.This prototype can be operated on PCs where JAVA programs can be run.
In this study, a learning support system for honorific forms displays typical sentences which include specific iJAC -Volume 2, Issue 4, November 2009

DEVELOPMENT AND EVALUATION OF A PROTOTYPE OF A LEARNING SUPPORT SYSTEM
FOR HONORIFIC FORMS IN JAPANESE verbs.To use this tool, a learner selects a verb and chooses from the following: who the person speaking is, who the person being spoken to is, who a third party individual is, and the type of relationship between these three people.These parameters can be set using radio buttons and drop down menus on a web page.
To make the situation regarding the third party individual clear, and to consider what information to present, the system asks speakers to provide objective parameters.The tool then suggests some reactions in the form of a presentation.Also, users can learn some forms of presentation which people often use incorrectly.This support system provides some grammatical and presentational explanations which users can study further.However, this system can only deal with simple presentations, so users have to create their own forms of presentation using this system.
Subjects need to identify the situational parameters and select them from the menu.The system displays correct honorific verb forms suitable for use in four situations.These four situations are labeled relationship situations A to D. The system also indicates the modified figures in response to the four situations A to D, as shown in Figure 3.
• Situation A: the person spoken to recognizes that both the speaker and a third party individual belong to a like group of people who are familiar with each other.All persons belong to the same group and the third party individual is closer to the speaker.• Situation B: the speaker recognizes that both, the person spoken to and a third party individual, belong to a group of people who are familiar with each other.
All persons belong to a like group when the order of their positions is as follows: speaker < a third party individual < person spoken to, or the position of the third party individual is higher than the position of the speaker.
• Situation C: The speaker is closer to a third party individual except in situations such as the above.• Situation D: The third party individual is not specified.In order for test subjects to learn the correct expressions, some exercises based on actual human relationships were created using the above listed situations.As a first step, a pre-test to observe subjects' readiness was conducted using 15 questions about honorific expression forms.This was a paper and pencil test, but subjects could also use an electronic dictionary on an Internet web site (Yahoo Japan).Their responses were collected without revealing the correct answers.
In the second step, the subjects learned honorific expression forms using several examples which were the same as in the pre-test, such as the structure of expressions in human relationship of types A to D, which this system was intended to explain.
Finally, a post-test was given to subjects, and they were asked to complete the expressions using the new tool.The post-test questions were identical to those of the pre-test.When a subject asked about how to operate the system, an experimenter instructed him or her without using the contents of the test.After completion of the test, subjects were asked to assess it using a 5-point scale of the degree of satisfaction to answer the following three questions: Q1 This system seems to be a better tool than a conventional dictionary.Q2 I have confidence in the use of honorific forms of expression.Q3 This system is easy to use (usability).
The test subjects were 10 university undergraduate and postgraduate students.Because the purpose of the test was the confirmation of the effectiveness of the prototype system, the assessment experiment was conducted on a small scale.

A. Learning performance
Mean correct scores are 8.6 for the pre-test and 12.3 for the post-test, out of 15 questions.When the subject responded to the questions using this system, only 2 of them could complete all of the questions.This suggests that the honorific forms of expression in the questions are not easy for the subjects, though all are commonly used expressions.
As the questions in both the pre-test and the post-test are identical, and since subjects can use the system during the post-test, the scores of the post-test are naturally higher than those of the pre-test.Therefore, a performance score to evaluate the effectiveness of the system is calculated using a regression relationship (Ikeda 1982).A scatter gram of pre-and post-test scores is shown in Figure 4.The horizontal axis shows the pre-test scores and the vertical axis shows the post-test scores.
As the figure shows, there is a correlation relationship and a regression function has been calculated.The test score as a difference between the post-test score and the estimated regression score suggests that standardized learning performance occurs uniformly.The mean of the standardized learning performance score is 0.
Firstly, the effectiveness of this system was determined by considering the relationship between the performance score and the assessment of the system.Means of performance scores are summarized in Figure 5. Two groups (Q1-high and Q1-low) were generated using the scores of Q1: "This system seems better tool than a conventional dictionary".The total performance scores were compared between these two groups.Each group's sub-total of the performance scores for Type IV verbs is also compared, as the purpose of this system is the learning of Type IV honorific verb forms.Figure 5 suggests that both total and sub total scores for Type IV in the Q1-high group are positive.This means that their performance is higher than the estimated score.Scores for the Q1-low group are negative; this means that their performance is lower than the estimated score.There are significant differences in total and sub total scores between Q1-high and Q1-low groups (p<0.05).
When the learner positively evaluates this system, the performance score is high.However, when the learner does not evaluate this system positively, the performance is low.
The effectiveness of this learning method may be affected by the learner's readiness, though the level of performance is compensated for using a regression relationship.The subjects were classified into another two groups (pre-high and pre-low), which were based on the scores of the pre-tests.The mean of the performance scores is illus-iJAC -Volume 2, Issue 4, November 2009

DEVELOPMENT AND EVALUATION OF A PROTOTYPE OF A LEARNING SUPPORT SYSTEM
FOR HONORIFIC FORMS IN JAPANESE trated in Figure 6.The figure suggests that the mean for pre-low subjects is higher than that for pre-high subjects, but there is no significant difference, however.Therefore, a learner's readiness may not affect his or her learning activity and performance.The performance score was analyzed using learner's confidence of usage of honorific forms Q2: "I have confidence in the use of honorific forms of expression" as a method of self-assessment.Learners were classified into two groups, and the means of performance are compared in Figure 7.The mean of the Q2-high group is higher than the mean for the Q2-low group, but there is no significant difference.This result confirms once again that a learner's readiness does not affect his or her learning performance.

B. System assessment
According to the results of the previous section, learning performance depends on the user's preference for this system such as the Q1 scores show.In this section, the factors in their assessment are determined.
The relationship between the preference score and the usability score (Q3: This system is easy to use) is illustrated in Figure 8.The usability score of high-preference learners is significantly higher than the score for lowpreference learners (p<0.05).
Also, usability assessment scores were compared between two groups of learners using pre-test scores (Figure 9) and confidence in the use of honorific forms (Figure 10).There were no significant differences.Learner's pretest scores and their confidence did not affect usability scores.V. DISCUSSION According to the results, learners who preferred to use this system rated the usability of the system positively, and their learning performance was also significant.In particular, learning performance for Type IV topics is significant, as this system has been developed mainly for DEVELOPMENT AND EVALUATION OF A PROTOTYPE OF A LEARNING SUPPORT SYSTEM FOR HONORIFIC FORMS IN JAPANESE learning Type IV topics.Therefore the effectiveness of this system has been confirmed.On the other hand, both learning performance and usability assessment are low when learners do not evaluate the usability of the system highly.In order to maximize the performance of this learning system, learners need to understand well the human relationship involved, and also need to understand the parameters of the system when they use it.If users do not have enough grammatical knowledge when learning, the system will not be effective.This means another set of factors about each individual has to be considered, in order to design a learning support system.
This experiment was too simple and the number of participants was too small because the purpose was to determine the possibility of learning support for Japanese honorific forms.According to the preliminary results, the positive tendency was observed though various factors affected learning performance.In particular, it is interesting that a proper understanding of the functions of this system is required.More prototype assessment may be required to establish a formal learning system.These improvements will be a subject of our further study.Also, implementation with cell phones of a learning support system of this kind and a determination of the effectiveness of this kind of system for use in business settings are also subjects of our future work.

VI. CONCLUSION
The authors have conducted a survey of honorific forms of expressions and extracted a body of knowledge people need, in order to develop a learning support system for the correct use of honorific speech and writing which individuals can use in a networked environment.In this paper, we proposed a learning and support methodology for the use of difficult honorific forms verb where "a third party individual" is concerned, and we have conducted an assessment experiment using a prototype of this new learning support system.
The effectiveness of the system and the relationship between learning performance and subject's preference for use of the system are examined.A proper understanding of the functions of this system is required in advance, such as the appropriate parameters and the human relationship wherein the honorific form of expression is used.
The improvement of this system and further development of the supporting instructional procedures will be subjects of our further study.

Figure 1 .
Figure 1.A screenshot of the system interface.

Figure 2 .
Figure 2. Typical human relationships amongst three persons: a speaker, a person spoken to and a third party individual.

Figure 3 .
Figure 3. Four situations are labeled relationship situations A to D

Figure 4 .Figure 5 .
Figure 4. Scatter gram of scores between pre-and post-tests

Figure 6 .
Figure 6.Comparison in performance scores between two groups of learner's readiness

Figure 8 .
Figure 8.Comparison of usability assessment between groups of high and low preference learners.

Figure 9 .
Figure 9.Comparison of usability assessments between high and low pre-test score groups