Problem-centric Process for Research-based Learning

— Research-based Learning (RbL) extends Inquiry and Project-based Learning by facilitating an early stage exposure and training for future scientists through authentic research activities. In this paper, an iterative problem-centric RbL process is introduced, and its activities and management aspects are described. The process helps implement course-integrated research systematically and practically. Furthermore, the novel process follows constructivist methods in incorporating inquiry, scaffolding, open-ended projects, as well as a goal oriented learning approach. The RbL process is adopted in two advanced computing courses, at two different universities: a leading comprehensive Western university and a new university in a developing country. The paper summarizes new lessons learned in these rewarding experiences. In particular, the instructor should help students start their projects, by providing them with previous work or data and pre-approving the papers to review by students. He should also maintain a continuous feedback to and from students to keep the students motivated and help the instructor refine and adapt the RBL process. We note that research collaborators can help students in identifying the research topics early. The paper also shows how to alleviate difficulties that may be encountered by students who find the novel approach demanding, and consequently it also helps the instructors better manage the course contents.


INTRODUCTION I.
Constructivist pedagogy approaches perceive learning as a process of constructing knowledge by students themselves as opposed to the passive teacher-student pedagogy [1][2][3][4][5][6][7][8]. Constructivist approaches became popular in reforms of engineering education [9,10]. Inquiry-based Learning (IbL) and Problem-based Learning (PbL) are constructivist methods that have been used in medicine since the 1960s [11] and in engineering sciences since the 1980s [12]. IbL is a student-centered approach whereby students acquire the targeted competencies and learning outcomes through immersive open-ended experience. PbL involves students normally in a project work that is defined as "tasks based on challenging problems that involve the students in designing solutions, problem solving, decision making, and giving the students an opportunity to work in rather autonomous way, and results in a realistic product" [13]. These projects "include authentic contents, reliable and effective assessments, clear objectives, and a teacher role as a facilitator" [14]. There is no unique model of PbL, and the literature on this subject varies; howev-er, PbL projects have some generalities. For instance, PbL projects have clear goals [16], improve students' autonomy and foster their experiential learning [17] and problem-solving [15] skills. PbL projects are not trivial tasks [15] and normally address non-trivial challenges [18]. PbL engages the students in authentic experiences that foster self-regulated learning, and in defining their own objectives within the limits of the course [19]. PbL methods have been successfully applied in K-12 [17] and in higher education [20].
Research-based Learning (RbL) is an approach in higher education that fits into IbL/PbL domain. RbL focuses on the development of learners as independent researchers. RbL also helps the learners liberate their thinking, develop their writing and presentation skills, and gain confidence in their intellectual abilities. Sometimes, forms of IbL/PbL are referred to as RbL [64]; however, the boundaries between them should, in our opinion, be more distinct. Indeed, RbL falls into IbL/PbL approaches; however, not every IbL/PbL is a rigorous RbL approach. For instance, many PbL projects emphasize on design and implementation, and not on research. IbL focuses on inquiry, and in a general sense is closely tight with RbL, which also has inquiry in its core; however, while it is not necessary to have a research outcome in IbL, the emphasize on research is clear in RbL. Thus, the relationship between IbL/PbL and RbL can be shown in a Venn diagram as shown in Figure 1, where RbL represents a subset of IbL/PbL. RbL enhances students' perception and interest in science careers [57][58][59], confidence and self-capacity [31,52,60]; learning outcomes [53,61,62]; management, communication, organizational, and leadership skills [24,61]; and likelihood to pursue graduate studies [31]. Nevertheless, due to lack of maturity and motivation, undergradu-PAPER PROBLEM-CENTRIC PROCESS FOR RESEARCH-BASED LEARNING ate students especially those new to RbL can show skepticism to RbL [50,63]. Moreover, it is often difficult to identify an open-ended research problem that could be finalized within a time frame suitable for undergraduate studies [63].
The main method to conducting RbL in undergraduate research (UR) is through senior projects or curriculumindependent projects [64]. Other methods include summer research programs [28,65], paid part-time jobs [62], and internships with local industries [66]. As opposed to UR in senior projects, course-integrated research [61] can expose a larger number of students to research experience and as early as possible [64,67,67].
There are limited studies on research-integratedcourses [57] since this practice is not widely spread; however, it is considered as an effective and low cost way for incorporating larger portions of undergraduate students in the research experience [52].In this paper, we introduce a process and a learning experience that exercises applied research in two computing subjects for senior students at two leading research institutions. The process has its roots in engineering design, and software development methodologies, which are both normally of a problem centric nature. Engineering design and software development methods have been widely adopted and successfully applied in engineering and computing for rapidly and systematically tackling the targeted problems. Our framework, suggest that such methods can be transferrable into the learning sciences for rapid production of meaningful solutions, e.g. publishable applied research.
The remainder of this paper comes in four sections: Section 2 presents the proposed RbL process. Section 3 describes assessments and evaluation tools used in the courses. Section 4 discusses issues and findings of the work. Finally, Section 5 concludes with future directions to extend the work.

RESEARCH-BASED LEARNING PROCESS II.
Embedding research in courses can be accomplished through the pedagogical modeled noted in Figure 2 as an iterative process. The courses may offer a mix of theory and practice, and the RbL process helps achieving the practical objectives and learning outcomes of these courses. The process starts after essential theoretical topics are covered, then extends over the research course. It includes activities that are iteratively performed in order to solve a research problem. These activities aim to train students on research and help them to publish their findings in peerreviewed venues. This process would help the setting of objectives, milestones, and work timeline. Detailed guidelines on these activities are introduced through written and oral notes, meetings, and feedback from the instructor. The process activities and related management issues are detailed in the following subsections: In this first activity students acquire background and foundations of the subject matter. The activity continues throughout the course. However, the intensity (both the breadth and the depth) of the gained knowledge gradually decreases as the research components of the course take over. This may vary from one subject to another, but all fundamentals and theory material should be covered before the start of the last third of the course time period. This activity aims at setting goals and objectives of the research exercise. Initially, instructors may have more influence on choosing research problems to be solved. Gradually, students could be allowed to propose alternative directions as their research competencies improve. Students are to be encouraged to interact with professionals and experts in their chosen project problem areas. Deliverables of this activity are proposals to state motivations, challenges, and significance of the identified goals and objectives. Students can later modify the details of the objectives and aims of the project, to accommodate new insights and ideas gained throughout the project development. This activity is not expected to last for a long time, however it may be revisited for refinement.

Review C.
This activity involves reviewing, summarizing and critiquing published research work. In particular, students review a reasonable number of recent and relevant research papers that are published in reputable journals and conferences. In its early stages, this activity targets general research papers in the field of the research problem, and then gradually focuses on publications closely related to the students' respective research problems. Furthermore, to ensure quality of students' write-ups, some guidance pertaining to reading comprehension and technical writing skills is required especially in the early stages. A byproduct of this activity is to familiarize students with the composition and formatting of high quality papers, evident in the smooth flow of logic, result interpretation and conclusion drawing. Students may spend considerable amount of time to perform this activity, especially for those experiencing reviewing research papers for the first time. Furthermore, the activity may be performed more than one time to included more related articles in the review.

Define D.
In this activity, the students define project problems formally. They formulate the problem objectives and constraints, and may map it into a well-known problem, previously tackled in the literature. They can, for example, iJEP -Volume 5, Issue 2, 2015 PAPER PROBLEM-CENTRIC PROCESS FOR RESEARCH-BASED LEARNING formulate a problem that aims to minimize or maximize some variables as an instance of a known class of optimization problems. Moreover, during advanced stages of their work, students formally describe their developed techniques and relate them to the defined problems. The time needed to perform this activity may vary, nevertheless it should not be long.

Collect E.
This activity consists of searching in private, third-party or Internet sources for data necessary for the investigation, and for tools to solve the problem. Such data and tools can be in primitive and basic forms. For example, data could be represented as tables, spreadsheets or even raw text files; and tools could be available as executable programs or as source code written in various languages. Therefore, students are expected to explore, self-learn, prepare and customize. This activity may take as much time as the literature review activity. Students are not expected to repeat this activity several times as the students will be immersed in working with data/tools they collected and unlikely .

Solve F.
This activity involves designing, implementing, and integrating solution techniques, as well as tuning their parameters, executing them, and collecting their results. It is important to ensure that solution design and implementation are sound, parameter settings are described and justified, and results are evaluated for their quality and significance. Students may start this activity by trial and error or by reproducing and improving existing work, especially in projects whose objectives are difficult to identify clearly or precisely. Often, the involvement of experienced personnel, such as course instructors, is needed to help in assessing the interestingness and usefulness of the solutions. Consequently, this activity can be time consuming since the work may be extensive and repeatedly performed for refinement.

Interpret G.
Finally, interesting results are compiled and adequately displayed to aid in result interpretation and conclusion drawing. The interpretation may confirm or reject initial hypotheses, or indicate new trends and findings in the application. Proper compilation is essential here as it helps emphasize the significance of the work, interpret the findings and highlight the contributions. The outcomes of this activity show whether the accomplishments are significant enough to bring the work to a conclusion. As with the other research project activities, interpreting outcomes and summarizing findings could be redone as much as new results from the previous activity, solve, are collected. The time it takes to do this activity is relatively short, also compared to the previous one.
At the end of each of the above mentioned research activities; 2 through 7, all students are required to write reports for proper and a timely documentation in a project portfolio. Instructors assess the value of the research work when the reports are handed in. Depending on this assessment, students are asked to either start subsequent phases or redo their work partially or as a whole. As a part of the evaluation of students' work, students could also be asked to prepare a comprehensive presentation. Moreover, final reports are requested to be in a format ready for publication, since it is expected that a large number of these reports will become well-rated articles in reputed journals and conferences.
EVALUATION AND ASSESSMENT III.
Several tools can be used to evaluate and measure the success of the learning process, including the number of publications arising from the work, student evaluation, general surveys and feedback from individual students.

Publications
A.
The number of project reports accepted in peer reviewed conferences and journals is the primary measure of success. All students are expected to submit their work for inclusion in reputable journals and conference proceedings. Before submission, the instructor and the project collaborators, if exist, review the final reports, and refine them and ensure that they are formatted according to the publishers' requirements.

Feedback and Interviews B.
Students' feedback and opinions can be evaluated qualitatively by surveys and interviews. Student feedback is not only important for the overall evaluation of the RbL process, it is also used to guide and refine the process itself. Students are constantly encouraged to give their feedback throughout the courses. Interviews with students are conducted to collect further in-depth informal feedback. Moreover, owing to the open-ended nature of research problems, students may need guidance in estimating the project time and in meeting the deadlines.

Assessment Tools C.
In addition to midterm exams, students are required to individually submit assignments, which are also openended type of problems, albeit on a smaller scale than the projects. The assignments and the traditional exam will help in assessing the important objective of providing students with skills to generate research work that can be benchmarked internationally. The assignments also enable students to gradually master the fundamental topics in a solo-mode before or parallel to the main research project, which conforms to the Zone of Proximal Development theory of Vygotsky [87].
Evaluation of the in-class presentations of the assignments and the projects is carried out by the students themselves, who act as peer reviewers of their classmates' work. Such evaluation will also hone their skills and widen their involvement in the research work.

CASE STUDIES IV.
This section outlines the results of two case studies. It also summarizes the observations and lessons learned from the two experiences of learning through research and their evaluation.
The RbL process was applied in two computing subjects for senior students at two leading research institutions: the Faculty of Engineering at the University of Waterloo (UW), Canada, which ranks among the top 75 engineering and computer science institutes [69], and Qatar University (QU) in Qatar, which aims to develop a knowledge-based society by 2030 [70,71], and funds Undergraduate Research Experience Program since 2006 [72]. The process was adopted in UW for an elective course titled 'Machine Intelligence and Soft-computing' (SYDE 422) offered to fourth-year students from the de-PAPER PROBLEM-CENTRIC PROCESS FOR RESEARCH-BASED LEARNING partment of Systems Design (SD) and the department of Electrical and Computer Engineering in Winter 2008. The total number of registered students in the course was 34 and they were all full-time students and the majority of them were from the SD department. The process was similarly applied in an evening Data Mining course (CMPT 563) offered to Master of Science in Computing students from Computer Science and Engineering Department, College of Engineering at QU, during Spring term of 2011. There were nine students registered in the course, and all held fulltime jobs during the day.
Assessment and discussion A.

Publication. 1)
About 50% of UW course projects and 10% of QU course projects were published. Seven publications were published from the UW course: six in conference proceedings, and one in a journal. From the QU course, four project papers were submitted and one was accepted in conference proceedings. Due to the unavailability of students, all the conference papers but one were presented in the conferences by the instructor. More projects could be published if they had been expanded and improved. This was hard to accomplish after the course ends and the students' enthusiasm fades away Feedback and Interviews. 2) QU students were interviewed at the end of the study term to collect further in-depth informal feedback. The interviewed students agreed on a number of issues: First, the students found RbL motivating, challenging and unusual. Second, after participating in RbL they became aware of both the direct impact and the indirect impact of the research approach and the use of data mining techniques on their scholar and industrial work. Third, they believe that they would have benefitted from RbL as undergraduate students as well. Fourth, they emphasized the importance of research and development for their organizations. The feedback from UW students was generally positive because RbL had opened new career choices for the students, and let them experience a complete cycle of a research process in one course. Some students from UW and QU, however, found it difficult to keep up with the RbL demands and needed constant extensive guidance all the way. As expected, a few students found the openended nature of the research problems challenging and had difficulty in estimating the project time and in meeting the deadlines. The students appreciated the experience of learning and actually conducting research. They also appreciated knowing future career perspective. RbL has also improved QU students' awareness of the importance of R&D in their organization. Such awareness is of particular importance to states establishing a research culture, like the State of Qatar.

Assessment Tools 3)
For the QU course, students wrote an open-book midterm test to encourage them to review the course fundamentals before going in depth in the research project work. In addition to the projects, typical and open-ended assignment exercises were required. Peer assessment was employed to evaluate the presentations of the UW projects. Noticeably, the outcomes of these peer-evaluations did not differ greatly from the instructor's grading.
The projects and midterm exam grades in the QU course (plotted in Figure 3) correlated highly with final grades of the students. Correlation coefficients, calculated between 'midterm exams' and 'final marks' and between 'projects' and 'final marks' are 0.9. In the contrary, the correlation coefficients between 'final marks' and 'assignments' range between 0.2 and 0.4. Therefore, in order to reflect students' performance, it was wise to distribute the weights of 'final marks' as follows: Assignments are 25%, Midterm Exam is 25%, and Project, which was composed of 3 milestones, is 50%. Moreover, the correlation shows that relying on projects only (as done in the UW course) for assessment is sufficient to reflect students' performance. This section highlights the requirements necessary to perform the RbL process activities from the acquiring of background knowledge, and identification of the research objectives to the interpretation of the results: Acquire 1) Introducing fundamental topics of subject should be planned to be covered within the first two thirds of the course. After that, advanced topics could be covered. This allows equipping students with the basics to start their projects, and focus more on them towards the end of the course.

Identify 2)
Providing students with ready research topics is more effective than letting the students propose the topics. Projects with preset research topics constituted two thirds of the works successfully published. Moreover, 80% of the projects conducted in the UW course were co-supervised by research collaborators.

Review 3)
Students tend to choose articles they find through quick web searches. If they are left on their own, they could select low quality papers. Therefore, the instructor should approve the papers to be reviewed.

Define 4)
This activity is new to most students. They had been studying and taking definitions as is, and it would be challenging to shift their paradigms to how to come up with those definitions. Making students aware of this activity early helps them to perform it successfully.

Collect 5)
Like the 'Identify' activity, the 'Solve' activity can be performed better when the students are provided with some previous work or data to start with. Such tools and data can be provided by research collaborators, who had worked on similar problems. Students would appreciate iJEP -Volume 5, Issue 2, 2015 PAPER PROBLEM-CENTRIC PROCESS FOR RESEARCH-BASED LEARNING research collaborators guiding them to the changes or extensions required to solve their own problems. In fact, most of the contributions made by the students resulted from this activity.

Solve 6)
Noticeably also, students do well when they work on problems previously tackled. Students design for improvement, and implement extensions of existing solutions with much more ease than starting from scratch and slight ease when integrating solutions. To elaborate on this, solution integration aims to merge all pieces of accumulated solutions in one seamless process, either by integrating a solution across all the existing parts, or by integrating these parts to interface with one another to accomplish a common goal. Unexpected issues may arise during solution integration since the existing parts are often developed by several researchers (students, collaborators, and others). Therefore, this activity depends in general on several factors: the nature of the problem at hand, the quality of the pieces, and their ability to integrate.

Interpret 7)
In this activity the students analyze their findings and highlight their work contributions. To students, this activity can be challenging, especially when interpreting qualitative rather than quantitative measures. Most of UW projects results were from the latter, however in Data Mining (QU course) some tasks (e.g., clustering, and association rule extraction) are considered exploratory more than confirmatory.
Generally, a close review of students' produced work (documents, codes, etc.) from all performed activities and continuous provision of feedback is essential. It is expected that the quality of students' work would not be high and would have many and different deficiencies. QU students needed extra help in this matter. The one major difference, between the two classes (QU, and UW), is the strength of intrinsic motivation and intellectual quality of students, influenced mainly by different samples of admitted cohorts. For instance, UW has been historically attracting high quality applicants from Canada, and also from overseas. Often, high quality students in Qatar will continue their education abroad supported by a significant number of available governmental and industrial scholarships instead of applying to Qatar University. Furthermore, a significantly higher competition in the graduates' job market in Canada, as compared to Qatar, provides an extrinsic motivational factor to work harder for UW students as compared to QU students. In 2011 Qatar unemployment rate was 0.4% (ranked 2 worldwide) as compared to 7.4% unemployment rate in Canada (ranked 86 worldwide) [88].
Another aspect to watch is the time it takes to carry out the different activities. As projects may vary, some turn out to be more difficult than expected. Hence, more time is needed to finish these projects activities. In a fifteen weeks long courses, the project could start by the end of the forth week, and the following are typical durations for the activities; identify: 1 week, review: 2 weeks, define: 1 week, collect: 2 weeks, solve: 3 weeks, and interpret: 1.5 weeks.
We note that it is not necessary for every undergraduate research (UR) work to become of a publishable quality in indexed specialized journals. Yet, there are a number of research journals and conferences that were established to publish undergraduate research. They accept UR work for publication after passing a specific quality threshold. They provide venues for students to practice academic writing, submission and reviewing processes. Moreover, UR conferences introduces networking and academic conference experience to students and let them exchange ideas with like-minded peers. CONCLUSIONS V.
In this paper, we introduced a systematic process to guide students in conducting research. In the course of demonstrating this process, students learned to master subjects by applying abstract concepts in practical research projects. It was evident that research skills of a student are more effectively developed and improved while working on real-world problems under the supervision of experts and practitioners. This work was a pioneering pilot realization of the RbL process, and its success depended on the commitment, collaboration and active participation of all its members.
It should be noted that the work is quite involving and may be resisted by students who do not have all steps clear for them. Therefore, we highlight the requirements necessary to perform the process activities effectively. In particular, having collaborators would help alleviate many difficulties and take some of the supervision and guidance load off the instructor. Collaborators can help in identifying the research topics early to the students. The instructor should provide students with some previous work or data to start with, and pre-approve the papers to be reviewed by students. A close review of students' produced work from all performed activities and continuous provision of feedback is essential.
The quality of students' work can have deficiencies, depending on the maturity of the students and the university, and the competition in the work market which can be a major motivation for students to excel. It is important to research for pedagogical and psychological interventions to improve the interest of students of low motivations in universities such as QU as compared to students in leading universities. Potential areas of investigation from the learning sciences and educational psychology are selfregulated learning, self-efficacy, motivation, and selfesteem. We also suggest applying the RbL process to more senior graduate and MSC courses, and to analyze the outcomes of this application.