Virtual Clay Prototyping System - A Framework for Real-Time Modeling

In order to provide users a real-time force-feedback immersive virtual modelling environment, powerful computing capabilities are often needed. As this potentially results in a calculation bottleneck, virtual modelling scenes are often delayed and resulting in a poor user experience. In this paper, the authors present a novel approach to improve the calculation speed for the volume pixel (voxel) amount of force feedback generated based on the amount of pressure exerted on the ‘virtual material’ per unit time. Results from the experiments reveal that this approach required less computing power, thereby offering a better user experience. This has been confirmed through a series of trials that investigated the amount of time students spent during sketching, physical modelling and virtual prototyping. Results found that the remapped virtual prototyping method was more effective than physical model in productivity in terms of time, and efficiency in terms of data conversion.


I. INTRODUCTION
In broad terms, prototypes can be categorized as physical or virtual. Within New Product Development (NPD), they are used for testing and evaluating functional, aesthetic, and ergonomic aspects of a design [1]. Research on the real-time integration of physical and virtual models was first proposed by Burns [2], who suggested the concept of a "Graspable User Interface" -an environment that could potentially combine seeing and feeling physical objects within a virtual environment. Such prototyping has also been found to be useful within the NPD process, as it encourages innovation, supports collaboration, and inspires creativity [3].

A. Sight and Touch Sense
In NPD, physical objects are perceived by engineers, designers, and stakeholders through seeing and touching. Schifferstein and Cleiren highlighted that our visual and tactile senses provide a holistic understanding about an object and these senses dominate product perception and our experience in real-life. Our vision allows us to form the first impression of a product, such as its appearance. For example, vision is the only sensory modality that can convey information about colour [4], and Xu suggested that visual information is cross-modally integrated with haptic judgments of weight distribution [5].
In contrast, touching is often referred to as a proximal sense that differs from sight, smell, and hearing as they operate from a distance and through the air. Touching a product can increase one's judgment of product evaluation, and tactile input is often used to sense a product's substance in terms of stiffness, roughness, softness, and smoothness rather than its macro-spatial aspects such as shape and size. Kahrimanovic investigated the human ability to discriminate the size of 3D objects by touch [6]. Overvliet conducted a haptic search experiment to investigate the influence of the Gestalt principles of proximity, similarity, and good continuation [7].

B. Haptic Force-Feedback
Today's computing technology now allows digital virtual models to be "felt" like a physical object. This is mainly achieved through the use of real-time force feedback, also known as haptics that utilize force, pressure, vibration or motion independently or combined. To make the 3D CAD system become the best carrier of professional technologies and realize the accumulation and reuse of enterprise design knowledge and design processes, a design knowledge management method based on 3D CAD system was presented [8]. Haptic feedback technology integrated with a graphical display could potentially enable a significant increase in the realism of virtual prototyping and further enable a more immersive user experience. The use of such technology has been shown to enhance the quality of man-machine interaction such as in research by Coles [9], who investigated how haptics helped in medical training simulators. Coles found that the use of force feedback reduced 40% of the amount of the time to complete tasks. Other researchers also looked at how the manipulation of virtual objects can be made more realistic through the use of 6 degrees of freedom (DOF) haptic feedback devices [10]. Such 6 DOF systems allow the position of coordinates to be achieved by up/down, left/right, and forward/backward motions [11]. Jones [12] reported on a small scale experimental study in haptic discrimination using a mobile device with a fixed rein interface. Xia [13] demonstrated that hapticbased virtual assembly was a valuable tool for assembly design and process planning.

II. VIRTUAL CLAY PROTOTYPING SYSTEM
Virtual prototyping occurs in a digital environment with processing similar to how manual modeling takes places with physical clay. Virtual prototyping is highly procedural and systematic. But as a number of computational processes are involved, current systems are PAPER VIRTUAL CLAY PROTOTYPING SYSTEM -A FRAMEWORK FOR REAL-TIME MODELING still limited in terms of integrating these actions that take place at the hardware and software level. In an attempt to resolve the issue of force feedback control in a virtual clay prototyping system (VCPS), a mapping relationship between virtual sculpting (cutting and pasting operations) was first established.

A. Mapping Operation in VCPS
During the process of conventional sculpting, the operations of cutting and pasting are the main methods of modeling. As shown in Fig. 1, these two modes of operation can be further extended into the third operative mode -compensating. The cutting operation on the clay model surface can be represented by cut sim , as shown in Figure 1(a).
The add operation using styling tools can be represented by pst sim , as shown in Figure 1(b).
The operation of scraping the convex surface and filling the hollow at the same time can be represented with cmp sim , as shown in Figure 1(c).
Assume function E represents the modeling tools, function W represents the operations of cutting, pasting, and compensating, as shown in Figure 1. Three kinds of operating modes can be mapped by functions The eight-tuple can be defined by the mapping between the operations of cutting, pasting, and compensating using conventional sculpting methods and the three kinds of operating modes in the virtual environment. The simulated mapping of the cutting operation is shown in formula 2-1, the simulated mapping of the pasting operation is shown in formula 2-2, and the simulated mapping of the compensating operation is shown in formula 2-3. Through the above mapping method, the operations of cutting, pasting, and compensating with conventional manual kneading methods can be mapped to the computerized virtual space.  (2-3) When the progress between manual modeling and virtual prototyping is mapped, the calculation of force feedback should be considered. Only through the appropriate feedback force can VCPS provide operators with real experience just like in manual modeling. When the tools contact the surface of the virtual model in the virtual computerized environment, the feedback force emerges. In the modeling process, a user is able to control the amount of scraping force exerted onto a virtual surface and the amount of material filled or removed, but not the distance of the tool being moved.
Interaction with virtual objects results in plausible animation of the virtual hand since its configuration not only depends on the desired gesture but also contact with the environment.

B. Simulation Environment of VCPS
The virtual system consists of compressed voxel models along the X, Y, Z axis. In simple terms, a voxel model is a point of data, also known as a volumetric pixel, that represents a value in a three dimensional space similar to how a pixel represents image data in a 2D bitmap. This volumetric voxel structure is made up of three dexel models that are connected to form a three dimensional grid. Hence the compressed voxel model represents the virtual space in which the object or virtual prototype can be viewed or manipulated in a dynamic fashion. When the modeling process occurs, new voxel models are generated that discretize dexel models along the X, Y, and Z axis. This means that in a Virtual Prototyping world, the voxel discretization process is parallelized.
In a force feedback simulation system, the refresh rate for each scene and force feedback is separate and independent of each other. Within the virtual space in the graphics scene, the digital model is displayed and the interactive function is achieved by manipulating the modeling tools. The force feedback generated is based on the voxel rate calculated in real-time, and the haptic effect is felt through the force feedback device itself.
The Virtual Clay Prototyping System (VCPS) can be used to perform a variety of tasks within a virtual environment, such as moving, touching, and manipulating rigid or deformable bodies.

III. EXPERIMENTS
Yan [13] proposed that haptics could be improved through the use of force feedback. Going a step further, Liu [14] provided recommendations on how real-time integration of both physical and virtual prototypes could potentially streamline the NPD process. The researchers did separate studies with different methods, but they all agreed that touching was valuable for haptic feedback. So in the section of experiments, based on the previous research, we added some new tasks to further explain things.  Fig. 2) was used and connected through the computer via a parallel port link. The rendering and display of the graphics was supported by use of the OpenInventor Software Development Kit that achieves a 1k Hz high frequency simulation cycle.

A. The Procedure
A total of 20 final year undergraduate students from Shandong University of Technology and Northwestern Polytechnical University were recruited for the experiment involving a product design activity. All of them were aware of haptic devices and had experience in the use of such systems for design projects. The students were divided into four groups: groups A1 and A2 came from Shandong University of Technology and groups B1 and B2 came from Northwestern Polytechnical University.
Each group was given three tasks. The first task was to design an automotive vehicle by means of manual sketching and rendering; the second task was to design an automotive vehicle through use of modeling clay (physical material); and the last task was to design a similar automotive vehicle using virtual prototyping tools. To aid clarity, all students were given a handbook that explained the user interface and the functionality of the virtual prototyping tools.
To ensure that the tasks were consistent throughout the groups, a series of procedures were in put place: (1) introducing the experiment; (2) grouping of students and explanation of the equipment; (3) standardizing design parameters such as the shape and size of the automotive vehicle; (4) duration of activities; (5) tasks the students were expected to undertake: Task 1 -manual sketching and rendering, Task 2 -physical modeling (clay), and Task 3 -virtual prototyping; (6) feedback and conclusion.
At any one time, there were two researchers who acted as observers to ensure the process was smooth. After completion of the three tasks, each student was given a survey form to complete a questionnaire regarding the design exercise and the three tasks and to provide a short statement that would reflect how they felt about the overall design process. The questionnaire can be found in the appendix.

B. Task 1 -Manual Sketching and Rendering
While many forms of design representations are available, sketching is seen as being central during the early stages of NPD to visualize and communicate ideas at an individual level. The aim of the first task was to provide a sketch using manual media such as a pen, pencil, markers and paper. Having a design drawn on paper, the teams were now ready to translate the concept as three dimensional through physical clay modeling (Task 2) and virtual prototyping (Task 3).

C. Task 2 -Physical Modeling
Models are often recognized as a three-dimensional (3D) object that can be created in physical or virtual domains. It is the progression of having more information embedded within physical representation that enhances understanding of the concept, thereby leading to more accurate and effective representations of the idea [1]. As part of Task 2, the aim of physical modeling was to provide a 3D representation of the design conceptualized from Task 1. The objective was to allow the students within the four groups to achieve a physical model using manual methods such as shaping and sculpting by hand.  The process of physical modeling includes several steps: (1) creating a wireframe that defines the size, shape and general features of the vehicle.
(2) using Styrofoam as a soft material to provide volume.
(3) using clay as a material to cover the wireframe and foam, as shown in Fig. 3.
(4) shaping, sculpting, and finishing the clay. (5) Scanning the physical model into computerized data using laser 3D scanner, as shown in Fig. 4.

D. Task 3 -Virtual Prototyping
Virtual prototyping was used to ascertain the effectiveness of the haptic tool based on the proposed algorithm discussed in the previous section. For this task, the students utilized the PHANToM device that allowed them to apply a virtual scraping tool that could sculpt (or scrape) the surface of a virtual object with a controlled amount of force and depth. As part of the operating procedure, all groups started by modeling a cube measuring 500mm and using the sculpting tool to shape the automotive vehicle. Sketch views were scanned and incorporated into the Virtual Clay Prototyping System (VCPS). The basic geometric form was first shaped using cutting, pasting, and blending tools before mirroring symmetrical features to achieve the final form of the automotive vehicle. The processes of all three tasks are shown in Fig. 5. PAPER VIRTUAL CLAY PROTOTYPING SYSTEM -A FRAMEWORK FOR REAL-TIME MODELING To make a car model in the Virtual Clay Prototyping System requires the following steps: (1) Start and initialize setting the VPCS parameters and calibrate the force feedback device.
(2) Build a cube 500 200 150 cm and generate the initial clay model.
(3) Define the modeling tool templates, modify the shape of the modeling templates, import 2D reference lines into the scene. The interface of VCPS and of the templates parameter dialog are shown in Fig. 6.
(4) Sculpt the main body profile to get a rough model.

IV. RESULTS AND DISCUSSION
In previous studies [15], we demonstrated that the virtual methods could increase students' effective haptics and found that 85% of the respondents utilized forcefeedback to gain a better tactile understanding of surfaces.
The results from the data analysis were classified into four categories: (1) operation, whereby the users analyzed the hands-on control of the physical or virtual tools; (2) evaluation, where the members made judgments of their idea, drawing or model; (3) information management, in which the students determined important and insignificant data obtained during the design process; and (4) use of resources, in which the participants evaluated and determined how the material was to be used.
To clarify the results from the study, four independent variables were formulated to give a better understanding of the design process and its outcomes: (1) Physical Modeling Time (PMT): combination of thinking, foam making, clay covering, modeling, smoothing, feasibility analysis and evaluation, as seen in Fig. 8.  Fig. 10.
As can be seen in Figure 10, the four grids represent the four groups' scores (Groups A1, A2, B1, B2) determined by the experts and industrial designers. The VM for Group B2 achieved the smoothest results, while the PM of Group A2 was the worst in the indexes of smoothness. Group A1 had the biggest span in scoring, indicating that the experts and industrial designers were divided over the model, neither PM or VM. Figure 10. The SQ scores of experts and designers

A. Physical Model Spent Time vs. Virtual Model Spent
Time In this section, we compared the two modeling methods of physical and virtual prototyping. We looked at the amount of time the participants took to gather information, the sketching activity, the physical modeling activity, and the virtual prototyping activity (Table 1).
When the students began modeling, the results varied considerably. Although group B1's physical modeling and group B2's virtual prototyping received the same SQ (90), the amount of virtual prototyping activity was only twothirds than that of the physical modeling. Group B2 completed the virtual prototyping task in less than 600 minutes, whereas group B1 completed the physical modeling task by taking longer than 900 minutes.   Thinking  5  7  8  5  Thinking  25  30  21  18   Foam making  16  13  14  12  Parameter  inputting  5  6  5  6   Clay covering  82  96  75  60  Tools defining  80  76  85  70  Template  &  Reference line  45  56  42  52  Template  generation  50  35  50  45   Scraping physical  model  70  96  172  210  Virtual  model  making  150  140  135  175   Refined scraping  model  145  168  246  218  -----Smoothing  80  82  73  74  Smoothing  12  17  22  13  Feasibility  analysis  20  16  30  21  Analyzing  4  3  3  5 Evaluation (Artificial) 10 12 13 15 Evaluation (Intelligent)  5  5  6  6   Scan data  ----Data transition  1  1  1  1  Total  473  546  673  667  Total  332  313  328  339  Solution Quality  77  80  90  86  Solution Quality  75  88  92  90 PAPER VIRTUAL CLAY PROTOTYPING SYSTEM -A FRAMEWORK FOR REAL-TIME MODELING In the physical modeling, all of the groups spent more than ten minutes completing the foam model prototype; it took five to six minutes for the four groups to complete entering the input parameters for the virtual prototyping. An average of 328 minutes was calculated for all groups to complete the virtual model, while they took an average of 590 minutes to complete the physical model. During the evaluation phase, the operator's experience of seeing and touching is crucial for the physical model, while in the virtual modeling environment a specialized plug-in could detect whether the sculpted surface was smooth. Another key advantage of virtual modeling is the conversion of computer data. Data conversion of virtual models to the desired format is usually instantaneous (depending on file size and processor speed), whereas the physical modeling requires the use of 3D scanners and reverse engineering.
As shown in Table 1, under physical modeling, group B2 received an 86 SQ and spent the most time (918 minutes), while the group B1 spent 16 minutes less (902 minutes) but received a 90 SQ (the highest score in the physical model). In the virtual prototyping, both groups A1 and B2 spent nearly equal time (592 minutes and 590 minutes); the virtual modeling activity of group B2 was 90 SQ, while the modeling activity of group A1 was 75 SQ (the lowest score in virtual modeling).
In addition, we also found that the group that had a higher overall total time also received higher SQ scores in physical modeling and virtual prototyping. For instance, group B1 spent 74.61 percent of their time in physical modeling, which is the highest rate in the four groups, and they received the highest score (90 SQ). Interestingly the same group also received the highest score (92 SQ) in virtual prototyping. The 65.14 percent recorded under the category of virtual prototyping was also the highest rating among all four groups.

B. Operation Analysis in VCPS
The operation process of vehicle clay model making is complicated. To achieve a unified visual effect, we usually considered the integral shape during the manual modeling. All the auto parts were made and modified in the meantime instead of being made respectively. Due to such a manual operation habit, it is very hard to time and to quantize the manual operation in the form of auto parts.
While in VCPS, as stated earlier, many operations of virtual prototyping were convenient and easy to be quantized. For instance, a virtual model can be modified directly or through parameterization by the command of smooth or scale. Users could adjust the position, scale, or overall shape when they finished every auto part separately. These characteristic in VCPS provided a good condition to study the complexity of vehicle body modeling.
In the experiments of virtual prototyping, we respectively counted the time that every auto part took in the process of modeling. These data, which represented the spent time of various components, reflected the difficulty degree of the operation. More complex parts and more difficult operations cost a longer time in VCPS. Automotive clay modeling in VCPS is a very complex process; few auto parts can be modeled at once, the templates were defined multiple times, and the virtual model were scraped again and again.
The modeling schedule of each auto part for the four groups is shown in Figure 11. The advantages and disadvantages of VCPS in the vehicle clay modeling process can be determined. The tire production took the shortest in all four groups because the cylinder can be easily defined in the parameterization in VCPS. Simple and large area auto parts, such as the roof, trunk and windshield (front and tail), could be built quickly and accurately. Those parts that have many complex surfaces or details like fenders, mud guards and side walls need to be modified and redefined over and over again; they generally cost a lot of time. The details of the doorknobs, rearview mirror, radiator grille and so on cost much more time than the integral shape. Some groups did not finish the model of these auto parts in the given time.
In addition, because of the differences in the body shapes for the four groups, their time of operation for the hood and bumpers (front and tail) were widely available. These data were not universally illustrative or representative.
Based on the final model of the four groups, virtual prototyping did well in smoothness for large and simple surfaces, but not was good at modeling detail and irregular PAPER VIRTUAL CLAY PROTOTYPING SYSTEM -A FRAMEWORK FOR REAL-TIME MODELING parts. Manual modeling was not significantly different between the details and the overall compared to virtual prototyping. There were large differences in surface smoothness. The results showed that the VCPS had great advantages in helping build a model with a large area surface, but it was not perfect on detail scraping.
The interviews and questionnaires were conducted after the experiments, so the difference between virtual prototyping and manual modeling could be seen in another way.
Despite the fact that virtual prototyping was less preferred in terms of idea generation and habit of operation, virtual prototyping still had the advantage of having access to unlimited resources (virtual shapes) in the digital world once the initial infrastructure (settings and parameters) was in place. Precision and modeling speed also depended on the students' proficiency with the software and familiarity of its interface. The third advantage of virtual prototyping was that the virtual model could be realistically modelled with virtual textures, materials, and lighting that took less than a minute to render, unlike a physical clay model that would require several hours of painting and retouching. Lastly, the use of the virtual prototyping system also allowed online tutorials to be offered, thereby providing instructions on demand. This helped less capable students or those who were fairly new to the interface.

V. CONCLUSION
A virtual prototyping method based on haptic force feedback was proposed, and a Virtual Clay Prototyping System (VCPS) was used in this study. To verify the efficiency of the VCPS, grouping contrast experiments between virtual prototyping and physical modeling were conducted. The experiments explored the benefits and challenges of virtual prototyping and physical modeling. It was found that physical modeling still offered several advantages over a virtual approach. However, the experiment provided evidence that a virtual approach offered the benefit of unlimited digital resources and allowed the students to sculpt shapes and forms faster than that of a physical model, enabling photo-realistic materials, textures and lighting to be rendered for a better immersive experience.