GISdevelopment.net --> Application --> Archaeology
Abstract
By using a motion capture system we can measure and record human body motion like ballet, dance or other performing arts. The data obtainedby the system are very accurate, but the volume of the data is very large. E_cient handling and management of the motion captured data are desired.
This paper describes a method of converting motion-captured data into dance notation called Labanotation, which is well known among Western dance communities. Although the description by Labanotation is not necessarily accurate, it is compact and powerful enough to describe the overall body motion of dance and other intangible cultural properties. An example of the result of processing is shown. 3D CG character animation has been produced from converted Labanotation data. The Labanotation data produced from motion capture data may also be used as an index for the archives of motion data.
Introduction
Today, motion capture systems have been used for measuring human body motion in the _eld of CG-animation and movie production. The system can also be applied to recording and archiving intangible cultural properties like dance and other performing arts. We can obtain accurate 3-dimensional position data during body motion. However, it is still an open issue how to generate a compact description of body motion like the one expressed as \raising arms" etc .
The method of dance notation called Labanotation, which was devised by Rudolf Laban in the early 20th century, has been used in recording body motions of ballet and dance[1]. Of course Labanotation is originally paper-based and was not intended to be computerized. Moreover, the notation is not necessarily accurate. However, it is easy for us to roughly grasp the body motion by browsing the score. A role of Labanotation for motion captured data is just the same as the role of the music score for music performance. Multimedia technology has already been introduced for preparing and editing Labanotation score[2, 3], and it was proved useful for choreography and dance education.
This paper describes a method to generate data corresponding to Labanotation from those obtained by motion capture systems.
Motion Capturing
Motion capture is a technique for measuring the motion of a multiply-linked rigid object like human body. Several techniques have been devised, but we use an optical system, inwhich several markers are attached to the appropriate positions on the surface of subject human body and the 3-dimensional positions of markers are algorithmically determined from images captured by several, say 6 to 10, video cameras (Figure 1).
Figure 1: Motion capturing
The position of a marker does not necessarily represent the position of a joint of the human body, which is essential for analyzing and reproducing human body motion. The position of a joint is estimated by a software from the position of several markers around it. We can get this kind of motion data which corresponds to the calculated joint position. We use about 30 markers in measuring a motion, and we get 20 joint positions, which are shown in Figure 2.
Figure 2: Joints of the body model
Labanotation and LND
Figure 5: Format of LND
Generating LND from Motion Data
Experiments
The internal representation LND obtained can be displayed and printed as a Labanota- tion score by the system called LabanEditor[3]. Figure 12 shows a screen image of theLabanEditor when LND was read in and the corresponding Labanotation was displayed. The Labanotation is able not only to edit, display and print Labanotation scores, but also to output a VRML _le corresponding to the LND. Hence, we can reproduce the motion of human body by a 3D CG model.
Figure 13 shows an example of the processing. Figure 13(a) is the motion data used. The result of reproduction of 3D motion by LabanEditor is shown in Figure 13(b). Although only the motions of the left and right arms have been dealt with so far, we can con_rm that the motions of both arms are almost correctly reproduced.
Figure 12: LabanEditor
Figure 13: Processing result
Conclusions
We were able to convert successfully the body motion data into Labanotation and reproduce the motion in 3-dimensional CG. Up to now, however, the method takes into account only the horizontal and vertical motion of a child joint. We are now trying to handle twisting motion about proximal joint.
This system can be used for describing and recording the human body motion of intangible cultural properties. Although recording human body motion by Labanotation is not as accurate as by motion capture systems, the capability of abstraction is an advantage of Labanotation. The data are compact and transferrable, and printed score can be used in dance education. Also, the LND produced by the system can be used as index data for the database of motion captured data. We are now thinking of developing a sophisticated system in which capabilities of both the detailed description by motion capture and the compact description by Labanotation are utilized.
Acknowledgments
This research was partially supported by the grant from Matsushita Audio Visual Education Foundation.
References
Generating labanotation from motion-captured human body motion data
Toshiro Matsumoto, Kozaburo Hachimura
Department of Computer Science
Ritsumeikan University, Kusatsu, Shiga Japan
Minako Nakamura
Tokyo Institute of Technology
Tokyo Japan
Toshiro Matsumoto, Kozaburo Hachimura
Department of Computer Science
Ritsumeikan University, Kusatsu, Shiga Japan
Minako Nakamura
Tokyo Institute of Technology
Tokyo Japan
Abstract
By using a motion capture system we can measure and record human body motion like ballet, dance or other performing arts. The data obtainedby the system are very accurate, but the volume of the data is very large. E_cient handling and management of the motion captured data are desired.
This paper describes a method of converting motion-captured data into dance notation called Labanotation, which is well known among Western dance communities. Although the description by Labanotation is not necessarily accurate, it is compact and powerful enough to describe the overall body motion of dance and other intangible cultural properties. An example of the result of processing is shown. 3D CG character animation has been produced from converted Labanotation data. The Labanotation data produced from motion capture data may also be used as an index for the archives of motion data.
Introduction
Today, motion capture systems have been used for measuring human body motion in the _eld of CG-animation and movie production. The system can also be applied to recording and archiving intangible cultural properties like dance and other performing arts. We can obtain accurate 3-dimensional position data during body motion. However, it is still an open issue how to generate a compact description of body motion like the one expressed as \raising arms" etc .
The method of dance notation called Labanotation, which was devised by Rudolf Laban in the early 20th century, has been used in recording body motions of ballet and dance[1]. Of course Labanotation is originally paper-based and was not intended to be computerized. Moreover, the notation is not necessarily accurate. However, it is easy for us to roughly grasp the body motion by browsing the score. A role of Labanotation for motion captured data is just the same as the role of the music score for music performance. Multimedia technology has already been introduced for preparing and editing Labanotation score[2, 3], and it was proved useful for choreography and dance education.
This paper describes a method to generate data corresponding to Labanotation from those obtained by motion capture systems.
Motion Capturing
Motion capture is a technique for measuring the motion of a multiply-linked rigid object like human body. Several techniques have been devised, but we use an optical system, inwhich several markers are attached to the appropriate positions on the surface of subject human body and the 3-dimensional positions of markers are algorithmically determined from images captured by several, say 6 to 10, video cameras (Figure 1).
Figure 1: Motion capturing
The position of a marker does not necessarily represent the position of a joint of the human body, which is essential for analyzing and reproducing human body motion. The position of a joint is estimated by a software from the position of several markers around it. We can get this kind of motion data which corresponds to the calculated joint position. We use about 30 markers in measuring a motion, and we get 20 joint positions, which are shown in Figure 2.
Figure 2: Joints of the body model
Labanotation and LND
- Labanotation
- Internal representation LND
Figure3 illustrates an example of a Labanotation score, which is extracted from a Labano- tation textbook written by Hutchinson[1]. Labanotation score is drawn in a form of vertical sta_ where each column represents the motion of a part of the body. Figure 4(a) shows the basic arrangement of columns in the sta_. The motion of each body part is expressed by a sequence of geometrical symbols placed in the corresponding column.
Figure 3: Example of Labanotation score[1]
Figure 4: Symbolism of Labanotation
Symbols for motion expression are classi_ed into several categories, but the most fun- damental ones are the direction signs and the rotation signs. Direction signs describe the translation of joints with achievement of a position in space by the joint, or with motion of the joint in that direction. Rotation signs describe various forms of rotational motion.
Direction signs are expressed by polygonal symbols. Figure 4(b) shows the structure of the symbol for direction signs. A single symbol for a direction sign represents three basic items of information about a motion: Direction of horizontal motion is expressed by its shape, vertical motion (levels) by its inside shading, and the duration by its size (i.e. length).
LND (Labanotation Data), which is an internal representation for Labanotation, has been designed with reference to the structure of Labanotation. We use alphanumeric characters to represent basic symbols.
Figure 5 shows the format of LND representation. Lines followed by # are for specifying fundamental parameters of Labanotation, and lines followed by // are comments. The motion of a body part is speci_ed by a line followed by a command direction .
| #speed=80 #rhythm 4/4 #unit_per_line 5 #unit_total 10 #unit1 |
|||||
| //direction | joint | direction | level | start | end |
| direction | l_arm | left | mid | 0.0 | 1.0 |
| direction | r_arm | right | high | 1.5 | 2.5 |
Figure 5: Format of LND
Generating LND from Motion Data
- Outline
- Extraction of motion segments
- Generation of LND In each motion segment a joint will exhibit various motions. The motion of a joint in a motion segment is next detected by analyzing the change of orientation of the joint, and is represented by LND convention. The process consists of two steps: quantization of motion direction and detection of achievement points of the motion.
- Quantization of the motion direction
- Detecting points of achievement for the motion
- Integration of LND
- Quantization of duration
The overall process of generating LND from motion data is shown in Figure 6.
Figure 6: Overall processing
The motion corresponding to each column of the Labanotation is decided based on the combination of a parent joint and a child joint. A parent joint is aproximal joint which is considered to be a reference point for the movementof a distal child joint. For instance, when we focus on Right Arm column of the Labanotation, RShoulder and RWrist would be a parent joint andachild joint respectively. Position of each child joint is represented in reference to the corresponding parent joint.
Coordinates for each joint are so normalized that a vector from LHip to RHip aligns to the positive direction of Z axis. Motion data are smoothed by using a simple running average for the reduction of noise component.
The motion data of an object joint is divided into several segments. This is done by ex- tracting periods in which the body part does not move. Namely, the period in which themagnitude of the speed of a joint islower than a small threshold value is detected and taken as a resting period.
By eliminating the resting periods from the motion data we can segment motion data into several sections which are in motion. Each section extracted this way is calledamotion segment.
An example of the result of extracting motion segments from the motion data of Right Arm is shown in Figure 7. In the _gure a graph of magnitude of speed is presented, and the start and the end of segments are indicated by dots. In this case the threshold for magnitude of speed is set to 10mm/frame.
Figure 7: Extraction of motion segments
As shown in Figure 4(b), in Labanotation, the horizontal direction of movement is repre- sented by 9 orientations (including place) and vertical directions by 3 orientations. There- fore, we divide the space into 27 3-dimensional sectors placing the parent joint at the origin of the space, and decide the sector where the child joint exists. Thus the direction of the child joint is represented by 27 quantized orientations. This process is called a quantization of motion direction.
As shown in Figure 8, if the horizontal distance between a parent joint andachild joint is below a threshold value, child jointissaid to be inplace. Otherwise, the horizontal direction of a child joint is quantized into 8 directions. The direction of vertical motion is quantized into 3 directions i.e., high, middle and low according to the position of a child joint in reference to the parent joint.
Because the quantization of the direction of a child joint is carried out in each frame indepen- dently, a sequence of one direction would last as long as the joint exists in the corresponding sector. We call a sequence of frame data in the same direction a run (Figure 9).
The size, i.e. length, of Labanotation symbol written in the sta_ is determined by the duration of the motion. The symbol may be placed in the corresponding column of the sta_, the start of the symbol being aligned to the start of a run. However, the end of the run does not necessarily correspond to the end of a symbol. We have to determine the appropriate frame corresponding to the achievement of the motion in each run.
We de_ne a key orientation for each Labanotation symbol, and when the direction of the child joint approaches the key orientation in the run we think that the motion has been achieved at this point. Namely, asshown in Figure 9 we extract a frame in which the direction of child joint approaches most closely to the key orientation. We decide that this frame corresponds to the end of the Labanotation symbol.
Figure 8: Quantization of direction
Figure 9: Detection of motion achievement
By the process described in the previous section, every motion segment is represented by its successive LND lines and consequently a series of many small Labanotation symbols will be produced. The LND expression determined so far is tentative and a part of it may be redundant. Therefore, we check this tentative LND in accordance with the customary convention of Labanotaion.
A tentative LND will be converted into well-formed LND by applying rewriting rules, i.e. rules for deleting and unifying LND lines. Figure 10 shows an example of rewriting rules.
| #rule_start# | ||
| L | l_forward | mid |
| L | l_forward | low |
| D | forward | low |
| L | place | low |
| L | right | low |
| L | right | low |
| #rule_end# | ||
Figure 11: Intetration of LND
As shown in Figure 11 some of the LND units which are found to be redundant are deleted and merged into their neighboring units. This process is repeated until the lines of LND that meet the conditions given in the rewriting rules are exhausted.
In the _nal step, the duration of each LND generated by the processing is adjusted. An unit of time in LND is de_ned as time between bars divided by an appropriate positive integer. Thus, the duration of the symbol is adjusted so as to become the multiple of the unit time. This process is called a quantization of duration.
Experiments
The internal representation LND obtained can be displayed and printed as a Labanota- tion score by the system called LabanEditor[3]. Figure 12 shows a screen image of theLabanEditor when LND was read in and the corresponding Labanotation was displayed. The Labanotation is able not only to edit, display and print Labanotation scores, but also to output a VRML _le corresponding to the LND. Hence, we can reproduce the motion of human body by a 3D CG model.
Figure 13 shows an example of the processing. Figure 13(a) is the motion data used. The result of reproduction of 3D motion by LabanEditor is shown in Figure 13(b). Although only the motions of the left and right arms have been dealt with so far, we can con_rm that the motions of both arms are almost correctly reproduced.
Figure 12: LabanEditor
 |
|
| (a) Original motion data | (b) Reproduced motion |
Figure 13: Processing result
Conclusions
We were able to convert successfully the body motion data into Labanotation and reproduce the motion in 3-dimensional CG. Up to now, however, the method takes into account only the horizontal and vertical motion of a child joint. We are now trying to handle twisting motion about proximal joint.
This system can be used for describing and recording the human body motion of intangible cultural properties. Although recording human body motion by Labanotation is not as accurate as by motion capture systems, the capability of abstraction is an advantage of Labanotation. The data are compact and transferrable, and printed score can be used in dance education. Also, the LND produced by the system can be used as index data for the database of motion captured data. We are now thinking of developing a sophisticated system in which capabilities of both the detailed description by motion capture and the compact description by Labanotation are utilized.
Acknowledgments
This research was partially supported by the grant from Matsushita Audio Visual Education Foundation.
References
- A. Hutchinson; Labanotation, Theatre Arts Books, 1977
- M. Nakamura, K. Hachimura; Labanotation and New Technology: Application of Hypermedia to Choreography and Dance Education, Proc. World Dance 2000, pp.131-135, 2000
- Y. Yoshida, Y. Matsuoka and K. Hachimura; Processing of Human Body Movement based on Labanotation { LabanReader and LabanEditor { , IPSJ SIG-CH, Vol.98 , No.51 , pp. 61-68, 1998 (in Japanese)