Compensating for Soft-Tissue Artifact Using the Orientation of Distal Limb Segments During Electromagnetic Motion Capture of the Upper Limb

Motion capture systems, which include optoelectronic and electromagnetic (EM) sensors, have been used extensively to record human movement for applications as diverse as animation, gaming, surgery, and biomechanics research [1–7]. Traditionally, optoelectronic motion capture systems, which involve multiple cameras and passive or active markers placed on the body, have been considered the gold standard [8]. However, EM motion capture systems, which track movement using an electromagnetic field emitted from a stationary transmitter and detected by sensors attached to the body, have also seen significant use in diverse applications [1,9–12]. Unlike optoelectronic systems, EM sensors measure all six rigid-body degrees-of-freedom (DOF) with a single sensor (instead of multiple markers), do not require a line of sight, and are generally less expensive; however, they also have a small range (of the order of one to several meters) and can be affected by ferromagnetic objects [13]. This constellation of characteristics makes them particularly well suited for recording movement of the upper limb.

All motion capture systems that use markers or sensors attached to the skin suffer from soft-tissue artifact (STA). The goal of most motion capture is to record the position and/or orientation of the skeletal structure over time. Because the skin is not rigidly attached to the skeletal structure, the skin—and therefore the markers or sensors attached to the skin—can move relative to the skeletal structure. The difference between the recorded movement of the markers or sensors and the actual movement of the underlying skeletal structure is STA. STA can lead to egregious errors [14–17] and can result from multiple causes. For example, STA can occur because of muscle activation or displaced soft tissue, such as when the deltoid muscle moves a sensor attached to the acromion during overhead reaching movements, or when a sensor placed on the lateral epicondyle is displaced when the elbow is fully flexed. This type of STA can often be reduced by well-chosen sensor placement as suggested by Refs. [18] and [19], especially for electromagnetic sensors, which do not require a line of sight. However, what is more, difficult to control is STA that occurs when most of the skin surrounding a DOF moves considerably less than the underlying skeletal structure. This can occur for DOF along the long axis of a body segment, such as humeral internal-external rotation (HIER) and forearm pronation-supination (FPS) [14,15,20,21]. For example, during humeral internal-external rotation, a sensor placed on the biceps rotates only about two-thirds as much as the humerus [14].

Multiple methods have been developed to compensate for STA using optoelectronic systems. These STA compensation methods include deriving HIER from forearm orientation [14,15] and using optimal weighting [16,19–22], which employ algorithms such as least squares, global optimization, and weighting matrices to correct erroneous data with a numerical model of the arm [16].

Unfortunately, most STA compensation algorithms were developed specifically for optoelectronic motion capture systems and cannot be directly applied to EM motion capture systems because the algorithms take advantage of the individual markers used in optoelectronic systems but not in EM systems. To our knowledge there is only one STA compensation method developed specifically for EM systems: Cao et al. used an optimal weighting method to compensate for STA in HIER [21]. More specifically, this method requires a calibration movement to train a regression equation that relates true shoulder angles (established using forearm orientation) to measured shoulder angles (from the sensor attached to the skin of the upper arm). This regression equation is then employed on subsequent movements to estimate true shoulder angles from the measured shoulder angles. Unlike methods that derive HIER directly from forearm orientation [15,20], the method by Cao et al. produced reliable estimates of HIER even as the elbow approached full extension. However, this method does not compensate for STA in FPS and requires additional data beyond the initial static calibration, so it cannot be applied retroactively to datasets collected without the additional calibrations.

The purpose of this paper is to present and evaluate a method that (1) is developed specifically for EM systems, (2) compensates for STA in HIER and FPS, and (3) does not require additional calibration or data. Although designed for EM systems, this method is based on the method developed by Schmidt et al. for optoelectronic systems [15]. To compensate for STA, calculation of the HIER angle relies on the orientation of the forearm, and calculation of the FPS angle relies on the orientation of the hand.

We first derive the STA compensation algorithms and then describe our evaluation experiment.

The STA compensation method builds on conventional inverse kinematics algorithms (i.e., algorithms that do not compensate for a soft-tissue artifact) for determining global upper-limb motion. The global motion refers to the aggregate rotation of multiple bones; for example, instead of considering the complex articulations of individual carpal bones, we define wrist motion as the rotation of the hand relative to the forearm. These inverse kinematics algorithms are described in detail in Ref. [23]. In summary, the kinematic chain of the upper limb was divided into four segments: thorax, upper arm, distal forearm, and hand. Fixed in each segment was a body-segment coordinate system (BCS) that rotated with the body segment (Fig. 1(a)). Relative rotation between neighboring BCS constituted three joints: the thoracohumeral joint, humeroulnar and radio ulnar joint (grouped as a single joint), and wrist joint. Each of these three joints was defined by a joint coordinate system (JCS) with three rotational DOF, and each JCS was defined by three axes of rotation and the order of rotation about these axes, as listed in Table 1. The elbow and wrist joint definitions follow ISB recommendations [24], but the shoulder joint definition was modified to minimize the effects of gimbal lock (see Discussion). In addition, attached to each body segment was an EM sensor with its own sensor coordinate system (SCS) (Fig. 1(b)). EM motion capture systems record the position and orientation of each SCS relative to the universal frame of the stationary EM transmitter.

The conventional inverse kinematics process described in Ref. [23] takes as input the orientation of each SCS relative to the universal frame (recorded by the EM motion capture system as Euler angles or rotation matrices) and outputs the three joint angles of each JCS. This process involves four steps (Fig. 2): first, the orientation of each SCS relative to the universal frame is converted from Euler angles to a rotation matrix (if the orientation of the SCS is recorded as a rotation matrix, this step can be skipped). Second, assuming the relative orientation between each SCS and its corresponding BCS (i.e., the BCS of the body segment to which the sensor is attached) is constant over time, the relative orientation measured during calibration is used to calculate the orientation of each BCS relative to the universal frame. Third, the orientations of the BCS relative to the universal frame are combined to calculate the orientation of each BCS relative to its neighboring BCS. Fourth, the joint angles of each JCS are extracted from the relative orientation of neighboring BCS.

One of the key assumptions of the conventional inverse kinematics method is that the relationship between a sensor's SCS and the BCS of the body segment to which the sensor is attached is constant over time, allowing one to use the relationship established during calibration at later times during movement (Fig. 2). However, movement of the skin relative to the bone causes movement of the SCS relative to the BCS, invalidating this assumption and creating a false prediction of joint angles, i.e., soft-tissue artifact. DOF that rotate about the longitudinal axis of the limb segment is particularly susceptible to soft-tissue artifact [14,15,17,20,21]. This DOF include humeral internal-external rotation and forearm pronation-supination (⁠$γs$ and $γe$⁠). Here we present detailed instructions for compensating for STA in these two DOF.

The mathematical notation used throughout this paper follows [25]. Specifically, unit vectors describing a frame are $[X̂,Ŷ,Ẑ]$⁠. Trailing subscripts denote the frame to which the unit vectors belong, i.e., $[X̂B,ŶB,ẐB]$ are the unit vectors of frame $B$⁠. Leading superscripts denote the frame in which vectors are expressed, e.g., $[X̂ AB ,Ŷ AB ,Ẑ AB ]$ are the unit vectors of frame $B$⁠, expressed in frame $A$⁠. The rotation matrix describing the orientation of frame $B$ relative to frame $A$ is $RBA$⁠, i.e., $RBA=[X̂ AB ,Ŷ AB ,Ẑ AB ]$⁠. Rotation matrices are functions of time; the rotation matrix at time $t$ is $RBA(t)$⁠, whereas the rotation matrix at calibration is $RBA(0)$⁠.

As discussed in Ref. [21], this strategy of using $ŶB×ŶC$ to calculate the true orientations of $X̂B$ and $ẐB$ becomes unreliable as the elbow approaches full extension or flexion: as $αe$ approaches 0 deg or 180 deg, $ŶB$ and $ŶC$ approach parallel orientation, so their cross-product goes to zero, leading to a singularity in the equations for $ẐB$ (and therefore also $X̂B$⁠). One benefit of our algorithm over that proposed by Ref. [15] is that it allows a nonzero carrying angle (⁠$βe≠0$⁠); this prevents the true angle between $ŶB$ and $ŶC$ from ever being smaller than the carrying angle, which is commonly around 5–15 deg for men and 10–25 deg for women [26], thus avoiding the severest effects of the singularity. Nevertheless, the instability has an effect (though decreasing) even beyond the range of the carrying angle (see Discussion).

where $RŶD,−γw$ is a rotation about $ŶD$ by $−γw$⁠. The definition of $RŶD,−γw$ follows the same form as $RŶB,ρ$ (Eq. (3)), but with $ŶD$ substituted for $ŶB$ and $−γw$ substituted for $ρ$⁠.

where $R(t)CG$ and $R(t)DH$can be obtained as follows.

Similar to the strategy for estimating HIER, this strategy of using a cross-product to estimate FPS becomes unreliable when the axes involved in the cross-product approach a parallel orientation, which occurs as the wrist approaches 90 deg of flexion or extension.

To compensate for STA in both the upper arm and in the forearm, one can combine the two methods under the following two assumptions: (1) the elbow carrying angle (⁠$βe$⁠) and the axial rotation of the wrist (⁠$γw$⁠) are both constant and known, and (2) STA affects only HIER and FPS, i.e., it affects only the estimates of $X̂B$⁠, $ẐB$⁠, $X̂C$⁠, and $ẐC$⁠, not $ŶB$ or $ŶC$⁠. To combine the methods, one can follow the procedure outlined in “Deriving humeral internal-external rotation from forearm orientation” to obtain $R(t)BU$ according to Eq. (11) and the procedure described in “Deriving forearm pronation-supination from hand orientation” to calculate $R(t)CU$ according to Eq. (22).

To implement and evaluate this STA compensation algorithm, we conducted the following experiment.

This study included eight healthy subjects (four male, four female). The subjects were 23.4±4.5 (mean±SD) years old (range 17–33 years), with an average height of 1.78 ± 0.16 m (range 1.52–1.96 m) and weight of 78.9 ± 11.9 kg (range 56–94 kg), resulting in a BMI of 25.4 ± 6.1 (range 20.2–39.1). Of the eight subjects, six were right-handed, one was left-handed, and one was ambidextrous. All subjects reported that they were free from injury or disorder that would affect upper-limb movement. Informed consent was obtained from all subjects following procedures approved by Brigham Young University's Institutional Review Board.

Subjects were seated in an armless chair and instrumented with five EM motion tracking sensors (trakSTAR by Ascension Technology Corp, Shelburne, VT). These sensors can record their position and orientation in all 6DOF with static accuracy of the order of 1 mm and 1 deg [1,27–29]. Each sensor was placed in a small, custom-made plastic holder with a 3.2 cm-by-2.5 cm base to minimize rolling and taped in place in the following locations (Fig. 1(c)): on the sternum approximately 5 cm inferior to the incisura jugularis (sensor E), the dorsal aspect of the upper arm approximately 9 cm proximal to the olecranon (sensor F), the dorsal aspect of the forearm approximately 7 cm proximal to the wrist joint center (sensor G), and the back of the hand straddling the third and fourth metacarpals (sensor H). In addition, to determine the center of rotation of the glenohumeral joint [23], a fifth sensor was taped to the scapula over the acromion, straddling the two legs of the acromial angle, and removed after calibration (sensor I). These locations were chosen to minimize the effect of STA, as explained in [23]. Position and orientation data were recorded at 60 samples/sec.

To calibrate the sensor setup, we used the landmark calibration described in detail in Ref. [23]. This method uses the landmarks suggested in Ref. [24] except for the landmarks on the hand, which were modified for in vivo use. Following this method, landmark locations were marked and their locations were recorded using a stylus instrumented with an EM sensor (also trakSTAR), providing the position of each landmark with respect to both the transmitter and the sensors attached to the subject.

Subjects performed 12 simple tasks involving the upper limb. The first seven tasks explored the range of motion (ROM) in each of the major DOF from the shoulder to the wrist: shoulder adduction-abduction, shoulder flexion–extension, humeral internal–external rotation, elbow extension–flexion, forearm pronation–supination, wrist flexion–extension, and wrist radial–ulnar deviation. For each DOF, subjects started in neutral posture, moved to the limit of the ROM in the direction listed first (e.g., elbow extension), then to the limit of the ROM in the other direction (e.g., elbow flexion), and then back to the neutral position. Full shoulder adduction was considered the same as neutral posture. At the limit of the ROM in each direction, the subject paused briefly. The eighth task (wiggle fingers) was performed so we could determine whether the STA compensation algorithm for estimating FPS was sensitive to small perturbations of sensor H (on the hand) during the movement of the fingers. The last four tasks represented a range of functional activities [30]. Starting in neutral posture, subjects moved their right hand to their left shoulder and paused briefly. From there they moved their hand to the mouth and paused again, then touched the back of their head and paused, and finally moved their hand to their back right pocket. Since the purpose of data collection was to characterize algorithm performance across a variety of movements, and not to establish differences between movements, there was no repetition or randomization of movements.

We have presented three STA compensation algorithms (HIER, FPS, and HIER+FPS). To test the effect of these algorithms, we calculated joint angles using the following progression:

1. Approximation 0 (A0): No STA compensation, inverse kinematics with all 9DOF

2. Approximation 1 (A1): No STA compensation, inverse kinematics with only 7DOF (⁠$βe$ and $γw$ assumed constant)

3. Approximation 2 (A2): STA compensation in HIER only

4. Approximation 3 (A4): STA compensation in FPS only

5. Approximation 4 (A4): STA compensation in HIER and FPS

Estimating elbow carrying angle ($βe$) and wrist axial rotation angle ($γw$): As mentioned above, our STA compensation method requires known, constant angles for $βe$ and $γw$⁠. Although the carrying angle has been shown to vary, its variation is relatively small [21]. These angles could be obtained by direct measurement, but one of our goals was to develop a STA compensation method that did not require extra measurements and could therefore be applied retroactively to data collected without the intention to compensate for STA. Therefore, instead of measuring $βe$ and $γw$ directly, we estimated them from the movement data using a two-step approach. First, the conventional inverse-kinematics algorithm without STA compensation (A0) was used to calculate all nine joint angles, including $βe$ and $γw$⁠, as functions of time. We then calculated mean values of $βe$ and $γw$⁠, averaged across all movements for each subject, resulting in a subject-specific, constant value for each angle. Second, these values were then used in approximations A1–A4.

The actual orientation of the skeletal structure is unknown, so we evaluated the effect of the algorithms (A1–A4) by comparing them to the conventional inverse kinematics algorithm (A0) and each other.

To visualize the effect of various STA compensation approximations, we created a mean joint angle trajectory for each STA compensation approximation by averaging across subjects as follows. After calculating the joint angles of all subjects according to a given STA compensation approximation, we segmented the joint angle trajectories into the individual tasks that were performed, synchronized all subjects' movements at the beginning of each task, and stretched them in time to have the same duration. For each task, subjects' joint angle trajectories were resampled to have the same number of samples (the average number of samples for each task), and joint angles were averaged using the circular mean across all eight subjects at each time point [31]. This process was implemented for all five approximations of all DOF for all eight subjects.

Since no gold standard is available, it is difficult to quantify the degree of instability associated with extreme EFE and WFE angles. Nevertheless, we provide a rough estimate, calculated as follows. During movements involving only EFE, a significant change in HIER angle calculated according to approximation A2 or A4 is likely caused by algorithm instability. In contrast, approximation A0 does not suffer from this instability. Therefore, we estimated the instability in HIER angle as a change in approximation A2 of HIER (relative to approximation A0 of HIER) during movements involving only EFE. Similarly, during movements involving only WFE, significant change in FPS angle calculated according to approximation A3 or A4 is likely caused by algorithm instability, whereas approximation A0 of FPS does not suffer from instability. Therefore, we estimated the instability in FPS as a change in approximation A3 of FPS (relative to approximation A0 of FPS) during movements involving only WFE.

To quantify the effect of the various STA compensation approximations on all DOF (not just HIER and FPS), approximations A1–A4 were compared to the original approximation (A0) in two ways. First, after calculating a subject's joint angles according to a given STA compensation approximation, we calculated the difference between this approximation and A0 as a function of time, computed the mean of the absolute difference in a given DOF and subject by averaging across the duration of each task, and finally averaged the mean differences across all subjects, resulting in a mean absolute difference for each DOF and task. Second, the DOF and task for which the various STA compensation approximations (A1–A4) were statistically different from A0 were determined as follows. After computing a mean joint angle approximation across the duration of each task in a given DOF and subject, we performed for each DOF and task a paired t-test across subjects, compensating for multiple comparisons using a pseudo-Bonferroni correction, with a significance level at 0.001. All means and paired t-tests of angle data were performed using circular statistics to account for the circular nature of angle data [31].

Raw sensor data from individual subjects were converted to joint angles according to each of the five approximations outlined above (Fig. 4).

Overall, the five approximations produced similar joint angles for most tasks (Fig. 5). The angles of two DOF (shoulder flexion–extension and abduction–adduction) showed no change across all five approximations, as expected. In contrast (but also as expected), we found large differences between certain approximations for HIER and FPS, especially for some tasks. These differences can be divided into main effects and secondary effects.

Approximations A2 and A4, which were designed to compensate for STA in HIER, had significant effects on the HIER angle during movements involving substantial HIER (Fig. 6(a)). During HIER, the upper-arm skin rotates less than the humerus, so traditional (noncompensating) inverse-kinematics algorithms (such as A0 and A1) underestimate HIER angles during HIER movements. As expected, A2 and A4 estimated greater excursions in HIER than the other approximations. In full internal rotation, HIER angles calculated according to A2 was 18.1 deg beyond those calculated according to A0. Similarly, in full external rotation, HIER angles calculated according to A2 was 25.2 deg beyond those calculated according to A0. Since none of the approximations affected the DOF proximal to HIER (SFE and SAA), there were no differences in HIER angle between approximations A0, A1, and A3, or between approximations A2 and A4.