3.1 Force Feedback Devices

There are two types of haptic feedback systems with force feedback capability that can be considered as wearable devices: exoskeletons and fingertip-mounted devices. Exoskeletons allow users to manipulate virtual objects based on various types of haptic feedback. A glove-type exoskeleton is provided as a simple and practical design for fingers and hands.

The CyberGrasp [21] from Immersion Corporation is a commercially available exoskeleton system. It has four fingers and allows force feedback on different phalanxes with a cable-driven exoskeleton structure attached to the back of the user’s hand. SPIDAR-8 [22] is a two-handed multi-finger haptic interface device. It has eight fingertip attachment devices connected with strings to enable calculation of the 3D position of each fingertip. Haptic force is displayed by controlling the tension of the strings. HapticGEAR also employs a wire tension mechanism to display force, but the user feels force via the tip of a pen-type grip with a backpack-type mechanism to minimize fatigue for the wearer [23]. The Rutgers Master II [24] is a light and compact system composed of four small air cylinders placed inside the palm and attached to the fingertips. Its configuration does not depend on finger size, and its calibration is simplified. The telexistence cockpit system [25] represents the force of gravity by displaying multi-DOF haptic force at the operator’s wrist. Yang et al. proposed a seven-DOF haptic device that can be worn on a human arm. Its operation involves the use of mechanical links/joints and human bones/joints [26]. Kobayashi et al. developed a two-fingered body-grounded haptic device called ExoPhalanx, which provides force to the distal segment of the human operator’s thumb and middle finger and to the basipodite of the middle finger [27]. They provide both tactile and visual feedback using servo motor actuators and tendons attached to each of the user’s fingers.

The primary problem with exoskeletons is the need for user-specific calibration. Considering the strength-mass and power-mass ratios of existing materials and actuators, fingertip devices are an interesting solution compared to exoskeletons [28]. A variety of fingertip-mounted devices have been developed for finger-based interaction [29–31]. For example, Gosselin et al. proposed a wearable haptic interface developed for precise finger interaction within virtual reality applications in large environments [32]. Aoki et al. developed a fingertip-mounted haptic device to provide contact for cutaneous sensation using wires for mixed-reality environments [33]. Minamizawa et al. integrated an underactuated mechanism with one-point kinesthetic feedback from the arm with multipoint tactile feedback [34]. Kawasaki et al. developed a hand haptic interface for use in palpation training in virtual reality environments. The device consists of finger-pad force display devices and a 3D fingertip haptic display device [35]. Chinello et al. developed a small-scale lightweight wearable haptic display that allows fingertip stimulation with a wide range of contact forces [36]. Ando et al. developed a fingernail-mountable tactile display that allows the user to feel various textures when touching smooth objects [37].

The complexity, size, and weight of fingertip-mounted devices is lower than that of exoskeletons, but they still allow most interactions including kinematic and tactile senses. However, only precision grasping can be efficiently simulated because no force is fed back to the phalanxes.

3.2 Vibro-Tactile Feedback Devices

Vibration is widely used as a haptic technology for cutaneous feedback to stimulate the tactile senses. Haptic display based on vibro-tactile feedback is easy to implement and provides robust sensation. Such feedback is a useful tool for guiding users with limited visual cues. Konyo et al. proposed a wearable stimulation device that can produce various distributed stimuli on human skin in response to hand movements using ICPF (Ionic Conducting Polymer gel Film) actuators [38]. Kevin et al. developed a wearable wireless haptic piano instruction system composed of five small vibration motors (one for each finger) inside a glove [39]. Kapur et al. developed a rehabilitation system that employs vibro-tactile feedback for stroke victims [40]. Phong and Chellali proposed two forms of vibro-tactile information known as continuous and frequency-modulation-based vibrotactile (FMBV) feedback [41]. Garcia-Hernandez et al. evaluated subjects’ tactile ability to discriminate small virtual ridge patterns using a portable-wearable tactile device [42]. Ding et al. developed the wearable InterfaceSuit, which enables human motion replication and learning in co-space, and also proposed a human-to-human motion replication methodology with multi-modal feedback mechanisms [43].

The use of vibro-tactile feedback has expanded to many different applications. Haptic illusion displays have been developed in consideration of vibration sensitivity differences by applying carefully designed vibration patterns to cutaneous mechanoreceptors [44–46]. However, tactile sensation of motion from vibrating motors can be masked by kinesthetic movement of the human body in mobile conditions [47].

3.3 Electro-Tactile Feedback Devices

Electro-tactile stimulation evokes tactile sensation on the skin by passing a current through surface electrodes. Although electro-tactile display requires direct contact between the skin and the surface electrode, small flexible electrodes are now commercially available.

Kajimoto et al. developed various electro-tactile display approaches, including a haptic interface with electrotactile-kinesthetic integration for dexterous manipulation [48], an electro-tactile display with a repeated electrode structure for enlarged display [49], and an electro-tactile display with real-time impedance feedback using pulse width modulation [50]. Lee and Starner proposed a design for wearable textile-based electro-tactile display in a wristband that can be integrated with current mobile phones and wearable computers [51]. Tamaki et al. developed PossessedHand, which controls the user’s fingers by applying electrical stimulus to the muscles around the forearm [52].

The sensation delivered through surface electrodes sometimes causes pain to users. In this context, controlling sensation to a comfortable level is challenging in the design of electro-tactile displays [53].

4.1 Creation of Haptic Sensation via Finger Pulp Manipulation

Other studies have investigated the creation of haptic sensation based on finger pulp manipulation to stimulate cutaneous mechanoreceptors by producing slip/shear force on the skin. Lee et al. developed a type of display that pushes pins up to apply pressure to the user’s fingertips [54]. The device was found to be capable of forming shapes and patterns for subjects to identify. Webster et al. proposed a tactile haptic slip display involving the combination of a slip display with a force feedback haptic device [55]. Chen et al. developed a shear/slip feedback device that stimulates different texture sensations by rotating at various speeds [56]. Minamizawa et al. proposed a wearable ungrounded haptic display that creates realistic gravity sensation for virtual objects via fingerpad manipulation [57]. Yokota et al. produced the sensation of softness using an electrostatic device with a slider film that controls spacing between the user’s finger and the device [58]. Bark et al. developed a wearable haptic feedback device that imparts rotational stretch to hairy skin for motion display [59]. Tsukada and Yasumura developed a wearable interface called ActiveBelt that enables users to obtain multiple directional information via the tactile sense [60]. Damian et al. developed a wearable single-actuator haptic device that relays multi-modal haptic information on variables such as grip force and slip speed [61].

Although the skin stretch method is considered to generate more direct sensation than vibro- and electro-feedback because the skin is physically deformed, these devices can display relative haptic sensation (i.e., levels heavier/lighter or rougher/smoother than the default value) based on an ad hoc control strategy. A sophisticated control strategy is needed to display target object parameters such as weight and friction. Kurita et al. explored the challenge of displaying friction and weight illusions based on fingertip deformation [62] when lifting a weight with two fingers. The proposed device has three significant characteristics. The first is its wearability, which ensures that it occupies only a small space. Users pinch the device with the thumb and index finger. The device is fixed to the finger phalanges, and the contact area at the fingertip is manipulated via the sliding of a plate at the part where the user holds it. The second characteristic is the device’s absence of a force sensor in favor of a small built-in camera that allows estimation of manipulation at the fingertip by visualizing the contact area between a transparent plate and the fingertip (Figure 1).

The third characteristic is the device’s capability to display arbitrary weight and friction values using eccentricity control. Eccentricity is used as a quantitative index of deformation at the contact area. Its mathematical relationship with skin deformation can be determined based on known material parameters, normal and tangential forces applied to the contact area, and the friction coefficient [63].

4.2 Deformation of the Contact Area

Contact between an elastic sphere and a rigid plate is called Hertzian contact. Figure 2 shows a schematic image of the contact occurring when a normal (vertical) force $f_g$ and a tangential (lateral) force $f_l$ are applied. Here, relative displacement (i.e., $\delta$ in Figure 2) occurs due to the deformation of the elastic object. The analytic solution of $\delta$ is given by:

$\delta =\frac{3\mu f_g}{16a}\left(\frac{2-v}{G}\right)\left\{1-{(1-\frac{f_l}{\mu f_g})}^{\frac{2}{3}}\right\}\cdots$ (1)

(1)

where the radius of the contact area $a={\left(\frac{3f_{g}R}{2E}\right)}^{\frac{1}{3}},G=\frac{E}{2(1+v)},$ R is the radius of the elastic object, $\mu$ is the friction coefficient of the contact area, and $E,\nu$ are the equivalent Young’s modulus and Poisson’s ratio of the elastic object.

Variables $f_l$ and $\mu$ correspond to the weight of the object and the static friction of the contact surface when the object is held. This implies that haptic illusions in weight and friction sensation can be created by applying the appropriate relative displacement $\delta$, which can be determined by calculating the distance between the contact center at the default position and that observed when the fingertip is deformed. However, it is difficult to measure relative displacement precisely with a low-resolution camera because displacement relating to fingertip deformation is very small. In this study, eccentricity [63,64] was used as an index of elastic object deformation. It is easier to determine eccentricity than to directly measure displacement because its calculation is based on the contact area between an elastic object and a rigid plate.

The contact area can be separated into four parts as shown in Figure 3 based on its center. The deformation of this area along the $y$ axis $S_y$ is calculated as follows:

$S_y=(S_3+S_4)-(S_1+S_2)\cdots$ (2)

(2)

where $S_1,S_2,S_3$ and $S_4$ represent the respective areas of each part. The eccentricity $e_y$ along the $y$ axis is defined as follows:

$e_y=\frac{S_{ty}}{S_t}-\frac{S_{sy}}{S_s}\cdots$ (3)

(3)

where $S_s$ and $S_t$ represent the whole contact area, and $S_{sy}$ and $S_{ty}$ are calculated for the neutral position and that after the application of traction force, respectively. Eccentricity is a dimensionless value normalized by the contact area.

When $\frac{\delta }{a}\ll 1$, the following approximation is obtained:

$e_y\approx \frac{4\delta }{\pi a}\cdots$ (4)

(4)

Eccentricity along the y axis is calculated by substituting Eq. (1) into Eq. (4):

$e_y=K\mu {f_g}^{\frac{1}{3}}\left\{1-{\left(1-\frac{f_l}{\mu f_g}\right)}^{\frac{2}{3}}\right\}\cdots$ (5)

(5)

$K=\frac{1}{\pi }(2-\nu )(1+\nu ){\left(\frac{3}{2E{R}^2}\right)}^{\frac{1}{3}}\cdots$ (6)

(6)

When an object is lifted up, $f_l$ and $\mu$ correspond to its weight and friction. This implies that weight and friction sensation can be represented by measuring and controlling $e_y$ rather than $\delta$. Figure 4 shows the eccentricity calculated using Eq. (6) when the object weight changes from 50 to 500 g in (a), and the friction changes from 0.4 to 1.8 in (b). The influence of weight on eccentricity is relatively large; the difference is observed at the beginning (t<0.1). The influence of friction is smaller than that of weight; the difference appears late (around t=0.3) and becomes larger as the friction coefficient decreases.

4.3 Weight and Friction Illusion Display

A prototype display device was developed to produce haptic illusions of weight and friction via eccentricity control for the contact surface of a human fingertip. The device consists of a transparent acrylic plate, a motor to actuate the plate, and a camera to capture images of the contact surface.

An overview of the device and a schematic image of its operating principle are shown in Figure 1 and Figure 5. The total weight of the device is 210 g. The user inserts the thumb and index finger into the rings attached to the device and pinches it so that the index finger is on the center of the transparent plate. Here, the user’s hand was fixed to the body of the device at the middle phalanx of the thumb and at the middle and proximal phalanges of the index finger using plastic belts. The camera captures images of the contact surface at 60 fps. Here, morphological image processing for opening and smoothing was applied to the images captured, and binarized contact images as shown in Figure 6 were obtained. The contours of the contact images were used to calculate eccentricity.

The actuator controls motor torque to achieve the desired eccentricity. As shown in Figure 7, the desired and measured eccentricity with the target weight and friction were set as $(m,\mu )=(100,1.0),(200,1.0)$, and $(300,1.0)$. Highly favorable eccentricity display performance was confirmed.

The efficacy and usefulness of the prototype were investigated in three evaluation experiments involving weight display and friction display with 10 healthy subjects aged from 22 to 24 years old. The weight-friction illusion device was attached to the dominant hand of the subjects, who were instructed to place the same hand on a table and to relax their torso and non-dominant hand. They were then asked to lift the device to a height of 200 mm in 2 seconds. Eccentricity feedback control was started when the device left the table. After lifting and deposition were complete, the experimenter detached the device from the subject’s hand. The subjects were then asked to lift a test object up and down freely.

In the weight display test, the displayed weights were $m=100,200$, and 300 g. When the target weight is heavier than the weight of the device itself, the actuator moves the plate downward where the subjects pinched; when the target weight is lighter than the device, the actuator moves the plate upward. The texture of the contact surface between the display device and the fingertip was fixed using oil to have a value of approximately $\mu =0.9$. The subjects could change the weight of the test object to 50, 100, 150, 200, 250, 300, 350, and 400g by placing weights on the test object container. The surface texture of the test object was the same as that of the display device. The subjects were asked to select the one they judged to have the same weight as the displayed value. In the friction display test, the displayed friction values were μ=0.6, 0.9, and 1.2. The weight of the display device was fixed at 200 g. The weight of the test object was the same as that of the display device. The subjects could change the friction coefficient of the test object to approximately μ=0.4, 0.6, 0.9, 1.2, and 1.7 by replacing the contact plate. The subjects were asked to select the one they judged to have the same friction as the displayed value. Figure 8 and Figure 9 show the subjects’ answers in the weight and friction display test, respectively. The results of the experiments indicate that the prototype successfully displayed weight and friction illusions to the subjects. However, gaps were also observed between the target sensations displayed by the device and the subjective sensations actually felt by the subjects. Measurement of forces during the task and utilization of data to modulate the desired eccentricity are expected to improve the range of weight and friction values that can be presented to users using more sophisticated models and devices.

5.1 Improvement of Haptic Sensory Capability for Enhanced Motor Performance

Improving haptic and tactile sensory capability helps to enhance motor performance. In this context, a fingertip-wearable device that improves motor performance could be used to assist workers engaged in tasks requiring high-precision manual dexterity. A variety of such devices have been developed in the past.

Mascaro and Asada developed a photoplethysmograph fingernail sensor that measures two-dimensional variations in blood volume beneath the fingernail for estimation of normal force, lateral shear force, longitudinal shear force and bending angle based on readings from the sensor [65]. Provancher and Sylvester revealed that the perceived magnitude of friction rendered by conventional force feedback can be increased with the addition of fingertip skin stretch [66]. Tanaka et al. developed a tactile device called a tactile nail chip that can be mounted on the fingernail and deforms it to change tactile perceptivity [67]. Jeong et al. reported that vibration sensitivity changes as a result of tangential vibration application to the skin surface [68]. Romano et al. developed the SlipGlove, which provides tactile cues associated with slippage between the glove and a contact surface [69].

Some researchers have also reported that Stochastic Resonance (SR) improves tactile and haptic sensitivity in the feet, hands, and fingers [70–73]. Improving tactile sensitivity is known to influence the sensorimotor functions of humans [74,75]. Kurita et al. reported the results of a study on a wearable device called a sensorimotor enhancer, which is considered to improve the tactile sensitivity of fingertips. The proposed device has two important qualities [76]. The first is its utilization of a piezoelectric stack actuator featuring high-frequency vibration generation capacity in a compact body. The second is its design, which promotes dexterity by leaving the palmar region of the fingertips free.

5.2 A Wearable Sensorimotor Enhancer Based on the Stochastic Resonance Effect

Figure 10 shows a fingercap-type wearable sensorimotor enhancer prototype. In the device, a compact lead zirconate titanate (PZT) piezoelectric stack actuator is attached to the fingercap. The actuator is used to generate white-noise vibration, which is transmitted to tactile receptors around the finger pulp.

Four tests were conducted with 11 healthy subjects aged from 24 to 38. The amplitude threshold of vibration perception for each subject was determined prior to the experiment. Each subject was asked to report the perception of vibration at the contact point with the actuator when the signal amplitude changed. The maximum amplitude that the subject could not feel was recorded using a limit-based method. In the subsequent experiments, random order was used to apply conditions of no vibration and five different amplitudes (with 50, 75, 100, 125, and 150% of the perception threshold, denoted as 0.5T, 0.75T, 1.0T, 1.25T, and 1.5T, respectively). The sensorimotor enhancer was attached to the subject’s index finger.

5.2.1 Two-Point Discrimination Test

The subjects were instructed to place the hand with the sensorimotor enhancer attached on a table and close their eyes. The experimenter gently pressed two sharp points of a measuring compass against the palmar surface of each subject’s fingertip as shown in Figure 11. The subjects were asked to report when they felt definite contact from the two points. Two series of ascending and descending distances between the points (i.e., a total of four) were tested for each vibration condition, and the average was recorded as the result of each trial. The distance interval was 0.5 mm. The measured distance in the controlled case was normalized against the mean distance measured in the no-vibration case. Figure 12 shows the results of the experiments. The data show that the mean distances in all five controlled cases were smaller than that in the no-vibration case. The results of post hoc Dunnett testing also revealed significant differences against the no-vibration case for the 0.75T (p<0.01) and 1.0T (p<0.01) cases.

5.2.2 One-Point Touch Test

The subjects were instructed to place the hand with the sensorimotor enhancer attached on a table and close their eyes. The experimenter pressed a monofilament against the palmar surface of each subject’s fingertip until buckling occurred, held it, and then removed it. The subjects were asked to report when they felt contact from the filament. Two series of ascending and descending loads (i.e., a total of four) were tested for each vibration condition, and the average was recorded as the result of each trial. A total of five Semmes-Weinstein monofilaments (Touch-Test Sensory Evaluator; 0.008, 0.02, 0.04, 0.07, and 0.1 g) were used as stimuli. The measured load in the controlled case was normalized against the mean load measured in the no-vibration case. Figure 13 shows the results of the experiment. Smaller values of mean force equate to better tactile sensitivity. These outcomes show that the forces in all five controlled cases were smaller than that in the non-vibration case. The results of post hoc Dunnett testing also revealed significant differences against the non-vibration case for the 0.5T (p<0.01), 0.75T (p<0.01), and 1.0T (p<0.01) cases.

5.2.3 Active Sensory Test – Texture Discrimination

The subjects were asked to touch nine pieces of sandpaper with CAMI grit sizes of #40, #80, #120, #150, #180, #220, #240, #280, and #320 and select the one they judged to have the same texture as the piece on the other side. All the pieces were glued to one side of a plastic board provided to the subjects, who were not allowed to see the sandpaper but were permitted to touch and feel it. Attached to the other side of the board was a test piece of sandpaper with a grit size matching one of the nine sandpaper types. Figure 14 shows the results of the experiment. Higher ratios equate to better tactile sensitivity. As the figure indicates, the mean correct ratios for all the controlled cases tended to be higher than that of the no-vibration case. The results of post hoc Dunnett testing also revealed significant differences against the non-vibration case for the 0.5T (p<0.05) and 1.0T (p<0.05) cases.

5.2.4 Motor Skill Test – Minimal-Force Grasping

The subject was asked to pinch and hold an object with a weight of 140 g for 3 seconds with as little force as possible without allowing it to slip. The force measured in the controlled case was normalized against the mean force measured in the no-vibration case. Figure 15 shows the results of the experiment. Smaller forces equate to better motor performance in terms of pinch grasping. Improvements in motor performance were observed for all the controlled cases. The results of post hoc Dunnett testing also revealed significant differences against the no-vibration case for the 0.75T (p<0.01) and 1.25T (p<0.05) cases.