Full Length ArticleAdding vibrotactile feedback to a myoelectric-controlled hand improves performance when online visual feedback is disturbed
Introduction
Grasping and manipulating an object, as simple as it may seem, requires a complicated motor control process, using visual, proprioceptive, and tactile feedback mechanisms (Flanagan, Merritt, & Johansson, 2009). Visual and proprioceptive feedback are used to guide the hand to the chosen position and to determine the correct distance to the object. When the hand is grasping the object, tactile feedback provides information about the shape, texture and size of the object. In this way, a successful performance of grasping or manipulating of an object can be achieved (Flanagan, Bowman, & Johansson, 2006).
In chronic deafferentation, when no tactile feedback is possible, deficits are observed in performance despite full vision of the object (Hermsdörfer, Elias, Cole, Quaney, & Nowak, 2008). Following limb loss, amputees may use myoelectric prosthesis, in which the artificial hand is activated by signals from the muscles of the residual limb. This is done by recording electrical activity (surface EMG) via two electrodes placed on the muscles of the forearm. When the prosthesis user contracts his or her muscles, the electric signal generated by the muscles is detected by the electrodes, and translated into a hand movement, e.g. opening or closing the prosthetic hand (Farina & Amsüss, 2016). Using these state-of-the art devices for functional grasping tasks may be a complex task. Since the prosthesis is controlled by contracting the residual muscles in order to activate different hand movements, this non-natural hand control is not trivial (Fougner, Stavdahl, Kyberd, Losier, & Parker, 2012).
Another difficulty in using myoelectric prostheses arises from the lack of tactile feedback from the artificial hand. Since tactile feedback is missing, the prosthesis user has to compensate using his or her visual feedback resources during performance of functional tasks (Cordella et al., 2016). A study that examined the correlation between functional performance and visual attention found that prosthesis users who were less successful in functional task performance tended to allocate their visual attention to their artificial hand (Bouwsema, Kyberd, Hill, van der Sluis, & Bongers, 2012). In a recently published study, Parr and colleagues analyzed the gaze behavior during the functional task of picking coins. They found that when using the prosthesis simulator, subjects focused significantly more on their hand, and their task performance time was prolonged (Parr, Harrison, Vine, & Wood, 2017). These results suggest that when using a prosthesis, the natural mechanism of eye-hand coordination during task performance is affected, so that visual feedback is used more prominently. Therefore, performance of simple tasks using a myoelectric-controlled hand, which does not provide tactile feedback, can be highly challenging for the common user (Wijk & Carlsson, 2015). This may be even more relevant to users with poor daily performance, where visual feedback is used as the main resource to provide data about the grasped object.
Due to the loss of tactile feedback following a limb loss, when encountering different scenarios in real life, where visual feedback is disturbed, prosthesis users may face a challenge not faced by other individuals. For example, they can be required to perform a dual task, where they need to divide their visual attention between both hands, e.g. holding a coffee cup while reading a paper. Since tactile feedback is missing from the artificial hand, this is not a trivial functional task. In order to evaluate the relationship between tactile feedback and visual attention levels during functional tasks, we have recently examined the effects of adding vibrotactile feedback (VTF) to a myoelectric-controlled hand on visual attention and performance in a dual-task paradigm (Raveh, Friedman, & Portnoy, 2017). In our study, the subjects were required to perform a functional task with the artificial hand, while controlling a video game with the other hand, repeated in two conditions: with and without VTF. However, no significant differences between the two conditions were found. These results could be attributed to the complexity of the dual-task scenario, making it difficult to assess the actual effect of tactile feedback on task performance.
In order to examine this effect from a different perspective, we examined the role of tactile feedback when visual feedback is disturbed. In this scenario, prosthesis users have to utilize their prosthetic hand in a dark environment, where they cannot rely on their visual resources. This may happen when they try to turn the light on in a dark staircase corridor, when they search for their keys in a bag, or in any other case where visual feedback is not available. In these cases, tactile feedback plays an essential role during grasping, and its lack is significant for the prosthesis user (Cordella et al., 2016). A recent study assessed daily prosthesis usage using wireless activity monitors, and found that the prosthesis users barely used the artificial hand during the evening times (Chadwell et al., 2017). This could be explained due to the additive challenge of using a prosthesis in nighttime, when there is less visual feedback, increasing the risk for dropping objects. Although prosthesis users are unlikely to use their artificial hand in the dark, it is important to examine the performance in a surrounding where visual feedback is limited or disturbed, in order to have more knowledge of the actual role of tactile feedback in the grasping process.
Several studies used different methods to disturb visual feedback in order to evaluate the effects of adding tactile feedback to prosthesis users. One method is to cover the objects with dark material, so that the subjects cannot see the hand and its movements. Using this method with healthy subjects fitted with a robotic hand in a grasping task and tactile feedback, researchers showed that performance was improved (Saunders & Vijayakumar, 2011). Another method is to use a blindfold, so that the subject cannot see the object at all. This method was used in a novel case study with implanted tactile feedback, in order to examine if he could identify the stiffness and shape of three different objects by exploiting their different characteristics (Raspopovic et al., 2014).
All the aforementioned studies examined the effects of the addition of tactile feedback on performance time. Nevertheless, improvement in performance does not relate only to reduced performance time, but also to having the task performed in an accurate manner. Both time and accuracy are two important aspects of performance, and their assessment was widely studied (Latash & Fundamentals of Motor Control, 2012). In a review on using sensory feedback in upper limb prosthetics, the possible mechanism in which VTF has positive effects on the grasping of prosthesis users is working through a better control of grip force, and by lowering the number of errors during task performance (Antfolk et al., 2013). In our study, we aimed to examine not only performance time, but also the accuracy during grasping and manipulation, in which tactile feedback plays an important role (Bingham & Mon-Williams, 2013). Since performance of a grasping task is highly challenging with no vision at all, we occluded the vision only during the execution stage, so we could be able to see the actual role of the tactile feedback in the grasping phase. We therefore used a dark room, where visual feedback of the object is available to the subjects from a small LED light only for a very short time period, before the actual grasping of the objects. In that way, the subjects could not rely on the visual feedback during the actual grasping process, and were forced to use the provided tactile feedback. Despite the advances in recent research in this emerging field of myoelectric prostheses, the examination of tactile feedback in a functional task, performed with actual objects when visual feedback is disturbed, has not been fully explored yet. In addition, we used a myoelectric-controlled prosthetic simulator, mounted on the subject’s hand, as presented in several studies (Bouwsema et al., 2012, Parr et al., 2017). Other studies used a virtual hand, presented on a screen and controlled by the myoelectric signals. Since we aimed to examine the addition of VTF when using a hand prosthesis, an actual prosthesis can better simulate the use of a myoelectric prosthesis for grasping. Furthermore, only few studies have used these simulators in combination with tactile feedback, rather they were used as a tool to demonstrate myoelectric control of an artificial hand in able-bodied subjects.
If the addition of tactile feedback is found to have a positive effects on successful performance when visual feedback is lacking, it may be an important addition to the current knowledge in this field, and a step towards future implementation of tactile feedback in artificial hands, which may encourage the amputee to use his or her prosthesis daily for more hours per day, even when visual feedback is reduced, e.g. at night. Furthermore, positive effects of adding tactile feedback may add to the accumulating literature on this topic, so that future developments may consider the transfer of sensory data in both non-invasive and invasive manners, along with the efforts to improve muscle activation of artificial hands. Therefore, the aim of this study was to evaluate the effects of using VTF on performance of a functional test, while using a myoelectric-controlled artificial hand when visual feedback is disturbed. We expected that the addition of VTF would improve performance when performing tasks without online visual feedback.
Section snippets
Participants
We calculated the sample size for the repeated measures design using MorePower software version 6.0.1 (Campbell & Thompson, 2012). A preliminary experiment with 5 subjects showed that the standard deviation of our primary outcome measure, the performance time of the modified Box & Blocks test, was 20% of the mean performance time. If the expected difference in the performance time between the trials with and without feedback is 10%, then the calculated sample size is 15 subjects with power of
Results
The results of the Mann-Whitney U test showed no significant difference between the outcome measures of the subjects who began the trial with the VTF activated compared to the subjects who began with the VTF off. This means that there was no learning effect while repeating the trial.
The time to transfer each block (seconds) for all three conditions is presented in Fig. 3. The number of times the block dropped from the artificial hand during grasping, the number of grips with no block in it, and
Discussion
In our study, we tested the effects of adding VTF to a myoelectric-controlled hand on functional performance. Our main finding is that when visual feedback is disturbed, the addition of VTF improves the performance time of a functional test, and reduces the number of errors during object grasping and manipulation.
In a recent review on different methods for providing sensory feedback to upper limb prosthesis users (Antfolk et al., 2013), VTF is said to enable the user with better task execution
Conclusions
In conclusion, vibrotactile feedback can be an effective addition to myoelectric-controlled artificial hands when visual feedback is disturbed, assisting the user in functional performance. If VTF is provided to current myoelectric-controlled hands as an inherent feature, it may help prosthesis users to improve their functional ability during daily life activities in different environments.
Acknowledgements
The authors would like to acknowledge Efrat Gabay and Or Weinstein from the department of biomedical engineering, Fleischmann faculty of engineering, Tel Aviv University for their help with design and construction of the VTF system. This work was performed in partial fulfillment of the requirements for a PhD degree of Eitan Raveh, Sackler Faculty of Medicine, Tel Aviv University, Israel.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or
References (31)
- et al.
Consistent haptic feedback is required but it is not enough for natural reaching to virtual cylinders
Human Movement Science
(2008) - et al.
Control strategies in object manipulation tasks
Current Opinion in Neurobiology
(2006) - et al.
Sensory feedback in upper limb prosthetics
Expert Review of Medical Devices
(2013) - et al.
The dynamics of sensorimotor calibration in reaching-to-grasp movements
Journal of Neurophysiology
(2013) - et al.
Determining skill level in myoelectric prosthesis use with multiple outcome measures
Journal of Rehabilitation Research and Development
(2012) - et al.
MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis
Behavior Research Methods
(2012) - et al.
Visualisation of upper limb activity using spirals: A new approach to the assessment of daily prosthesis usage
Prosthet Orthot Int.
(2017) - et al.
Prosthetic jamming terminal device. a case study of untethered soft robotics
Soft Robotics 1
(2016) - et al.
Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand
Experimental Brain Research
(2014) - et al.
Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses
IEEE Transactions on Neural Systems and Rehabilitation Engineering
(2015)
Literature review on needs of upper limb prosthesis users
Frontiers in Neuroscience
Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching
Prosthetics and Orthotics International
Sensory feedback add-on for upper-limb prostheses
Prosthetics and Orthotics International
Reflections on the present and future of upper limb prostheses
Expert Review of Medical Devices
Predictive mechanisms and object representations used in object manipulation
Cited by (24)
State of the art of prosthesis simulators for the upper limb: A narrative review
2022, Annals of Physical and Rehabilitation MedicineCitation Excerpt :Three studies reported an overlength value: 12 cm [38], 18 cm [48] and 15 cm distally from the intact hand [40]. The terminal device position for most simulators was axial [2,4,7,8,12,21–25,27,29,32–39,41,43–47,52,53], 3 were palmar [5,6,42,50,51] and 3 were dorsal [40,48,54]. The type of terminal device influenced its position, for example, to make room for electrodes or a shoulder harness for control.
Myoelectric Prosthesis Users Improve Performance Time and Accuracy Using Vibrotactile Feedback When Visual Feedback Is Disturbed
2018, Archives of Physical Medicine and RehabilitationHaptic shared control improves neural efficiency during myoelectric prosthesis use
2023, Scientific ReportsDual-Modality Haptic Feedback Improves Dexterous Task Execution With Virtual EMG-Controlled Gripper
2023, IEEE Transactions on HapticsThe cognitive basis for virtual reality rehabilitation of upper-extremity motor function after neurotraumas
2023, Journal on Multimodal User InterfacesFeetBack–Redirecting touch sensation from a prosthetic hand to the human foot
2022, Frontiers in Neuroscience