Elsevier

Human Movement Science

Volume 58, April 2018, Pages 32-40
Human Movement Science

Full Length Article
Adding vibrotactile feedback to a myoelectric-controlled hand improves performance when online visual feedback is disturbed

https://doi.org/10.1016/j.humov.2018.01.008Get rights and content

Highlights

  • Myoelectric-controlled prosthetic hand users have problems grasping without vision.

  • We tested if vibrotactile feedback improves performance when vision is disturbed.

  • The goal is to enable better grasping performance, both in speed and accuracy.

  • We found that vibrotactile feedback was nearly as good as vision for grasping.

  • Vibrotactile feedback can sometimes replace vision, allowing greater flexibility.

Abstract

We investigated whether adding vibrotactile feedback to a myoelectric-controlled hand, when visual feedback is disturbed, can improve performance during a functional test. For this purpose, able-bodied subjects, activating a myoelectric-controlled hand attached to their right hand performed the modified Box & Blocks test, grasping and manipulating wooden blocks over a partition. This was performed in 3 conditions, using a repeated-measures design: in full light, in a dark room where visual feedback was disturbed and no auditory feedback – one time with the addition of tactile feedback provided during object grasping and manipulation, and one time without any tactile feedback. The average time needed to transfer one block was measured, and an infrared camera was used to give information on the number of grasping errors during performance of the test. Our results show that when vibrotactile feedback was provided, performance time was reduced significantly, compared with when no vibrotactile feedback was available. Furthermore, the accuracy of grasping and manipulation was improved, reflected by significantly fewer errors during test performance. In conclusion, adding vibrotactile feedback to a myoelectric-controlled hand has positive effects on functional performance when visual feedback is disturbed. This may have applications to current myoelectric-controlled hands, as adding tactile feedback may help prosthesis users to improve their functional ability during daily life activities in different environments, particularly when limited visual feedback is available or desirable.

Introduction

Grasping and manipulating an object, as simple as it may seem, requires a complicated motor control process, using visual, proprioceptive, and tactile feedback mechanisms (Flanagan, Merritt, & Johansson, 2009). Visual and proprioceptive feedback are used to guide the hand to the chosen position and to determine the correct distance to the object. When the hand is grasping the object, tactile feedback provides information about the shape, texture and size of the object. In this way, a successful performance of grasping or manipulating of an object can be achieved (Flanagan, Bowman, & Johansson, 2006).

In chronic deafferentation, when no tactile feedback is possible, deficits are observed in performance despite full vision of the object (Hermsdörfer, Elias, Cole, Quaney, & Nowak, 2008). Following limb loss, amputees may use myoelectric prosthesis, in which the artificial hand is activated by signals from the muscles of the residual limb. This is done by recording electrical activity (surface EMG) via two electrodes placed on the muscles of the forearm. When the prosthesis user contracts his or her muscles, the electric signal generated by the muscles is detected by the electrodes, and translated into a hand movement, e.g. opening or closing the prosthetic hand (Farina & Amsüss, 2016). Using these state-of-the art devices for functional grasping tasks may be a complex task. Since the prosthesis is controlled by contracting the residual muscles in order to activate different hand movements, this non-natural hand control is not trivial (Fougner, Stavdahl, Kyberd, Losier, & Parker, 2012).

Another difficulty in using myoelectric prostheses arises from the lack of tactile feedback from the artificial hand. Since tactile feedback is missing, the prosthesis user has to compensate using his or her visual feedback resources during performance of functional tasks (Cordella et al., 2016). A study that examined the correlation between functional performance and visual attention found that prosthesis users who were less successful in functional task performance tended to allocate their visual attention to their artificial hand (Bouwsema, Kyberd, Hill, van der Sluis, & Bongers, 2012). In a recently published study, Parr and colleagues analyzed the gaze behavior during the functional task of picking coins. They found that when using the prosthesis simulator, subjects focused significantly more on their hand, and their task performance time was prolonged (Parr, Harrison, Vine, & Wood, 2017). These results suggest that when using a prosthesis, the natural mechanism of eye-hand coordination during task performance is affected, so that visual feedback is used more prominently. Therefore, performance of simple tasks using a myoelectric-controlled hand, which does not provide tactile feedback, can be highly challenging for the common user (Wijk & Carlsson, 2015). This may be even more relevant to users with poor daily performance, where visual feedback is used as the main resource to provide data about the grasped object.

Due to the loss of tactile feedback following a limb loss, when encountering different scenarios in real life, where visual feedback is disturbed, prosthesis users may face a challenge not faced by other individuals. For example, they can be required to perform a dual task, where they need to divide their visual attention between both hands, e.g. holding a coffee cup while reading a paper. Since tactile feedback is missing from the artificial hand, this is not a trivial functional task. In order to evaluate the relationship between tactile feedback and visual attention levels during functional tasks, we have recently examined the effects of adding vibrotactile feedback (VTF) to a myoelectric-controlled hand on visual attention and performance in a dual-task paradigm (Raveh, Friedman, & Portnoy, 2017). In our study, the subjects were required to perform a functional task with the artificial hand, while controlling a video game with the other hand, repeated in two conditions: with and without VTF. However, no significant differences between the two conditions were found. These results could be attributed to the complexity of the dual-task scenario, making it difficult to assess the actual effect of tactile feedback on task performance.

In order to examine this effect from a different perspective, we examined the role of tactile feedback when visual feedback is disturbed. In this scenario, prosthesis users have to utilize their prosthetic hand in a dark environment, where they cannot rely on their visual resources. This may happen when they try to turn the light on in a dark staircase corridor, when they search for their keys in a bag, or in any other case where visual feedback is not available. In these cases, tactile feedback plays an essential role during grasping, and its lack is significant for the prosthesis user (Cordella et al., 2016). A recent study assessed daily prosthesis usage using wireless activity monitors, and found that the prosthesis users barely used the artificial hand during the evening times (Chadwell et al., 2017). This could be explained due to the additive challenge of using a prosthesis in nighttime, when there is less visual feedback, increasing the risk for dropping objects. Although prosthesis users are unlikely to use their artificial hand in the dark, it is important to examine the performance in a surrounding where visual feedback is limited or disturbed, in order to have more knowledge of the actual role of tactile feedback in the grasping process.

Several studies used different methods to disturb visual feedback in order to evaluate the effects of adding tactile feedback to prosthesis users. One method is to cover the objects with dark material, so that the subjects cannot see the hand and its movements. Using this method with healthy subjects fitted with a robotic hand in a grasping task and tactile feedback, researchers showed that performance was improved (Saunders & Vijayakumar, 2011). Another method is to use a blindfold, so that the subject cannot see the object at all. This method was used in a novel case study with implanted tactile feedback, in order to examine if he could identify the stiffness and shape of three different objects by exploiting their different characteristics (Raspopovic et al., 2014).

All the aforementioned studies examined the effects of the addition of tactile feedback on performance time. Nevertheless, improvement in performance does not relate only to reduced performance time, but also to having the task performed in an accurate manner. Both time and accuracy are two important aspects of performance, and their assessment was widely studied (Latash & Fundamentals of Motor Control, 2012). In a review on using sensory feedback in upper limb prosthetics, the possible mechanism in which VTF has positive effects on the grasping of prosthesis users is working through a better control of grip force, and by lowering the number of errors during task performance (Antfolk et al., 2013). In our study, we aimed to examine not only performance time, but also the accuracy during grasping and manipulation, in which tactile feedback plays an important role (Bingham & Mon-Williams, 2013). Since performance of a grasping task is highly challenging with no vision at all, we occluded the vision only during the execution stage, so we could be able to see the actual role of the tactile feedback in the grasping phase. We therefore used a dark room, where visual feedback of the object is available to the subjects from a small LED light only for a very short time period, before the actual grasping of the objects. In that way, the subjects could not rely on the visual feedback during the actual grasping process, and were forced to use the provided tactile feedback. Despite the advances in recent research in this emerging field of myoelectric prostheses, the examination of tactile feedback in a functional task, performed with actual objects when visual feedback is disturbed, has not been fully explored yet. In addition, we used a myoelectric-controlled prosthetic simulator, mounted on the subject’s hand, as presented in several studies (Bouwsema et al., 2012, Parr et al., 2017). Other studies used a virtual hand, presented on a screen and controlled by the myoelectric signals. Since we aimed to examine the addition of VTF when using a hand prosthesis, an actual prosthesis can better simulate the use of a myoelectric prosthesis for grasping. Furthermore, only few studies have used these simulators in combination with tactile feedback, rather they were used as a tool to demonstrate myoelectric control of an artificial hand in able-bodied subjects.

If the addition of tactile feedback is found to have a positive effects on successful performance when visual feedback is lacking, it may be an important addition to the current knowledge in this field, and a step towards future implementation of tactile feedback in artificial hands, which may encourage the amputee to use his or her prosthesis daily for more hours per day, even when visual feedback is reduced, e.g. at night. Furthermore, positive effects of adding tactile feedback may add to the accumulating literature on this topic, so that future developments may consider the transfer of sensory data in both non-invasive and invasive manners, along with the efforts to improve muscle activation of artificial hands. Therefore, the aim of this study was to evaluate the effects of using VTF on performance of a functional test, while using a myoelectric-controlled artificial hand when visual feedback is disturbed. We expected that the addition of VTF would improve performance when performing tasks without online visual feedback.

Section snippets

Participants

We calculated the sample size for the repeated measures design using MorePower software version 6.0.1 (Campbell & Thompson, 2012). A preliminary experiment with 5 subjects showed that the standard deviation of our primary outcome measure, the performance time of the modified Box & Blocks test, was 20% of the mean performance time. If the expected difference in the performance time between the trials with and without feedback is 10%, then the calculated sample size is 15 subjects with power of

Results

The results of the Mann-Whitney U test showed no significant difference between the outcome measures of the subjects who began the trial with the VTF activated compared to the subjects who began with the VTF off. This means that there was no learning effect while repeating the trial.

The time to transfer each block (seconds) for all three conditions is presented in Fig. 3. The number of times the block dropped from the artificial hand during grasping, the number of grips with no block in it, and

Discussion

In our study, we tested the effects of adding VTF to a myoelectric-controlled hand on functional performance. Our main finding is that when visual feedback is disturbed, the addition of VTF improves the performance time of a functional test, and reduces the number of errors during object grasping and manipulation.

In a recent review on different methods for providing sensory feedback to upper limb prosthesis users (Antfolk et al., 2013), VTF is said to enable the user with better task execution

Conclusions

In conclusion, vibrotactile feedback can be an effective addition to myoelectric-controlled artificial hands when visual feedback is disturbed, assisting the user in functional performance. If VTF is provided to current myoelectric-controlled hands as an inherent feature, it may help prosthesis users to improve their functional ability during daily life activities in different environments.

Acknowledgements

The authors would like to acknowledge Efrat Gabay and Or Weinstein from the department of biomedical engineering, Fleischmann faculty of engineering, Tel Aviv University for their help with design and construction of the VTF system. This work was performed in partial fulfillment of the requirements for a PhD degree of Eitan Raveh, Sackler Faculty of Medicine, Tel Aviv University, Israel.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or

References (31)

  • R.H. Cuijpers et al.

    Consistent haptic feedback is required but it is not enough for natural reaching to virtual cylinders

    Human Movement Science

    (2008)
  • J.R. Flanagan et al.

    Control strategies in object manipulation tasks

    Current Opinion in Neurobiology

    (2006)
  • C. Antfolk et al.

    Sensory feedback in upper limb prosthetics

    Expert Review of Medical Devices

    (2013)
  • G.P. Bingham et al.

    The dynamics of sensorimotor calibration in reaching-to-grasp movements

    Journal of Neurophysiology

    (2013)
  • H. Bouwsema et al.

    Determining skill level in myoelectric prosthesis use with multiple outcome measures

    Journal of Rehabilitation Research and Development

    (2012)
  • J.I.D. Campbell et al.

    MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis

    Behavior Research Methods

    (2012)
  • A. Chadwell et al.

    Visualisation of upper limb activity using spirals: A new approach to the assessment of daily prosthesis usage

    Prosthet Orthot Int.

    (2017)
  • N. Cheng et al.

    Prosthetic jamming terminal device. a case study of untethered soft robotics

    Soft Robotics 1

    (2016)
  • C. Cipriani et al.

    Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand

    Experimental Brain Research

    (2014)
  • F. Clemente et al.

    Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses

    IEEE Transactions on Neural Systems and Rehabilitation Engineering

    (2015)
  • F. Cordella et al.

    Literature review on needs of upper limb prosthesis users

    Frontiers in Neuroscience

    (2016)
  • A.L. Edwards et al.

    Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching

    Prosthetics and Orthotics International

    (2016)
  • N. Fallahian et al.

    Sensory feedback add-on for upper-limb prostheses

    Prosthetics and Orthotics International

    (2017)
  • D. Farina et al.

    Reflections on the present and future of upper limb prostheses

    Expert Review of Medical Devices

    (2016)
  • J.R. Flanagan et al.

    Predictive mechanisms and object representations used in object manipulation

  • Cited by (24)

    • State of the art of prosthesis simulators for the upper limb: A narrative review

      2022, Annals of Physical and Rehabilitation Medicine
      Citation Excerpt :

      Three studies reported an overlength value: 12 cm [38], 18 cm [48] and 15 cm distally from the intact hand [40]. The terminal device position for most simulators was axial [2,4,7,8,12,21–25,27,29,32–39,41,43–47,52,53], 3 were palmar [5,6,42,50,51] and 3 were dorsal [40,48,54]. The type of terminal device influenced its position, for example, to make room for electrodes or a shoulder harness for control.

    View all citing articles on Scopus
    View full text