Linking cognitive and reaching trajectories via intermittent movement control

https://doi.org/10.1016/j.jmp.2013.06.005Get rights and content

Highlights

  • We recorded arm movements and button press responses to random dot kinematograms.

  • We described how to fit a Wiener diffusion model to intermittent arm movements.

  • We predicted arm movements assuming intermittent access to the decision process.

  • This offers the potential for early access to the decision process.

Abstract

Theories of decision-making have traditionally been constrained by reaction time data. A limitation of reaction time data, particularly for studying the temporal dynamics of cognitive processing, is that they index only the endpoint of the decision making process. Recently, physical reaching trajectories have been used as proxies for underlying mental trajectories through decision space. We suggest that this approach has been oversimplified: while it is possible for the motor control system to access the current state of the evidence accumulation process, this access is intermittent. Instead, we demonstrate how a model of arm movements that assumes intermittent, not continuous, access to the decision process is sufficient to describe the effects of stimulus quality and viewing time in curved reaching movements.

Introduction

A core task for cognitive psychology is to uncover the mental states leading up to overt behaviour. Traditionally, theories about cognitive trajectories have been constrained by data recorded from their end point—the behavioural outcome. A recent series of high-profile publications have proposed that fine-grained and direct information about mental states can be found in the trajectories of reaching movements used to indicate the outcomes of decisions. For example, when participants are asked to choose between faces of different races (Wojnowicz, Ferguson, Dale, & Spivey, 2009), words of different categories (Dale, Kehoe, & Spivey, 2007), or numbers of different magnitudes (Song & Nakayama, 2008a), the trajectories of arm movements towards a response target deflect towards the alternative response target in ways that depend systematically on stimulus properties. Such effects suggest a correspondence between physical and mental trajectories whereby the observed reaching trajectory in a decision task serves as a proxy for the underlying mental trajectory through decision space. This view was originally championed by Spivey, Grosjean, and Knoblich (2005) and is now widely accepted. For example, in their influential review, Song and Nakayama (2009), asserted that “the continuity of reaching movements enables each sample point to be modulated by the real-time progress of internal processes” (p. 360; see also Freeman and Ambady (2009), Schmidt and Seydell (2008) and Song and Nakayama, 2008a, Song and Nakayama, 2008b).

The simplicity of the proposed link between mental and physical trajectories is compelling, and convenient. It promises close-to-continuous information about cognitive processing, which has previously been impossible. However, a simple link between mental and physical trajectories seems unlikely given that, in the motor control literature, many have argued that intermittent control is used in generating movements (Fishbach et al., 2007, Morasso et al., 2010, Morasso et al., 2010), although others dispute this point of view (see Desmurget and Grafton (2000) for a review). We do not aim to resolve this debate here. Rather, our goal is to demonstrate that a simple intermittentmodel of arm movements can account for the pattern of data observed in a perceptual decision task, providing a more nuanced way to link mental and physical trajectories that is plausible from a motor control perspective. Our intermittent model has been motivated in part by the suggestion of Van der Wel, Eder, Mitchel, Walsh, and Rosenbaum (2009) who proposed in their reply to Spivey et al. (2005) that the cognitive “trajectory” through decision space influences reaching movements at discrete time points.

Our model rests upon the widely held assumption that reaching movements are composed of discrete submovements, analogous to the way in which speech is composed of phonemes (Berthier, 1996, Flash and Henis, 1991, Flash and Hochner, 2005, Konczak and Dichgans, 1997, Krebs et al., 1999). Submovements are, by assumption, discrete and ballistic—their amplitude, direction and duration are all determined prior to their onset. This allows us to establish the state of the evidence accumulation process at the onset of a particular submovement.

We exploit the discrete and ballistic properties of reaching submovements using a method inspired by the way that others have exploited the discrete and ballistic nature of eye saccades. For example, in their seminal work, Gold and Shadlen, 2000, Gold and Shadlen, 2003 had monkeys indicate which direction a collection of dots were moving by performing an eye saccade in the same direction. On some trials, they stimulated the frontal eye fields to prematurely evoke a saccade with a known angle and amplitude (established when stimulation occurred in the absence of a perceptual stimulus). Critically, the landing spot of the prematurely evoked saccades varied systematically with stimulus quality and viewing time: the longer the monkey viewed the random dot kinematogram prior to being stimulated, the more the landing spot of the evoked eye saccade was deflected in the direction suggested by the stimulus. In this way, Gold and Shadlen were able to map out the time course of evidence accumulation in a simple perceptual decision task.

We designed a procedure inspired by this technique, using premature arm movements. We had human participants indicate their decisions by reaching towards targets, and we “evoked” premature movements by requiring them to start moving just after stimulus onset. Using this procedure, we show that the movements generated are consistent with predictions about partially-completed processing in a standard cognitive decision theory. This establishes that it is possible to link mental and physical trajectories via a plausible intermittent motor control system, rather than the simple direct mapping that is usually assumed.

Our domain of application is simple perceptual decision-making, which has been studied extensively by both cognitive psychologists (Green and Swets, 1966, Luce, 1986, Ratcliff, 1978, Smith and Vickers, 1988, Usher and McClelland, 2001) and, more recently, neuroscientists (Gold and Shadlen, 2000, Gold and Shadlen, 2003, Hanks et al., 2006, Newsome et al., 1989, Reddi and Carpenter, 2000, Roitman and Shadlen, 2002). Mathematical models of these processes (Brown and Heathcote, 2005, Brown and Heathcote, 2008, Ratcliff and Rouder, 1998, Ratcliff and Smith, 2004) are generally based on the notion that evidence is accumulated until a bound is reached, at which point a decision is made, and the models have traditionally been evaluated by data collected about the endpoint of the process, the response time. Nevertheless, there have been previous attempts to observe the evidence accumulation process before the final decision has been made. The most popular approach uses imperative signals designed to interrupt the decision process and force a premature response (Meyer, Irwin, Osman, & Kounios, 1988). However, as these techniques necessarily change the task by forcing a response, it is not clear whether they actually allow us to observe the accumulation before a final response is produced. For example, Ratcliff (1988, see also (2006)) showed that response signal techniques were unlikely to be able to address crucial questions, such as the ability of partially-completed processing to inform decision-making. Our work establishes that reaching movements might help solve this problem. Rather than requiring interrupted cognitive processing in order to probe the evidence accumulation process, our method naturally generates two types of movements. Some movements are straight to a target; these occur when a decision has been made before movement onset, and are analogous to standard button press responses. Other movements begin before a final decision has been made. The subjects spontaneously produce both types of trials, making it plausible that the same evidence accumulation process is occurring in both cases.

We test whether an intermittent model of arm movements in a perceptual decision making task can describe the data. Our perceptual decision task uses random dot kinematograms, with some decisions indicated by reaching movements to targets on a touch screen, and others by standard button presses. By analysing the directions of premature movements, and also by fitting models to both the movement data and button press reaction time (RT) data, we demonstrate that a model based on intermittent arm movements is sufficient to describe the observed reaching data, and that these findings are compatible with a standard RT model.

Section snippets

Participants

Three right-handed men participated in these experiments. All three were healthy with no known neurological or peripheral disorders, and had normal or corrected-to-normal vision. All gave informed consent according to the policies of the Macquarie University Human Research Ethics Committee.

Stimuli

The stimuli were presented on an LCD touch screen (70cm×39cm,1360×768pixels,60Hz). The stimuli used in this experiment were random-dot kinematograms (Gold & Shadlen, 2000), contained within a circular 5°

Reaction time data

Fig. 2 shows the data and predictions from the model for the reaction time data, from the button-press condition only. Decision accuracy and response speed (Fig. 2(a) and (b)) increased with coherence, as expected, and the model predictions closely match the data. Fig. 2(c) compares the reaction time quantiles (0.1, 0.3, 0.5, 0.7, and 0.9) between the model (circles) and the data (crosses). While there are some discrepancies between the predictions and the data, the quantile plots illustrate

Discussion

Our purpose was to determine whether an intermittent model of arm movements was a plausible way to link movement trajectories and cognitive processing, using decision making with a random dot kinematogram task. Our model rests upon the assumption that reaching movements are composed of discrete submovements. We assumed that a separate process to the evidence accumulation process generates these submovements at one or more times during a decision. We found that a model that generates

Acknowledgments

Thanks are due to J. Vandekerckhove, E.J. Wagenmakers and A. Heathcote for helpful discussions at the early stages of this project. The research was supported by the Australian Research Council to MF(DP0880806) and to SB(DP0878588).

References (54)

  • B. Awasthi et al.

    Processing of low spatial frequency faces at periphery in choice reaching tasks

    Neuropsychologia

    (2011)
  • N.E. Berthier

    Learning to reach: a mathematical model

    Developmental Psychology

    (1996)
  • D.H. Brainard

    The psychophysics toolbox

    Spatial Vision

    (1997)
  • S. Brown et al.

    A ballistic model of choice response time

    Psychological Review

    (2005)
  • S. Brown et al.

    The simplest complete model of choice response time: linear ballistic accumulation

    Cognitive Psychology

    (2008)
  • R. Dale et al.

    Graded motor responses in the time course of categorizing atypical exemplars

    Memory & Cognition

    (2007)
  • M. Desmurget et al.

    Forward modeling allows feedback control for fast reaching movements

    Trends in Cognitive Sciences

    (2000)
  • M. Finkbeiner et al.

    The flexibility of nonconsciously deployed cognitive processes: evidence from masked congruence priming

    PLoS One

    (2011)
  • M. Finkbeiner et al.

    Engaging the motor system with masked orthographic primes: a kinematic analysis

    Visual Cognition

    (2008)
  • A. Fishbach et al.

    Deciding when and how to correct a movement: discrete submovements as a decision making process

    Experimental Brain Research

    (2007)
  • T. Flash et al.

    Arm trajectory modification during reaching towards visual targets

    Journal of Cognitive Neuroscience

    (1991)
  • T. Flash et al.

    Motor primitives in vertebrates and invertebrates

    Current Opinion in Neurobiology

    (2005)
  • T. Flash et al.

    The coordination of arm movements: an experimentally confirmed mathematical model

    Journal of Neuroscience

    (1985)
  • J.B. Freeman et al.

    Motions of the hand expose the partial and parallel activation of stereotypes

    Psychological Science

    (2009)
  • Friedman, J., & Finkbeiner, M. (2011). Selection of arm movements during evidence accumulation. In Advances in...
  • J.I. Gold et al.

    Representation of a perceptual decision in developing oculomotor commands

    Nature

    (2000)
  • J.I. Gold et al.

    The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands

    Journal of Neuroscience

    (2003)
  • D.M. Green et al.

    Signal detection theory and psychophysics

    (1966)
  • T.D. Hanks et al.

    Microstimulation of macaque area LIP affects decision-making in a motion discrimination task

    Nature Neuroscience

    (2006)
  • A. Heathcote

    Fitting Wald and ex-Wald distributions to response time data: an example using functions for the S-PLUS package

    Behavior Research Methods

    (2004)
  • S. Kirkpatrick et al.

    Optimization by simulated annealing

    Science

    (1983)
  • M.C. Klein-Flügge et al.

    Time-dependent changes in human corticospinal excitability reveal value-based competition for action during decision processing

    The Journal of Neuroscience

    (2012)
  • J. Konczak et al.

    The development toward stereotypic arm kinematics during reaching in the first 3 years of life

    Experimental Brain Research

    (1997)
  • H.I. Krebs et al.

    Quantization of continuous arm movements in humans with brain injury

    Proceedings of the National Academy of Sciences of the United States of America

    (1999)
  • D. Lee et al.

    Manual interception of moving targets II, on-line control of overlapping sub-movements

    Experimental Brain Research

    (1997)
  • R.D. Luce

    Response times: their role in inferring elementary mental organization

    (1986)
  • D.E. Meyer et al.

    The dynamics of cognition and action: mental processes inferred from speed-accuracy decomposition

    Psychological Review

    (1988)
  • Cited by (35)

    • Tracking priors and their replacement: Mental dynamics of decision making in the number-line task

      2022, Cognition
      Citation Excerpt :

      Note, however, that our findings cannot arbitrate whether this prior is a part of the decision stage, as we hypothesized, or of the motor-control processes (it was already proposed that an anticipated movement may have an effect within the motor control processes, Marinovic, Poh, de Rugy, & Carroll, 2017). Methodologically, the present study confirms that finger tracking can shed light on the temporal dynamics of cognitive tasks (Dotan et al., 2018; Dotan & Dehaene, 2016; Finkbeiner, Coltheart, & Coltheart, 2014; Freeman, Dale, & Farmer, 2011; Friedman, Brown, & Finkbeiner, 2013; Pinheiro-Chagas et al., 2017; Song & Nakayama, 2008). Even more importantly for the present topic, we propose that the paradigm we introduced here may prove extremely useful to study decision making.

    • Gray matter analysis of MRI images: Introduction to current research practice

      2021, Encyclopedia of Behavioral Neuroscience: Second Edition
    • Parallel and serial processes in number-to-quantity conversion

      2020, Cognition
      Citation Excerpt :

      The trajectory-tracking paradigm is therefore sensitive enough to track the time course of the processing of the two digits with a high temporal resolution – and this time course is incompatible with any model assuming that motor gestures are only programmed once complete information has been obtained. The continuous nature of movement programming, and the ability to capture it with finger tracking, is supported by several previous studies, both with continuous spatial response like here (Dotan, submitted for publication; Pinheiro-Chagas et al., 2017) and with binary responses (Dotan et al., 2018; Erb et al., 2016; Finkbeiner et al., 2008; Finkbeiner et al., 2014; Finkbeiner & Friedman, 2011; Freeman et al., 2011; Marghetis et al., 2014; Santens et al., 2011; Song & Nakayama, 2009, 2008a, 2008b), and was also modeled mathematically (Alonso-Diaz et al., 2018; Friedman et al., 2013; also see a review of the continuous relation between the decision processes and manual movement in Gallivan et al., 2018 and in Dotan et al., 2019). Our findings clearly refute any model that assumes a single decision point for number comprehension and/or movement – in particular the lexical model presented in the Introduction, which postulates that the entire two-digit string is recognized and mapped to a whole-number quantity, and the max model, which postulates that the finger deviates only after both digits have been quantified and merged.

    • Track It to Crack It: Dissecting Processing Stages with Finger Tracking

      2019, Trends in Cognitive Sciences
      Citation Excerpt :

      We can similarly detect within-trial changes in confidence - for example, by finding points with high positive or negative acceleration in the vertical axis, in which speed is affected by confidence but not by the left–right decision [13] (Figure 3B, blue curve). At the motor level, to account for within-trial changes of mind, motor control theories can assume that a trial consists of several movement plans [14,81]. The term change of mind supposes that a trial involves a series of interim decisions, each of which may differ from the previous one.

    View all citing articles on Scopus
    View full text