Krishna Shenoy

All Publications

• Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nature neuroscience Paulk, A. C., Kfir, Y., Khanna, A. R., Mustroph, M. L., Trautmann, E. M., Soper, D. J., Stavisky, S. D., Welkenhuysen, M., Dutta, B., Shenoy, K. V., Hochberg, L. R., Richardson, R. M., Williams, Z. M., Cash, S. S. 1800

  Abstract

  Recent advances in multi-electrode array technology have made it possible to monitor large neuronal ensembles at cellular resolution in animal models. In humans, however, current approaches restrict recordings to a few neurons per penetrating electrode or combine the signals of thousands of neurons in local field potential (LFP) recordings. Here we describe a new probe variant and set of techniques that enable simultaneous recording from over 200 well-isolated cortical single units in human participants during intraoperative neurosurgical procedures using silicon Neuropixels probes. We characterized a diversity of extracellular waveforms with eight separable single-unit classes, with differing firing rates, locations along the length of the electrode array, waveform spatial spread and modulation by LFP events such as inter-ictal discharges and burst suppression. Although some challenges remain in creating a turnkey recording system, high-density silicon arrays provide a path for studying human-specific cognitive processes and their dysfunction at unprecedented spatiotemporal resolution.

  View details for DOI 10.1038/s41593-021-00997-0

  View details for PubMedID 35102333

• Cortical preparatory activity indexes learned motor memories. Nature Sun, X., O'Shea, D. J., Golub, M. D., Trautmann, E. M., Vyas, S., Ryu, S. I., Shenoy, K. V. 1800

  Abstract

  The brain's remarkable ability to learn and execute various motor behaviours harnesses the capacity of neural populations to generate a variety of activity patterns. Here we explore systematic changes in preparatory activity in motor cortex that accompany motor learning. We trained rhesus monkeys to learn an arm-reaching task1 in a curl force field that elicited new muscle forces for some, but not all, movement directions2,3. We found that in a neural subspace predictive of hand forces, changes in preparatory activity tracked the learned behavioural modifications and reassociated4 existing activity patterns with updated movements. Along a neural population dimension orthogonal to the force-predictive subspace, we discovered that preparatory activity shifted uniformly for all movement directions, including those unaltered by learning. During a washout period when the curl field was removed, preparatory activity gradually reverted in the force-predictive subspace, but the uniform shift persisted. These persistent preparatory activity patterns may retain a motor memory of the learned field5,6 and support accelerated relearning of the same curl field. When a set of distinct curl fields was learned in sequence, we observed a corresponding set of field-specific uniform shifts which separated the associated motor memories in the neural state space7-9. The precise geometry of these uniform shifts in preparatory activity could serve to index motor memories, facilitating the acquisition, retention and retrieval of a broad motor repertoire.

  View details for DOI 10.1038/s41586-021-04329-x

  View details for PubMedID 35082444

• Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex IEEE TRANSACTIONS ON HAPTICS Deo, D. R., Rezaii, P., Hochberg, L. R., Okamura, A. M., Shenoy, K., Henderson, J. M. 2021; 14 (4): 762-775

  Abstract

  Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.

  View details for DOI 10.1109/TOH.2021.3072615

  View details for Web of Science ID 000731146900007

  View details for PubMedID 33844633

• Non-linear dimensionality reduction on extracellular waveforms reveals cell type diversity in premotor cortex. eLife Lee, E. K., Balasubramanian, H., Tsolias, A., Anakwe, S. U., Medalla, M., Shenoy, K. V., Chandrasekaran, C. 2021; 10

  Abstract

  Cortical circuits are thought to contain a large number of cell types that coordinate to produce behavior. Current in vivo methods rely on clustering of specified features of extracellular waveforms to identify putative cell types, but these capture only a small amount of variation. Here, we develop a new method (WaveMAP) that combines non-linear dimensionality reduction with graph clustering to identify putative cell types. We apply WaveMAP to extracellular waveforms recorded from dorsal premotor cortex of macaque monkeys performing a decision-making task. Using WaveMAP, we robustly establish eight waveform clusters and show that these clusters recapitulate previously identified narrow- and broad-spiking types while revealing previously unknown diversity within these subtypes. The eight clusters exhibited distinct laminar distributions, characteristic firing rate patterns, and decision-related dynamics. Such insights were weaker when using featurebased approaches. WaveMAP therefore provides a more nuanced understanding of the dynamics of cell types in cortical circuits.

  View details for DOI 10.7554/eLife.67490

  View details for PubMedID 34355695

• Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Simeral, J. D., Hosman, T., Saab, J., Flesher, S. N., Vilela, M., Franco, B., Kelemen, J. N., Brandman, D. M., Ciancibello, J. G., Rezaii, P. G., Eskandar, E. N., Rosler, D. M., Shenoy, K. V., Henderson, J. M., Nurmikko, A. V., Hochberg, L. R. 2021; 68 (7): 2313-2325

  Abstract

  Individuals with neurological disease or injury such as amyotrophic lateral sclerosis, spinal cord injury or stroke may become tetraplegic, unable to speak or even locked-in. For people with these conditions, current assistive technologies are often ineffective. Brain-computer interfaces are being developed to enhance independence and restore communication in the absence of physical movement. Over the past decade, individuals with tetraplegia have achieved rapid on-screen typing and point-and-click control of tablet apps using intracortical brain-computer interfaces (iBCIs) that decode intended arm and hand movements from neural signals recorded by implanted microelectrode arrays. However, cables used to convey neural signals from the brain tether participants to amplifiers and decoding computers and require expert oversight, severely limiting when and where iBCIs could be available for use. Here, we demonstrate the first human use of a wireless broadband iBCI.Based on a prototype system previously used in pre-clinical research, we replaced the external cables of a 192-electrode iBCI with wireless transmitters and achieved high-resolution recording and decoding of broadband field potentials and spiking activity from people with paralysis. Two participants in an ongoing pilot clinical trial completed on-screen item selection tasks to assess iBCI-enabled cursor control.Communication bitrates were equivalent between cabled and wireless configurations. Participants also used the wireless iBCI to control a standard commercial tablet computer to browse the web and use several mobile applications. Within-day comparison of cabled and wireless interfaces evaluated bit error rate, packet loss, and the recovery of spike rates and spike waveforms from the recorded neural signals. In a representative use case, the wireless system recorded intracortical signals from two arrays in one participant continuously through a 24-hour period at home.Wireless multi-electrode recording of broadband neural signals over extended periods introduces a valuable tool for human neuroscience research and is an important step toward practical deployment of iBCI technology for independent use by individuals with paralysis. On-demand access to high-performance iBCI technology in the home promises to enhance independence and restore communication and mobility for individuals with severe motor impairment.

  View details for DOI 10.1109/TBME.2021.3069119

  View details for Web of Science ID 000663531500027

  View details for PubMedID 33784612

  View details for PubMedCentralID PMC8218873

• High-performance brain-to-text communication via handwriting. Nature Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2021; 593 (7858): 249–54

  Abstract

  Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. So far, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping1-5 or point-and-click typing with a computer cursor6,7. However, rapid sequences of highly dexterous behaviours, such as handwriting or touch typing, might enable faster rates of communication. Here we developed an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time, using a recurrent neural network decoding approach. With this BCI, our study participant, whose hand was paralysed from spinal cord injury, achieved typing speeds of 90characters per minute with 94.1% raw accuracy online, and greater than 99% accuracy offline with a general-purpose autocorrect. To our knowledge, these typing speeds exceed those reported for any other BCI, and are comparable to typical smartphone typing speeds of individuals in the age group of our participant (115characters per minute)8. Finally, theoretical considerations explain why temporally complex movements, such as handwriting, may be fundamentally easier to decode than point-to-point movements. Our results open a new approach for BCIs and demonstrate the feasibility of accurately decoding rapid, dexterous movements years after paralysis.

  View details for DOI 10.1038/s41586-021-03506-2

  View details for PubMedID 33981047

• The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia. eNeuro Rastogi, A., Willett, F. R., Abreu, J., Crowder, D. C., Murphy, B. A., Memberg, W. D., Vargas-Irwin, C. E., Miller, J. P., Sweet, J., Walter, B. L., Rezaii, P. G., Stavisky, S. D., Hochberg, L. R., Shenoy, K. V., Henderson, J. M., Kirsch, R. F., Ajiboye, A. B. 2021

  Abstract

  Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.Significance Statement Intracortical brain-computer interfaces (iBCIs) have emerged as a promising technology to potentially restore hand grasping and object interaction in people with tetraplegia. This study is among the first to quantify the degree to which hand grasp affects force-related, or kinetic, neural activity and decoding performance in individuals with tetraplegia. The study results enhance our overall understanding of how the brain encodes kinetic parameters across varying kinematic behaviors, and in particular, the degree to which these parameters have independent versus interacting neural representations. Such investigations are a critical step to incorporating force control into human-operated iBCI systems, which would move the technology toward restoring more functional and naturalistic tasks.

  View details for DOI 10.1523/ENEURO.0231-20.2020

  View details for PubMedID 33495242

• Decoding and perturbing decision states in real time. Nature Peixoto, D., Verhein, J. R., Kiani, R., Kao, J. C., Nuyujukian, P., Chandrasekaran, C., Brown, J., Fong, S., Ryu, S. I., Shenoy, K. V., Newsome, W. T. 2021

  Abstract

  In dynamic environments, subjects often integrate multiple samples of a signal and combine them to reach a categorical judgment1. The process of deliberation can be described by a time-varying decision variable (DV), decoded from neural population activity, that predicts a subject's upcoming decision2. Within single trials, however, there are large moment-to-moment fluctuations in the DV, the behavioural significance of which is unclear. Here, using real-time, neural feedback control of stimulus duration, we show that within-trial DV fluctuations, decoded from motor cortex, are tightly linked to decision statein macaques, predicting behavioural choices substantially better than the condition-averaged DV or the visual stimulus alone. Furthermore, robust changes in DV sign have the statistical regularities expected from behavioural studies of changes of mind3. Probing the decision process on single trials with weak stimulus pulses, we find evidence for time-varying absorbing decision bounds, enabling us to distinguish between specific models of decision making.

  View details for DOI 10.1038/s41586-020-03181-9

  View details for PubMedID 33473215

• Dendritic calcium signals in rhesus macaque motor cortex drive an optical brain-computer interface. Nature communications Trautmann, E. M., O'Shea, D. J., Sun, X., Marshel, J. H., Crow, A., Hsueh, B., Vesuna, S., Cofer, L., Bohner, G., Allen, W., Kauvar, I., Quirin, S., MacDougall, M., Chen, Y., Whitmire, M. P., Ramakrishnan, C., Sahani, M., Seidemann, E., Ryu, S. I., Deisseroth, K., Shenoy, K. V. 2021; 12 (1): 3689

  Abstract

  Calcium imaging is a powerful tool for recording from large populations of neurons in vivo. Imaging in rhesus macaque motor cortex can enable the discovery of fundamental principles of motor cortical function and can inform the design of next generation brain-computer interfaces (BCIs). Surface two-photon imaging, however, cannot presently access somatic calcium signals of neurons from all layers of macaque motor cortex due to photon scattering. Here, we demonstrate an implant and imaging system capable of chronic, motion-stabilized two-photon imaging of neuronal calcium signals from macaques engaged in a motor task. By imaging apical dendrites, we achieved optical access to large populations of deep and superficial cortical neurons across dorsal premotor (PMd) and gyral primary motor (M1) cortices. Dendritic signals from individual neurons displayed tuning for different directions of arm movement. Combining several technical advances, we developed an optical BCI (oBCI) driven by these dendritic signalswhich successfully decoded movement direction online. By fusing two-photon functional imaging with CLARITY volumetric imaging, we verified that many imaged dendrites which contributed to oBCI decoding originated from layer 5 output neurons, including a putative Betz cell. This approach establishes newopportunities for studying motor control and designing BCIsvia two photon imaging.

  View details for DOI 10.1038/s41467-021-23884-5

  View details for PubMedID 34140486

• Measurement, manipulation and modeling of brain-wide neural population dynamics. Nature communications Shenoy, K. V., Kao, J. C. 2021; 12 (1): 633

  View details for DOI 10.1038/s41467-020-20371-1

  View details for PubMedID 33504773

• Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus. Journal of neural engineering Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., Shenoy, K. V. 2020; 17 (6): 066007

  Abstract

  OBJECTIVE: To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak.APPROACH: Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times.MAIN RESULTS: A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio.SIGNIFICANCE: The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

  View details for DOI 10.1088/1741-2552/abbfef

  View details for PubMedID 33236720

• Structure in Neural Activity during Observed and Executed Movements Is Shared at the Neural Population Level, Not in Single Neurons. Cell reports Jiang, X., Saggar, H., Ryu, S. I., Shenoy, K. V., Kao, J. C. 2020; 32 (6): 108006

  Abstract

  In multiple cortical areas, including the motor cortex, neurons have similar firing rate statistics whether we observe or execute movements. These "congruent" neurons are hypothesized to support action understanding by participating in a neural circuit consistently activated in both observed and executed movements. We examined this hypothesis by analyzing neural population structure and dynamics between observed and executed movements. We find that observed and executed movements exhibit similar neural population covariation in a shared subspace capturing significant neural variance. Further, neural dynamics are more similar between observed and executed movements within the shared subspace than outside it. Finally, we find that this shared subspace has a heterogeneous composition of congruent and incongruent neurons. Together, these results argue that similar neural covariation and dynamics between observed and executed movements do not occur via activation of a subpopulation of congruent single neurons, but through consistent temporal activation of a heterogeneous neural population.

  View details for DOI 10.1016/j.celrep.2020.108006

  View details for PubMedID 32783934

• Power-saving design opportunities for wireless intracortical brain-computer interfaces. Nature biomedical engineering Even-Chen, N., Muratore, D. G., Stavisky, S. D., Hochberg, L. R., Henderson, J. M., Murmann, B., Shenoy, K. V. 2020

  Abstract

  The efficacy of wireless intracortical brain-computer interfaces (iBCIs) is limited in part by the number of recording channels, which is constrained by the power budget of the implantable system. Designing wireless iBCIs that provide the high-quality recordings of today's wired neural interfaces may lead to inadvertent over-design at the expense of power consumption and scalability. Here, we report analyses of neural signals collected from experimental iBCI measurements in rhesus macaques and from a clinical-trial participant with implanted 96-channel Utah multielectrode arrays to understand the trade-offs between signal quality and decoder performance. Moreover, we propose an efficient hardware design for clinically viable iBCIs, and suggest that the circuit design parameters of current recording iBCIs can be relaxed considerably without loss of performance. The proposed design may allow for an order-of-magnitude power savings and lead to clinically viable iBCIs with a higher channel count.

  View details for DOI 10.1038/s41551-020-0595-9

  View details for PubMedID 32747834

• Computation Through Neural Population Dynamics. Annual review of neuroscience Vyas, S., Golub, M. D., Sussillo, D., Shenoy, K. V. 2020; 43: 249–75

  Abstract

  Significant experimental, computational, and theoretical work has identified rich structure within the coordinated activity of interconnected neural populations. An emerging challenge now is to uncover the nature of the associated computations, how they are implemented, and what role they play in driving behavior. We term this computation through neural population dynamics. If successful, this framework will reveal general motifs of neural population activity and quantitatively describe how neural population dynamics implement computations necessary for driving goal-directed behavior. Here, we start with a mathematical primer on dynamical systems theory and analytical tools necessary to apply this perspective to experimental data. Next, we highlight some recent discoveries resulting from successful application of dynamical systems. We focus on studies spanning motor control, timing, decision-making, and working memory. Finally, we briefly discuss promising recent lines of investigation and future directions for the computation through neural population dynamics framework.

  View details for DOI 10.1146/annurev-neuro-092619-094115

  View details for PubMedID 32640928

• Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way. Cell Willett, F. R., Deo, D. R., Avansino, D. T., Rezaii, P., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2020

  Abstract

  Decades after the motor homunculus was first proposed, it is still unknown how different body parts are intermixed and interrelated in human motor cortical areas at single-neuron resolution. Using multi-unit recordings, we studied how face, head, arm, and leg movements are represented in the hand knob area of premotor cortex (precentral gyrus) in people with tetraplegia. Contrary to traditional expectations, we found strong representation of all movements and a partially "compositional" neural code that linked together all four limbs. The code consisted of (1) a limb-coding component representing the limb to be moved and (2) a movement-coding component where analogous movements from each limb (e.g., hand grasp and toe curl) were represented similarly. Compositional coding might facilitate skill transfer across limbs, and it provides a useful framework for thinking about how the motor system constructsmovement. Finally, we leveraged these results to create a whole-body intracortical brain-computer interface that spreads targets across all limbs.

  View details for DOI 10.1016/j.cell.2020.02.043

  View details for PubMedID 32220308

• Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control. Journal of neural engineering Stavisky, S. D., Willett, F. R., Avansino, D. T., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2020; 17 (1): 016049

  Abstract

  OBJECTIVE: Speech-related neural modulation was recently reported in 'arm/hand' area of human dorsal motor cortex that is used as a signal source for intracortical brain-computer interfaces (iBCIs). This raises the concern that speech-related modulation might deleteriously affect the decoding of arm movement intentions, for instance by affecting velocity command outputs. This study sought to clarify whether or not speaking would interfere with ongoing iBCI use.APPROACH: A participant in the BrainGate2 iBCI clinical trial used an iBCI to control a computer cursor; spoke short words in a stand-alone speech task; and spoke short words during ongoing iBCI use. We examined neural activity in all three behaviors and compared iBCI performance with and without concurrent speech.MAIN RESULTS: Dorsal motor cortex firing rates modulated strongly during stand-alone speech, but this activity was largely attenuated when speaking occurred during iBCI cursor control using attempted arm movements. 'Decoder-potent' projections of the attenuated speech-related neural activity were small, explaining why cursor task performance was similar between iBCI use with and without concurrent speaking.SIGNIFICANCE: These findings indicate that speaking does not directly interfere with iBCIs that decode attempted arm movements. This suggests that patients who are able to speak will be able to use motor cortical-driven computer interfaces or prostheses without needing to forgo speaking while using these devices.

  View details for DOI 10.1088/1741-2552/ab5b72

  View details for PubMedID 32023225

• Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping NEURON Williams, A. H., Poole, B., Maheswaranathan, N., Dhawale, A. K., Fisher, T., Wilson, C. D., Brann, D. H., Trautmann, E. M., Ryu, S., Shusterman, R., Rinberg, D., Olveczky, B. P., Shenoy, K. V., Ganguli, S. 2020; 105 (2): 246-+

  View details for DOI 10.1016/j.neuron.2019.10.02

  View details for Web of Science ID 000508633600009

• Neural Representation of Observed, Imagined, and Attempted Grasping Force in Motor Cortex of Individuals with Chronic Tetraplegia. Scientific reports Rastogi, A. n., Vargas-Irwin, C. E., Willett, F. R., Abreu, J. n., Crowder, D. C., Murphy, B. A., Memberg, W. D., Miller, J. P., Sweet, J. A., Walter, B. L., Cash, S. S., Rezaii, P. G., Franco, B. n., Saab, J. n., Stavisky, S. D., Shenoy, K. V., Henderson, J. M., Hochberg, L. R., Kirsch, R. F., Ajiboye, A. B. 2020; 10 (1): 1429

  Abstract

  Hybrid kinetic and kinematic intracortical brain-computer interfaces (iBCIs) have the potential to restore functional grasping and object interaction capabilities in individuals with tetraplegia. This requires an understanding of how kinetic information is represented in neural activity, and how this representation is affected by non-motor parameters such as volitional state (VoS), namely, whether one observes, imagines, or attempts an action. To this end, this work investigates how motor cortical neural activity changes when three human participants with tetraplegia observe, imagine, and attempt to produce three discrete hand grasping forces with the dominant hand. We show that force representation follows the same VoS-related trends as previously shown for directional arm movements; namely, that attempted force production recruits more neural activity compared to observed or imagined force production. Additionally, VoS-modulated neural activity to a greater extent than grasping force. Neural representation of forces was lower than expected, possibly due to compromised somatosensory pathways in individuals with tetraplegia, which have been shown to influence motor cortical activity. Nevertheless, attempted forces (but not always observed or imagined forces) could be decoded significantly above chance, thereby potentially providing relevant information towards the development of a hybrid kinetic and kinematic iBCI.

  View details for DOI 10.1038/s41598-020-58097-1

  View details for PubMedID 31996696

• A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces. Nature biomedical engineering Nason, S. R., Vaskov, A. K., Willsey, M. S., Welle, E. J., An, H. n., Vu, P. P., Bullard, A. J., Nu, C. S., Kao, J. C., Shenoy, K. V., Jang, T. n., Kim, H. S., Blaauw, D. n., Patil, P. G., Chestek, C. A. 2020

  Abstract

  The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.

  View details for DOI 10.1038/s41551-020-0591-0

  View details for PubMedID 32719512

• An Open Resource for Non-human Primate Optogenetics. Neuron Tremblay, S. n., Acker, L. n., Afraz, A. n., Albaugh, D. L., Amita, H. n., Andrei, A. R., Angelucci, A. n., Aschner, A. n., Balan, P. F., Basso, M. A., Benvenuti, G. n., Bohlen, M. O., Caiola, M. J., Calcedo, R. n., Cavanaugh, J. n., Chen, Y. n., Chen, S. n., Chernov, M. M., Clark, A. M., Dai, J. n., Debes, S. R., Deisseroth, K. n., Desimone, R. n., Dragoi, V. n., Egger, S. W., Eldridge, M. A., El-Nahal, H. G., Fabbrini, F. n., Federer, F. n., Fetsch, C. R., Fortuna, M. G., Friedman, R. M., Fujii, N. n., Gail, A. n., Galvan, A. n., Ghosh, S. n., Gieselmann, M. A., Gulli, R. A., Hikosaka, O. n., Hosseini, E. A., Hu, X. n., Hüer, J. n., Inoue, K. I., Janz, R. n., Jazayeri, M. n., Jiang, R. n., Ju, N. n., Kar, K. n., Klein, C. n., Kohn, A. n., Komatsu, M. n., Maeda, K. n., Martinez-Trujillo, J. C., Matsumoto, M. n., Maunsell, J. H., Mendoza-Halliday, D. n., Monosov, I. E., Muers, R. S., Nurminen, L. n., Ortiz-Rios, M. n., O'Shea, D. J., Palfi, S. n., Petkov, C. I., Pojoga, S. n., Rajalingham, R. n., Ramakrishnan, C. n., Remington, E. D., Revsine, C. n., Roe, A. W., Sabes, P. N., Saunders, R. C., Scherberger, H. n., Schmid, M. C., Schultz, W. n., Seidemann, E. n., Senova, Y. S., Shadlen, M. N., Sheinberg, D. L., Siu, C. n., Smith, Y. n., Solomon, S. S., Sommer, M. A., Spudich, J. L., Stauffer, W. R., Takada, M. n., Tang, S. n., Thiele, A. n., Treue, S. n., Vanduffel, W. n., Vogels, R. n., Whitmire, M. P., Wichmann, T. n., Wurtz, R. H., Xu, H. n., Yazdan-Shahmorad, A. n., Shenoy, K. V., DiCarlo, J. J., Platt, M. L. 2020

  Abstract

  Optogenetics has revolutionized neuroscience in small laboratory animals, but its effect on animal models more closely related to humans, such as non-human primates (NHPs), has been mixed. To make evidence-based decisions in primate optogenetics, the scientific community would benefit from a centralized database listing all attempts, successful and unsuccessful, of using optogenetics in the primate brain. We contacted members of the community to ask for their contributions to an open science initiative. As of this writing, 45 laboratories around the world contributed more than 1,000 injection experiments, including precise details regarding their methods and outcomes. Of those entries, more than half had not been published. The resource is free for everyone to consult and contribute to on the Open Science Framework website. Here we review some of the insights from this initial release of the database and discuss methodological considerations to improve the success of optogenetic experiments in NHPs.

  View details for DOI 10.1016/j.neuron.2020.09.027

  View details for PubMedID 33080229

• Causal Role of Motor Preparation during Error-Driven Learning. Neuron Vyas, S. n., O'Shea, D. J., Ryu, S. I., Shenoy, K. V. 2020

  Abstract

  Current theories suggest that an error-driven learning process updates trial-by-trial to facilitate motor adaptation. How this process interacts with motor cortical preparatory activity-which current models suggest plays a critical role in movement initiation-remains unknown. Here, we evaluated the role of motor preparation during visuomotor adaptation. We found that preparation time was inversely correlated to variance of errors on current trials and mean error on subsequent trials. We also found causal evidence that intracortical microstimulation during motor preparation was sufficient to disrupt learning. Surprisingly, stimulation did not affect current trials, but instead disrupted the update computation of a learning process, thereby affecting subsequent trials. This is consistent with a Bayesian estimation framework where the motor system reduces its learning rate by virtue of lowering error sensitivity when faced with uncertainty. This interaction between motor preparation and the error-driven learning system may facilitate new probes into mechanisms underlying trial-by-trial adaptation.

  View details for DOI 10.1016/j.neuron.2020.01.019

  View details for PubMedID 32053768

• High-fidelity musculoskeletal modeling reveals a motor planning contribution to the speed-accuracy tradeoff. eLife Al Borno, M. n., Vyas, S. n., Shenoy, K. V., Delp, S. L. 2020; 9

  Abstract

  A long-standing challenge in motor neuroscience is to understand the relationship between movement speed and accuracy, known as the speed-accuracy tradeoff. Here, we introduce a biomechanically realistic computational model of three-dimensional upper extremity movements that reproduces well-known features of reaching movements. This model revealed that the speed-accuracy tradeoff, as described by Fitts' law, emerges even without the presence of motor noise, which is commonly believed to underlie the speed-accuracy tradeoff. Next, we analyzed motor cortical neural activity from monkeys reaching to targets of different sizes. We found that the contribution of preparatory neural activity to movement duration variability is greater for smaller targets than larger targets, and that movements to smaller targets exhibit less variability in population-level preparatory activity, but greater movement duration variability. These results propose a new theory underlying the speed-accuracy tradeoff: Fitts' law emerges from greater task demands constraining the optimization landscape in a fashion that reduces the number of 'good' control solutions (i.e., faster reaches). Thus, contrary to current beliefs, the speed-accuracy tradeoff could be a consequence of motor planning variability and not exclusively signal-dependent noise.

  View details for DOI 10.7554/eLife.57021

  View details for PubMedID 33325369

• Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. eLife Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., Ajiboye, A. B., Druckmann, S., Shenoy, K. V., Henderson, J. M. 2019; 8

  Abstract

  Speaking is a sensorimotor behavior whose neural basis is difficult to study with single neuron resolution due to the scarcity of human intracortical measurements. We used electrode arrays to record from the motor cortex 'hand knob' in two people with tetraplegia, an area not previously implicated in speech. Neurons modulated during speaking and during non-speaking movements of the tongue, lips, and jaw. This challenges whether the conventional model of a 'motor homunculus' division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential of intracortical recordings for brain-computer interfaces to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a component that was mostly invariant across initiating different words, followed by rotatory dynamics during speaking. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.

  View details for DOI 10.7554/eLife.46015

  View details for PubMedID 31820736

• Publisher Correction: Development of an optogenetic toolkit for neural circuit dissection in squirrel monkeys. Scientific reports O'Shea, D. J., Kalanithi, P., Ferenczi, E. A., Hsueh, B., Chandrasekaran, C., Goo, W., Diester, I., Ramakrishnan, C., Kaufman, M. T., Ryu, S. I., Yeom, K. W., Deisseroth, K., Shenoy, K. V. 2019; 9 (1): 18775

  Abstract

  An amendment to this paper has been published and can be accessed via a link at the top of the paper.

  View details for DOI 10.1038/s41598-019-55025-w

  View details for PubMedID 31801956

• Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping. Neuron Williams, A. H., Poole, B., Maheswaranathan, N., Dhawale, A. K., Fisher, T., Wilson, C. D., Brann, D. H., Trautmann, E. M., Ryu, S., Shusterman, R., Rinberg, D., Olveczky, B. P., Shenoy, K. V., Ganguli, S. 2019

  Abstract

  Though the temporal precision of neural computation has been studied intensively, a data-driven determination of this precision remains a fundamental challenge. Reproducible spike patterns may be obscured on single trials by uncontrolled temporal variability in behavior and cognition and may not be time locked to measurable signatures in behavior or local field potentials (LFP). To overcome these challenges, we describe a general-purpose time warping framework that reveals precise spike-time patterns in an unsupervised manner, even when these patterns are decoupled from behavior or are temporally stretched across single trials. We demonstrate this method across diverse systems: cued reaching in nonhuman primates, motor sequence production in rats, and olfaction in mice. This approach flexibly uncovers diverse dynamical firing patterns, including pulsatile responses to behavioral events, LFP-aligned oscillatory spiking, and even unanticipated patterns, such as 7Hz oscillations in rat motor cortex that are not time locked to measured behaviors or LFP.

  View details for DOI 10.1016/j.neuron.2019.10.020

  View details for PubMedID 31786013

• Simultaneous motor preparation and execution in a last-moment reach correction task. Nature communications Ames, K. C., Ryu, S. I., Shenoy, K. V. 2019; 10 (1): 2718

  Abstract

  Motor preparation typically precedes movement and is thought to determine properties of upcoming movements. However, preparation has mostly been studied in point-to-point delayed reaching tasks. Here, we ask whether preparation is engaged during mid-reach modifications. Monkeys reach to targets that occasionally jump locations prior to movement onset, requiring a mid-reach correction. In motor cortex and dorsal premotor cortex, we find that the neural activity that signals when to reach predicts monkeys' jump responses on a trial-by-trial basis. We further identify neural patterns that signal where to reach, either during motor preparation or during motor execution. After a target jump, neural activity responds in both preparatory and movement-related dimensions, even though error in preparatory dimensions can be small at that time. This suggests that the same preparatory process used in delayed reaching is also involved in reach correction. Furthermore, it indicates that motor preparation and execution can be performed simultaneously.

  View details for DOI 10.1038/s41467-019-10772-2

  View details for PubMedID 31221968

• Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus-response associations are known NATURE COMMUNICATIONS Wang, M., Montanede, C., Chandrasekaran, C., Peixoto, D., Shenoy, K. V., Kalaska, J. F. 2019; 10

  View details for DOI 10.1038/s41467-019-09460-y

  View details for Web of Science ID 000464976200006

• Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus-response associations are known. Nature communications Wang, M., Montanede, C., Chandrasekaran, C., Peixoto, D., Shenoy, K. V., Kalaska, J. F. 2019; 10 (1): 1793

  Abstract

  How deliberation on sensory cues and action selection interact in decision-related brain areas is still not well understood. Here, monkeys reached to one of two targets, whose colors alternated randomly between trials, by discriminating the dominant color of a checkerboard cue composed of different numbers of squares of the two target colors in different trials. In a Targets First task the colored targets appeared first, followed by the checkerboard; in a Checkerboard First task, this order was reversed. After both cues appeared in both tasks, responses of dorsal premotor cortex (PMd) units covaried with action choices, strength of evidence for action choices, and RTs- hallmarks of decision-related activity. However, very few units were modulated by checkerboard color composition or the color of the chosen target, even during the checkerboard deliberation epoch of the Checkerboard First task. These findings implicate PMd in the action-selection but not the perceptual components of the decision-making process in these tasks.

  View details for PubMedID 30996222

• Volitional control of single-electrode high gamma local field potentials by people with paralysis JOURNAL OF NEUROPHYSIOLOGY Milekovic, T., Bacher, D., Sarma, A. A., Simeral, J. D., Saab, J., Pandarinath, C., Yvert, B., Sorice, B. L., Blabe, C., Oakley, E. M., Tringale, K. R., Eskandar, E., Cash, S. S., Shenoy, K., Henderson, J. M., Hochberg, L. R., Donoghue, J. P. 2019; 121 (4): 1428–50

  View details for DOI 10.1152/jn.00131.2018

  View details for Web of Science ID 000465083500028

• Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements (vol 8, 16357, 2018) SCIENTIFIC REPORTS Stavisky, S. D., Kao, J. C., Nuyujukian, P., Pandarinath, C., Blabe, C., Ryu, S. I., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2019; 9

  View details for DOI 10.1038/s41598-018-37930-8

  View details for Web of Science ID 000462586000001

• Publisher Correction: Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements. Scientific reports Stavisky, S. D., Kao, J. C., Nuyujukian, P., Pandarinath, C., Blabe, C., Ryu, S. I., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2019; 9 (1): 5528

  Abstract

  A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.

  View details for PubMedID 30918269

• Structure and variability of delay activity in premotor cortex. PLoS computational biology Even-Chen, N., Sheffer, B., Vyas, S., Ryu, S. I., Shenoy, K. V. 2019; 15 (2): e1006808

  Abstract

  Voluntary movements are widely considered to be planned before they are executed. Recent studies have hypothesized that neural activity in motor cortex during preparation acts as an 'initial condition' which seeds the proceeding neural dynamics. Here, we studied these initial conditions in detail by investigating 1) the organization of neural states for different reaches and 2) the variance of these neural states from trial to trial. We examined population-level responses in macaque premotor cortex (PMd) during the preparatory stage of an instructed-delay center-out reaching task with dense target configurations. We found that after target onset the neural activity on single trials converges to neural states that have a clear low-dimensional structure which is organized by both the reach endpoint and maximum speed of the following reach. Further, we found that variability of the neural states during preparation resembles the spatial variability of reaches made in the absence of visual feedback: there is less variability in direction than distance in neural state space. We also used offline decoding to understand the implications of this neural population structure for brain-machine interfaces (BMIs). We found that decoding of angle between reaches is dependent on reach distance, while decoding of arc-length is independent. Thus, it might be more appropriate to quantify decoding performance for discrete BMIs by using arc-length between reach end-points rather than the angle between them. Lastly, we show that in contrast to the common notion that direction can better be decoded than distance, their decoding capabilities are comparable. These results provide new insights into the dynamical neural processes that underline motor control and can inform the design of BMIs.

  View details for PubMedID 30794541

• Volitional control of single-electrode high gamma local field potentials (LFPs) by people with paralysis. Journal of neurophysiology Milekovic, T., Bacher, D., Sarma, A. A., Simeral, J. D., Saab, J., Pandarinath, C., Yvert, B., Sorice, B. L., Blabe, C., Oakley, E. M., Tringale, K. R., Eskandar, E., Cash, S. S., Shenoy, K. V., Henderson, J. M., Hochberg, L. R., Donoghue, J. P. 2019

  Abstract

  Intracortical brain-computer interfaces (BCIs) can enable individuals to control effectors, such as a computer cursor, by directly decoding the user's movement intentions from action potentials and local field potentials (LFPs) recorded within the motor cortex. However, the accuracy and complexity of effector control achieved with such "biomimetic" BCIs will depend on the degree to which the intended movements used to elicit control modulate the neural activity. In particular, channels that do not record distinguishable action potentials and only record LFP modulations may be of limited use for BCI control. In contrast, a biofeedback approach may surpass these limitations by letting the participants generate new control signals and learn strategies that improve the volitional control of signals used for effector control. Here, we show that, by using a biofeedback paradigm, three individuals with tetraplegia achieved volitional control of gamma LFPs (40-400Hz) recorded by a single microelectrode implanted in the precentral gyrus. Control was improved over a pair of consecutive sessions up to three days apart. In all but one session, the channel used to achieve control lacked distinguishable action potentials. Our results indicate that biofeedback LFP-based BCIs may potentially contribute to the neural modulation necessary to obtain reliable and useful control of effectors.

  View details for PubMedID 30785814

• Frequency shifts and depth dependence of premotor beta band activity during perceptual decision-making. The Journal of neuroscience : the official journal of the Society for Neuroscience Chandrasekaran, C., Bray, I. E., Shenoy, K. V. 2019

  Abstract

  Neural activity in the premotor and motor cortices shows prominent structure in the beta frequency range (13-30 Hz). Currently, the behavioral relevance of this beta band activity (BBA) is debated. The underlying source of motor BBA and how it changes as a function of cortical depth is also not completely understood. Here, we addressed these unresolved questions by investigating BBA recorded using laminar electrodes in the dorsal premotor cortex (PMd) of two male rhesus macaques performing a visual reaction time (RT) reach discrimination task. We observed robust BBA before and after the onset of the visual stimulus but not during the arm movement. While post-stimulus BBA was positively correlated with RT throughout the beta frequency range, pre-stimulus correlation varied by frequency. Low beta frequencies (12 to 20 Hz) were positively correlated with RT and high beta frequencies (22 to 30 Hz) were negatively correlated with RT. Analysis and simulations suggested that these frequency-dependent correlations could emerge due to a shift in the component frequencies of the pre-stimulus BBA as a function of RT, such that faster RTs are accompanied by greater power in high beta frequencies. We also observed a laminar dependence of BBA, with deeper electrodes demonstrating stronger power in low beta frequencies both pre- and post-stimulus. The heterogeneous nature of BBA and the changing relationship between BBA and RT in different task epochs may be a sign of the differential network dynamics involved in cue expectation, decision-making, motor preparation, and movement execution.SIGNIFICANCE STATEMENTBeta band activity (BBA) has been implicated in motor tasks, in disease states, and as a potential signal for brain-machine interfaces. However, the behavioral relevance of BBA and its laminar organization in premotor cortex have not been completely elucidated. Here we addressed these unresolved issues using simultaneous recordings from multiple cortical layers of the premotor cortex of monkeys performing a decision-making task. Our key finding is that BBA is not a monolithic signal. Instead, BBA consists of at least two frequency bands. The relationship between BBA and eventual behavior, such as reaction time, also dynamically changes depending on task epoch. We also provide further evidence that BBA is laminarly organized, with greater power in deeper electrodes for low beta frequencies.

  View details for PubMedID 30606756

• Accurate Estimation of Neural Population Dynamics without Spike Sorting. Neuron Trautmann, E. M., Stavisky, S. D., Lahiri, S. n., Ames, K. C., Kaufman, M. T., O'Shea, D. J., Vyas, S. n., Sun, X. n., Ryu, S. I., Ganguli, S. n., Shenoy, K. V. 2019

  Abstract

  A central goal of systems neuroscience is to relate an organism's neural activity to behavior. Neural population analyses often reduce the data dimensionality to focus on relevant activity patterns. A major hurdle to data analysis is spike sorting, and this problem is growing as the number of recorded neurons increases. Here, we investigate whether spike sorting is necessary to estimate neural population dynamics. The theory of random projections suggests that we can accurately estimate the geometry of low-dimensional manifolds from a small number of linear projections of the data. We recorded data using Neuropixels probes in motor cortex of nonhuman primates and reanalyzed data from three previous studies and found that neural dynamics and scientific conclusions are quite similar using multiunit threshold crossings rather than sorted neurons. This finding unlocks existing data for new analyses and informs the design and use of new electrode arrays for laboratory and clinical use.

  View details for DOI 10.1016/j.neuron.2019.05.003

  View details for PubMedID 31171448

• Principled BCI Decoder Design and Parameter Selection Using a Feedback Control Model. Scientific reports Willett, F. R., Young, D. R., Murphy, B. A., Memberg, W. D., Blabe, C. H., Pandarinath, C. n., Stavisky, S. D., Rezaii, P. n., Saab, J. n., Walter, B. L., Sweet, J. A., Miller, J. P., Henderson, J. M., Shenoy, K. V., Simeral, J. D., Jarosiewicz, B. n., Hochberg, L. R., Kirsch, R. F., Bolu Ajiboye, A. n. 2019; 9 (1): 8881

  Abstract

  Decoders optimized offline to reconstruct intended movements from neural recordings sometimes fail to achieve optimal performance online when they are used in closed-loop as part of an intracortical brain-computer interface (iBCI). This is because typical decoder calibration routines do not model the emergent interactions between the decoder, the user, and the task parameters (e.g. target size). Here, we investigated the feasibility of simulating online performance to better guide decoder parameter selection and design. Three participants in the BrainGate2 pilot clinical trial controlled a computer cursor using a linear velocity decoder under different gain (speed scaling) and temporal smoothing parameters and acquired targets with different radii and distances. We show that a user-specific iBCI feedback control model can predict how performance changes under these different decoder and task parameters in held-out data. We also used the model to optimize a nonlinear speed scaling function for the decoder. When used online with two participants, it increased the dynamic range of decoded speeds and decreased the time taken to acquire targets (compared to an optimized standard decoder). These results suggest that it is feasible to simulate iBCI performance accurately enough to be useful for quantitative decoder optimization and design.

  View details for DOI 10.1038/s41598-019-44166-7

  View details for PubMedID 31222030

• Closed-loop cortical control of virtual reach and posture using cartesian and joint velocity commands. Journal of neural engineering Young, D., Willett, F., Memberg, W. D., Murphy, B. A., Rezaii, P., Walter, B., Sweet, J. A., Miller, J., Shenoy, K. V., Hochberg, L., Kirsch, R. F., Ajiboye, A. B. 2018

  Abstract

  OBJECTIVE: Brain-computer interfaces (BCIs) are a promising technology for the restoration of function to people with paralysis, especially for controlling coordinated reaching. Typical BCI studies decode Cartesian endpoint velocities as commands, but human arm movements might be better controlled in a joint-based coordinate frame, which may match underlying movement encoding in the motor cortex. A better understanding of BCI controlled reaching by people with paralysis may lead to performance improvements in brain-controlled assistive devices. Approach. Two intracortical BCI participants in the BrainGate2 pilot clinical trial performed a 3D endpoint virtual reaching task using two decoders: Cartesian and joint velocity. Task performance metrics (i.e. success rate and path efficiency) and single feature and population tuning were compared across the two decoder conditions. The participants also demonstrated the first BCI control of a fourth dimension of reaching, the arm's swivel angle, in a 4D posture matching task. Main Results. Both users achieved significantly higher success rates using Cartesian control, and joint controlled trajectories were more variable and significantly more curved. Neural tuning analyses showed that most single feature activity was best described by a Cartesian kinematic encoding model, and population analyses revealed only slight differences in aggregate activity between the decoder conditions. Simulations of a BCI user reproduced trajectory features seen during closed-loop joint control when assuming only Cartesian-tuned features passed through a joint decoder. With minimal training, both participants controlled the virtual arm's swivel angle to complete a 4D posture matching task, and achieved significantly higher success using a Cartesian+swivel decoder compared to a joint velocity decoder. Significance. These results suggest that Cartesian command interfaces may provide better BCI control of arm movements than other kinematic variables, even in 4D posture tasks with swivel angle targets.

  View details for DOI 10.1088/1741-2552/aaf606

  View details for PubMedID 30523839

• Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements. Scientific reports Stavisky, S. D., Kao, J. C., Nuyujukian, P., Pandarinath, C., Blabe, C., Ryu, S. I., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2018; 8 (1): 16357

  Abstract

  Brain-machine interfaces (BMIs) that decode movement intentions should ignore neural modulation sources distinct from the intended command. However, neurophysiology and control theory suggest that motor cortex reflects the motor effector's position, which could be a nuisance variable. We investigated motor cortical correlates of BMI cursor position with or without concurrent arm movement. We show in two monkeys that subtracting away estimated neural correlates of position improves online BMI performance only if the animals were allowed to move their arm. To understand why, we compared the neural variance attributable to cursor position when the same task was performed using arm reaching, versus arms-restrained BMI use. Firing rates correlated with both BMI cursor and hand positions, but hand positional effects were greater. To examine whether BMI position influences decoding in people with paralysis, we analyzed data from two intracortical BMI clinical trial participants and performed an online decoder comparison in one participant. We found only small motor cortical correlates, which did not affect performance. These results suggest that arm movement and proprioception are the major contributors to position-related motor cortical correlates. Cursor position visual feedback is therefore unlikely to affect the performance of BMI-driven prosthetic systems being developed for people with paralysis.

  View details for PubMedID 30397281

• Inferring single-trial neural population dynamics using sequential auto-encoders NATURE METHODS Pandarinath, C., O'Shea, D. J., Collins, J., Jozefowicz, R., Stavisky, S. D., Kao, J. C., Trautmann, E. M., Kaufman, M. T., Ryu, S. I., Hochberg, L. R., Henderson, J. M., Shenoy, K. V., Abbott, L. F., Sussillo, D. 2018; 15 (10): 805-+

  Abstract

  Neuroscience is experiencing a revolution in which simultaneous recording of thousands of neurons is revealing population dynamics that are not apparent from single-neuron responses. This structure is typically extracted from data averaged across many trials, but deeper understanding requires studying phenomena detected in single trials, which is challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. We introduce latent factor analysis via dynamical systems, a deep learning method to infer latent dynamics from single-trial neural spiking data. When applied to a variety of macaque and human motor cortical datasets, latent factor analysis via dynamical systems accurately predicts observed behavioral variables, extracts precise firing rate estimates of neural dynamics on single trials, infers perturbations to those dynamics that correlate with behavioral choices, and combines data from non-overlapping recording sessions spanning months to improve inference of underlying dynamics.

  View details for PubMedID 30224673

• A Comparison of Intention Estimation Methods for Decoder Calibration in Intracortical Brain-Computer Interfaces IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Willett, F. R., Murphy, B. A., Young, D., Memberg, W. D., Blabe, C. H., Pandarinath, C., Franco, B., Saab, J., Walter, B. L., Sweet, J. A., Miller, J. P., Henderson, J. M., Shenoy, K. V., Simeral, J. D., Jarosiewicz, B., Hochberg, L. R., Kirsch, R. F., Ajiboye, A. 2018; 65 (9): 2066–78

  Abstract

  Recent reports indicate that making better assumptions about the user's intended movement can improve the accuracy of decoder calibration for intracortical brain-computer interfaces. Several methods now exist for estimating user intent, including an optimal feedback control model, a piecewise-linear feedback control model, ReFIT, and other heuristics. Which of these methods yields the best decoding performance?Using data from the BrainGate2 pilot clinical trial, we measured how a steady-state velocity Kalman filter decoder was affected by the choice of intention estimation method. We examined three separate components of the Kalman filter: dimensionality reduction, temporal smoothing, and output gain (speed scaling).The decoder's dimensionality reduction properties were largely unaffected by the intention estimation method. Decoded velocity vectors differed by <5% in terms of angular error and speed vs. target distance curves across methods. In contrast, the smoothing and gain properties of the decoder were greatly affected (> 50% difference in average values). Since the optimal gain and smoothing properties are task-specific (e.g. lower gains are better for smaller targets but worse for larger targets), no one method was better for all tasks.Our results show that, when gain and smoothing differences are accounted for, current intention estimation methods yield nearly equivalent decoders and that simple models of user intent, such as a position error vector (target position minus cursor position), perform comparably to more elaborate models. Our results also highlight that simple differences in gain and smoothing properties have a large effect on online performance and can confound decoder comparisons.

  View details for DOI 10.1109/TBME.2017.2783358

  View details for Web of Science ID 000442349500017

  View details for PubMedID 29989927

  View details for PubMedCentralID PMC6043406

• Single Neuron Firing Rate Statistics in Motor Cortex During Execution and Observation of Movement. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference Jiang, X., Ryu, S. I., Shenoy, K. V., Kao, J. C. 2018; 2018: 981–86

  Abstract

  Mirror neurons, which fire during both the execution and observation of movement, are believed to play an important role in motor processing and learning. However, much work still remains to understand the similarities and differences in how these neurons compute in the motor cortex during movement execution and observation. Here, we performed experiments where a monkey both executes and observes a center-out-and-back task within the same experimental session. By recording from putatively the same neural population, we were able to analyze and compare single neuron statistics between movement execution and observation. We found that a majority of neurons in the primary motor cortex (M1) and dorsal premotor cortex (PMd) have statistically different firing rate statistics between movement execution and observation. As a result of this difference, we then wondered if neurons during movement observation exhibited a similar characteristic to those during movement execution: changing of preferred directions as a function of movement speed. Interestingly, we found that while observed movement speed is encoded in the neural population, it only alters a small proportion of the neuron's firing rate statistics. These results suggest that neural populations in Ml and PMd process information related to movement differently between execution and observation.

  View details for DOI 10.1109/EMBC.2018.8512445

  View details for PubMedID 30440555

• Development of an optogenetic toolkit for neural circuit dissection in squirrel monkeys SCIENTIFIC REPORTS O'Shea, D. J., Kalanithi, P., Ferenczi, E. A., Hsueh, B., Chandrasekaran, C., Goo, W., Diester, I., Ramakrishnan, C., Kaufman, M. T., Ryu, S. I., Yeom, K. W., Deisseroth, K., Shenoy, K. V. 2018; 8: 6775

  Abstract

  Optogenetic tools have opened a rich experimental landscape for understanding neural function and disease. Here, we present the first validation of eight optogenetic constructs driven by recombinant adeno-associated virus (AAV) vectors and a WGA-Cre based dual injection strategy for projection targeting in a widely-used New World primate model, the common squirrel monkey Saimiri sciureus. We observed opsin expression around the local injection site and in axonal projections to downstream regions, as well as transduction to thalamic neurons, resembling expression patterns observed in macaques. Optical stimulation drove strong, reliable excitatory responses in local neural populations for two depolarizing opsins in anesthetized monkeys. Finally, we observed continued, healthy opsin expression for at least one year. These data suggest that optogenetic tools can be readily applied in squirrel monkeys, an important first step in enabling precise, targeted manipulation of neural circuits in these highly trainable, cognitively sophisticated animals. In conjunction with similar approaches in macaques and marmosets, optogenetic manipulation of neural circuits in squirrel monkeys will provide functional, comparative insights into neural circuits which subserve dextrous motor control as well as other adaptive behaviors across the primate lineage. Additionally, development of these tools in squirrel monkeys, a well-established model system for several human neurological diseases, can aid in identifying novel treatment strategies.

  View details for PubMedID 29712920

• ERAASR: an algorithm for removing electrical stimulation artifacts from multielectrode array recordings JOURNAL OF NEURAL ENGINEERING O'Shea, D. J., Shenoy, K. V. 2018; 15 (2): 026020

  Abstract

  Electrical stimulation is a widely used and effective tool in systems neuroscience, neural prosthetics, and clinical neurostimulation. However, electrical artifacts evoked by stimulation prevent the detection of spiking activity on nearby recording electrodes, which obscures the neural population response evoked by stimulation. We sought to develop a method to clean artifact-corrupted electrode signals recorded on multielectrode arrays in order to recover the underlying neural spiking activity.We created an algorithm, which performs estimation and removal of array artifacts via sequential principal components regression (ERAASR). This approach leverages the similar structure of artifact transients, but not spiking activity, across simultaneously recorded channels on the array, across pulses within a train, and across trials. The ERAASR algorithm requires no special hardware, imposes no requirements on the shape of the artifact or the multielectrode array geometry, and comprises sequential application of straightforward linear methods with intuitive parameters. The approach should be readily applicable to most datasets where stimulation does not saturate the recording amplifier.The effectiveness of the algorithm is demonstrated in macaque dorsal premotor cortex using acute linear multielectrode array recordings and single electrode stimulation. Large electrical artifacts appeared on all channels during stimulation. After application of ERAASR, the cleaned signals were quiescent on channels with no spontaneous spiking activity, whereas spontaneously active channels exhibited evoked spikes which closely resembled spontaneously occurring spiking waveforms.We hope that enabling simultaneous electrical stimulation and multielectrode array recording will help elucidate the causal links between neural activity and cognition and facilitate naturalistic sensory protheses.

  View details for PubMedID 29265009

  View details for PubMedCentralID PMC5833982

• Neural Population Dynamics Underlying Motor Learning Transfer NEURON Vyas, S., Even-Chen, N., Stavisky, S. D., Ryu, S. I., Nuyujukian, P., Shenoy, K. V. 2018; 97 (5): 1177-+

  Abstract

  Covert motor learning can sometimes transfer to overt behavior. We investigated the neural mechanism underlying transfer by constructing a two-context paradigm. Subjects performed cursor movements either overtly using arm movements, or covertly via a brain-machine interface that moves the cursor based on motor cortical activity (in lieu of arm movement). These tasks helped evaluate whether and how cortical changes resulting from "covert rehearsal" affect overt performance. We found that covert learning indeed transfers to overt performance and is accompanied by systematic population-level changes in motor preparatory activity. Current models of motor cortical function ascribe motor preparation to achieving initial conditions favorable for subsequent movement-period neural dynamics. We found that covert and overt contexts share these initial conditions, and covert rehearsal manipulates them in a manner that persists across context changes, thus facilitating overt motor learning. This transfer learning mechanism might provide new insights into other covert processes like mental rehearsal.

  View details for PubMedID 29456026

  View details for PubMedCentralID PMC5843544

• Feasibility of Automatic Error Detect-and-Undo System in Human Intracortical Brain-Computer Interfaces. IEEE transactions on bio-medical engineering Even-Chen, N. n., Stavisky, S. D., Pandarinath, C. n., Nuyujukian, P. n., Blabe, C. H., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2018; 65 (8): 1771–84

  Abstract

  Brain-computer interfaces (BCIs) aim to help people with impaired movement ability by directly translating their movement intentions into command signals for assistive technologies. Despite large performance improvements over the last two decades, BCI systems still make errors that need to be corrected manually by the user. This decreases system performance and is also frustrating for the user. The deleterious effects of errors could be mitigated if the system automatically detected when the user perceives that an error was made and automatically intervened with a corrective action; thus, sparing users from having to make the correction themselves. Our previous preclinical work with monkeys demonstrated that task-outcome correlates exist in motor cortical spiking activity and can be utilized to improve BCI performance. Here, we asked if these signals also exist in the human hand area of motor cortex, and whether they can be decoded with high accuracy.We analyzed posthoc the intracortical neural activity of two BrainGate2 clinical trial participants who were neurally controlling a computer cursor to perform a grid target selection task and a keyboard-typing task.Our key findings are that: 1) there exists a putative outcome error signal reflected in both the action potentials and local field potentials of the human hand area of motor cortex, and 2) target selection outcomes can be classified with high accuracy (70-85%) of errors successfully detected with minimal (0-3%) misclassifications of success trials, based on neural activity alone.These offline results suggest that it will be possible to improve the performance of clinical intracortical BCIs by incorporating a real-time error detect-and-undo system alongside the decoding of movement intention.

  View details for DOI 10.1109/TBME.2017.2776204

  View details for PubMedID 29989931

• Cortical control of a tablet computer by people with paralysis. PloS one Nuyujukian, P., Albites Sanabria, J., Saab, J., Pandarinath, C., Jarosiewicz, B., Blabe, C. H., Franco, B., Mernoff, S. T., Eskandar, E. N., Simeral, J. D., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2018; 13 (11): e0204566

  Abstract

  General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

  View details for PubMedID 30462658

• Stable long-term BCI-enabled communication in ALS and locked-in syndrome using LFP signals. Journal of neurophysiology Milekovic, T. n., Sarma, A. A., Bacher, D. n., Simeral, J. D., Saab, J. n., Pandarinath, C. n., Sorice, B. L., Blabe, C. n., Oakley, E. M., Tringale, K. R., Eskandar, E. n., Cash, S. S., Henderson, J. M., Shenoy, K. V., Donoghue, J. P., Hochberg, L. R. 2018

  Abstract

  Restoring communication for people with locked-in syndrome remains a challenging clinical problem without a reliable solution. Recent studies have shown that people with paralysis can use brain-computer interfaces (BCIs) based on intracortical spiking activity to efficiently type messages. However, due to neuronal signal instability, most intracortical BCIs have required frequent calibration and continuous assistance of skilled engineers to maintain performance. Here, an individual with locked-in syndrome due to brainstem stroke and an individual with tetraplegia secondary to amyotrophic lateral sclerosis (ALS) used a simple communication BCI based on intracortical local field potentials (LFPs) for 76 and 138 days, respectively, without recalibration and without significant loss of performance. BCI spelling rates of 3.07 and 6.88 correct characters/minute allowed the participants to type messages and write emails. Our results indicate that people with locked-in syndrome could soon use a slow but reliable LFP-based BCI for everyday communication without ongoing intervention from a technician or caregiver.

  View details for PubMedID 29694279

• Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. Journal of neural engineering Brandman, D. M., Hosman, T. n., Saab, J. n., Burkhart, M. C., Shanahan, B. E., Ciancibello, J. G., Sarma, A. A., Milstein, D. J., Vargas-Irwin, C. E., Franco, B. n., Kelemen, J. n., Blabe, C. n., Murphy, B. A., Young, D. R., Willett, F. R., Pandarinath, C. n., Stavisky, S. D., Kirsch, R. F., Walter, B. L., Bolu Ajiboye, A. n., Cash, S. S., Eskandar, E. N., Miller, J. P., Sweet, J. A., Shenoy, K. V., Henderson, J. M., Jarosiewicz, B. n., Harrison, M. T., Simeral, J. D., Hochberg, L. R. 2018; 15 (2): 026007

  Abstract

  Brain-computer interfaces (BCIs) can enable individuals with tetraplegia to communicate and control external devices. Though much progress has been made in improving the speed and robustness of neural control provided by intracortical BCIs, little research has been devoted to minimizing the amount of time spent on decoder calibration.We investigated the amount of time users needed to calibrate decoders and achieve performance saturation using two markedly different decoding algorithms: the steady-state Kalman filter, and a novel technique using Gaussian process regression (GP-DKF).Three people with tetraplegia gained rapid closed-loop neural cursor control and peak, plateaued decoder performance within 3 min of initializing calibration. We also show that a BCI-naïve user (T5) was able to rapidly attain closed-loop neural cursor control with the GP-DKF using self-selected movement imagery on his first-ever day of closed-loop BCI use, acquiring a target 37 s after initiating calibration.These results demonstrate the potential for an intracortical BCI to be used immediately after deployment by people with paralysis, without the need for user learning or extensive system calibration.

  View details for DOI 10.1088/1741-2552/aa9ee7

  View details for PubMedID 29363625

• Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis. Neuron Williams, A. H., Kim, T. H., Wang, F. n., Vyas, S. n., Ryu, S. I., Shenoy, K. V., Schnitzer, M. n., Kolda, T. G., Ganguli, S. n. 2018

  Abstract

  Perceptions, thoughts, and actions unfold over millisecond timescales, while learned behaviors can require many days to mature. While recent experimental advances enable large-scale and long-term neural recordings with high temporal fidelity, it remains a formidable challenge to extract unbiased and interpretable descriptions of how rapid single-trial circuit dynamics change slowly over many trials to mediate learning. We demonstrate a simple tensor component analysis (TCA) can meet this challenge by extracting three interconnected, low-dimensional descriptions of neural data: neuron factors, reflecting cell assemblies; temporal factors, reflecting rapid circuit dynamics mediating perceptions, thoughts, and actions within each trial; and trial factors, describing both long-term learning and trial-to-trial changes in cognitive state. We demonstrate the broad applicability of TCA by revealing insights into diverse datasets derived from artificial neural networks, large-scale calcium imaging of rodent prefrontal cortex during maze navigation, and multielectrode recordings of macaque motor cortex during brain machine interface learning.

  View details for PubMedID 29887338

• Laminar differences in decision-related neural activity in dorsal premotor cortex NATURE COMMUNICATIONS Chandrasekaran, C., Peixoto, D., Newsome, W. T., Shenoy, K. V. 2017; 8: 614

  Abstract

  Dorsal premotor cortex is implicated in somatomotor decisions. However, we do not understand the temporal patterns and laminar organization of decision-related firing rates in dorsal premotor cortex. We recorded neurons from dorsal premotor cortex of monkeys performing a visual discrimination task with reaches as the behavioral report. We show that these neurons can be organized along a bidirectional visuomotor continuum based on task-related firing rates. "Increased" neurons at one end of the continuum increased their firing rates ~150 ms after stimulus onset and these firing rates covaried systematically with choice, stimulus difficulty, and reaction time-characteristics of a candidate decision variable. "Decreased" neurons at the other end of the continuum reduced their firing rate after stimulus onset, while "perimovement" neurons at the center of the continuum responded only ~150 ms before movement initiation. These neurons did not show decision variable-like characteristics. "Increased" neurons were more prevalent in superficial layers of dorsal premotor cortex; deeper layers contained more "decreased" and "perimovement" neurons. These results suggest a laminar organization for decision-related responses in dorsal premotor cortex.Dorsal premotor cortex (PMd) is thought to be involved in making somatomotor decisions. Chandrasekaran et al. investigated the temporal response dynamics of PMd neurons across cortical layers and show stronger and earlier decision-related responses in the superficial layers and more action execution-related signals in the deeper layers.

  View details for PubMedID 28931803

• A High-Performance Neural Prosthesis Incorporating Discrete State Selection With Hidden Markov Models IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2017; 64 (4): 935-945

  Abstract

  Communication neural prostheses aim to restore efficient communication to people with motor neurological injury or disease by decoding neural activity into control signals. These control signals are both analog (e.g., the velocity of a computer mouse) and discrete (e.g., clicking an icon with a computer mouse) in nature. Effective, high-performing, and intuitive-to-use communication prostheses should be capable of decoding both analog and discrete state variables seamlessly. However, to date, the highest-performing autonomous communication prostheses rely on precise analog decoding and typically do not incorporate high-performance discrete decoding. In this report, we incorporated a hidden Markov model (HMM) into an intracortical communication prosthesis to enable accurate and fast discrete state decoding in parallel with analog decoding. In closed-loop experiments with nonhuman primates implanted with multielectrode arrays, we demonstrate that incorporating an HMM into a neural prosthesis can increase state-of-the-art achieved bitrate by 13.9% and 4.2% in two monkeys ( ). We found that the transition model of the HMM is critical to achieving this performance increase. Further, we found that using an HMM resulted in the highest achieved peak performance we have ever observed for these monkeys, achieving peak bitrates of 6.5, 5.7, and 4.7 bps in Monkeys J, R, and L, respectively. Finally, we found that this neural prosthesis was robustly controllable for the duration of entire experimental sessions. These results demonstrate that high-performance discrete decoding can be beneficially combined with analog decoding to achieve new state-of-the-art levels of performance.

  View details for DOI 10.1109/TBME.2016.2582691

  View details for Web of Science ID 000398738300020

• High performance communication by people with paralysis using an intracortical brain-computer interface. eLife Pandarinath, C., Nuyujukian, P., Blabe, C. H., Sorice, B. L., Saab, J., Willett, F. R., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2017; 6

  Abstract

  Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O'Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4-4.2) and information throughput (by a factor of 2.2-4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.Clinical Trial No: NCT00912041.

  View details for DOI 10.7554/eLife.18554

  View details for PubMedID 28220753

• Trial-by-Trial Motor Cortical Correlates of a Rapidly Adapting Visuomotor Internal Model. journal of neuroscience Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 37 (7): 1721-1732

  Abstract

  Accurate motor control is mediated by internal models of how neural activity generates movement. We examined neural correlates of an adapting internal model of visuomotor gain in motor cortex while two macaques performed a reaching task in which the gain scaling between the hand and a presented cursor was varied. Previous studies of cortical changes during visuomotor adaptation focused on preparatory and perimovement epochs and analyzed trial-averaged neural data. Here, we recorded simultaneous neural population activity using multielectrode arrays and focused our analysis on neural differences in the period before the target appeared. We found that we could estimate the monkey's internal model of the gain using the neural population state during this pretarget epoch. This neural correlate depended on the gain experienced during recent trials and it predicted the speed of the subsequent reach. To explore the utility of this internal model estimate for brain-machine interfaces, we performed an offline analysis showing that it can be used to compensate for upcoming reach extent errors. Together, these results demonstrate that pretarget neural activity in motor cortex reflects the monkey's internal model of visuomotor gain on single trials and can potentially be used to improve neural prostheses.SIGNIFICANCE STATEMENT When generating movement commands, the brain is believed to use internal models of the relationship between neural activity and the body's movement. Visuomotor adaptation tasks have revealed neural correlates of these computations in multiple brain areas during movement preparation and execution. Here, we describe motor cortical changes in a visuomotor gain change task even before a specific movement is cued. We were able to estimate the gain internal model from these pretarget neural correlates and relate it to single-trial behavior. This is an important step toward understanding the sensorimotor system's algorithms for updating its internal models after specific movements and errors. Furthermore, the ability to estimate the internal model before movement could improve motor neural prostheses being developed for people with paralysis.

  View details for DOI 10.1523/JNEUROSCI.1091-16.2016

  View details for PubMedID 28087767

  View details for PubMedCentralID PMC5320605

• Feedback control policies employed by people using intracortical brain-computer interfaces JOURNAL OF NEURAL ENGINEERING Willett, F. R., Pandarinath, C., Jarosiewicz, B., Murphy, B. A., Memberg, W. D., Blabe, C. H., Saab, J., Walter, B. L., Sweet, J. A., Miller, J. P., Henderson, J. M., Shenoy, K. V., Simeral, J. D., Hochberg, L. R., Kirsch, R. F., Ajiboye, A. B. 2017; 14 (1)

  Abstract

  When using an intracortical BCI (iBCI), users modulate their neural population activity to move an effector towards a target, stop accurately, and correct for movement errors. We call the rules that govern this modulation a 'feedback control policy'. A better understanding of these policies may inform the design of higher-performing neural decoders.We studied how three participants in the BrainGate2 pilot clinical trial used an iBCI to control a cursor in a 2D target acquisition task. Participants used a velocity decoder with exponential smoothing dynamics. Through offline analyses, we characterized the users' feedback control policies by modeling their neural activity as a function of cursor state and target position. We also tested whether users could adapt their policy to different decoder dynamics by varying the gain (speed scaling) and temporal smoothing parameters of the iBCI.We demonstrate that control policy assumptions made in previous studies do not fully describe the policies of our participants. To account for these discrepancies, we propose a new model that captures (1) how the user's neural population activity gradually declines as the cursor approaches the target from afar, then decreases more sharply as the cursor comes into contact with the target, (2) how the user makes constant feedback corrections even when the cursor is on top of the target, and (3) how the user actively accounts for the cursor's current velocity to avoid overshooting the target. Further, we show that users can adapt their control policy to decoder dynamics by attenuating neural modulation when the cursor gain is high and by damping the cursor velocity more strongly when the smoothing dynamics are high.Our control policy model may help to build better decoders, understand how neural activity varies during active iBCI control, and produce better simulations of closed-loop iBCI movements.

  View details for DOI 10.1088/1741-2560/14/1/016001

  View details for Web of Science ID 000390362600001

  View details for PubMedID 27900953

  View details for PubMedCentralID PMC5239755

• Corrigendum: Making brain-machine interfaces robust to future neural variability. Nature communications Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 8: 14490-?

  View details for DOI 10.1038/ncomms14490

  View details for PubMedID 28106034

  View details for PubMedCentralID PMC5263867

• A Nonhuman Primate Brain-Computer Typing Interface PROCEEDINGS OF THE IEEE Nuyujukian, P., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 105 (1): 66-72

  View details for DOI 10.1109/JPROC.2016.2586967

  View details for Web of Science ID 000391703100006

• A Non-Human Primate Brain-Computer Typing Interface. Proceedings of the IEEE. Institute of Electrical and Electronics Engineers Nuyujukian, P., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 105 (1): 66-72

  Abstract

  Brain-computer interfaces (BCIs) record brain activity and translate the information into useful control signals. They can be used to restore function to people with paralysis by controlling end effectors such as computer cursors and robotic limbs. Communication neural prostheses are BCIs that control user interfaces on computers or mobile devices. Here we demonstrate a communication prosthesis by simulating a typing task with two rhesus macaques implanted with electrode arrays. The monkeys used two of the highest known performing BCI decoders to type out words and sentences when prompted one symbol/letter at a time. On average, Monkeys J and L achieved typing rates of 10.0 and 7.2 words per minute (wpm), respectively, copying text from a newspaper article using a velocity-only two dimensional BCI decoder with dwell-based symbol selection. With a BCI decoder that also featured a discrete click for key selection, typing rates increased to 12.0 and 7.8 wpm. These represent the highest known achieved communication rates using a BCI. We then quantified the relationship between bitrate and typing rate and found it approximately linear: typing rate in wpm is nearly three times bitrate in bits per second. We also compared the metrics of achieved bitrate and information transfer rate and discuss their applicability to real-world typing scenarios. Although this study cannot model the impact of cognitive load of word and sentence planning, the findings here demonstrate the feasibility of BCIs to serve as communication interfaces and represent an upper bound on the expected achieved typing rate for a given BCI throughput.

  View details for DOI 10.1109/JPROC.2016.2586967

  View details for PubMedID 33746239

  View details for PubMedCentralID PMC7970827

• Leveraging neural dynamics to extend functional lifetime of brain-machine interfaces. Scientific reports Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 7 (1): 7395

  Abstract

  Intracortical brain-machine interfaces (BMIs) aim to restore lost motor function to people with neurological deficits by decoding neural activity into control signals for guiding prostheses. An important challenge facing BMIs is that, over time, the number of neural signals recorded from implanted multielectrode arrays will decline and result in a concomitant decrease of BMI performance. We sought to extend BMI lifetime by developing an algorithmic technique, implemented entirely in software, to improve performance over state-of-the-art algorithms as the number of recorded neural signals decline. Our approach augments the decoder by incorporating neural population dynamics remembered from an earlier point in the array lifetime. We demonstrate, in closed-loop experiments with two rhesus macaques, that after the loss of approximately 60% of recording electrodes, our approach outperforms state-of-the-art decoders by a factor of 3.2× and 1.7× (corresponding to a 46% and 22% recovery of maximal performance). Further, our results suggest that neural population dynamics in motor cortex are invariant to the number of recorded neurons. By extending functional BMI lifetime, this approach increases the clinical viability of BMIs.

  View details for PubMedID 28784984

• Motor Cortical Visuomotor Feedback Activity Is Initially Isolated from Downstream Targets in Output-Null Neural State Space Dimensions. Neuron Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 95 (1): 195–208.e9

  Abstract

  Neural circuits must transform new inputs into outputs without prematurely affecting downstream circuits while still maintaining other ongoing communication with these targets. We investigated how this isolation is achieved in the motor cortex when macaques received visual feedback signaling a movement perturbation. To overcome limitations in estimating the mapping from cortex to arm movements, we also conducted brain-machine interface (BMI) experiments where we could definitively identify neural firing patterns as output-null or output-potent. This revealed that perturbation-evoked responses were initially restricted to output-null patterns that cancelled out at the neural population code readout and only later entered output-potent neural dimensions. This mechanism was facilitated by the circuit's large null space and its ability to strongly modulate output-potent dimensions when generating corrective movements. These results show that the nervous system can temporarily isolate portions of a circuit's activity from its downstream targets by restricting this activity to the circuit's output-null neural dimensions.

  View details for PubMedID 28625485

• Feasibility Analysis of Genetically-Encoded Calcium Indicators as a Neural Signal Source for All-Optical Brain-Machine Interfaces Sun, X., Kao, J. C., Marshel, J. H., Ryu, S. I., Shenoy, K. V., IEEE IEEE. 2017: 174–80

  View details for Web of Science ID 000428143200042

• Augmenting intracortical brain-machine interface with neurally driven error detectors. Journal of neural engineering Even-Chen, N. n., Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2017; 14 (6): 066007

  Abstract

  Making mistakes is inevitable, but identifying them allows us to correct or adapt our behavior to improve future performance. Current brain-machine interfaces (BMIs) make errors that need to be explicitly corrected by the user, thereby consuming time and thus hindering performance. We hypothesized that neural correlates of the user perceiving the mistake could be used by the BMI to automatically correct errors. However, it was unknown whether intracortical outcome error signals were present in the premotor and primary motor cortices, brain regions successfully used for intracortical BMIs.We report here for the first time a putative outcome error signal in spiking activity within these cortices when rhesus macaques performed an intracortical BMI computer cursor task.We decoded BMI trial outcomes shortly after and even before a trial ended with 96% and 84% accuracy, respectively. This led us to develop and implement in real-time a first-of-its-kind intracortical BMI error 'detect-and-act' system that attempts to automatically 'undo' or 'prevent' mistakes. The detect-and-act system works independently and in parallel to a kinematic BMI decoder. In a challenging task that resulted in substantial errors, this approach improved the performance of a BMI employing two variants of the ubiquitous Kalman velocity filter, including a state-of-the-art decoder (ReFIT-KF).Detecting errors in real-time from the same brain regions that are commonly used to control BMIs should improve the clinical viability of BMIs aimed at restoring motor function to people with paralysis.

  View details for PubMedID 29130452

  View details for PubMedCentralID PMC5742283

• Signal-independent noise in intracortical brain-computer interfaces causes movement time properties inconsistent with Fitts' law. Journal of neural engineering Willett, F. R., Murphy, B. A., Memberg, W. D., Blabe, C. H., Pandarinath, C. n., Walter, B. L., Sweet, J. A., Miller, J. P., Henderson, J. M., Shenoy, K. V., Hochberg, L. R., Kirsch, R. F., Ajiboye, A. B. 2017; 14 (2): 026010

  Abstract

  Do movements made with an intracortical BCI (iBCI) have the same movement time properties as able-bodied movements? Able-bodied movement times typically obey Fitts' law: [Formula: see text] (where MT is movement time, D is target distance, R is target radius, and [Formula: see text] are parameters). Fitts' law expresses two properties of natural movement that would be ideal for iBCIs to restore: (1) that movement times are insensitive to the absolute scale of the task (since movement time depends only on the ratio [Formula: see text]) and (2) that movements have a large dynamic range of accuracy (since movement time is logarithmically proportional to [Formula: see text]).Two participants in the BrainGate2 pilot clinical trial made cortically controlled cursor movements with a linear velocity decoder and acquired targets by dwelling on them. We investigated whether the movement times were well described by Fitts' law.We found that movement times were better described by the equation [Formula: see text], which captures how movement time increases sharply as the target radius becomes smaller, independently of distance. In contrast to able-bodied movements, the iBCI movements we studied had a low dynamic range of accuracy (absence of logarithmic proportionality) and were sensitive to the absolute scale of the task (small targets had long movement times regardless of the [Formula: see text] ratio). We argue that this relationship emerges due to noise in the decoder output whose magnitude is largely independent of the user's motor command (signal-independent noise). Signal-independent noise creates a baseline level of variability that cannot be decreased by trying to move slowly or hold still, making targets below a certain size very hard to acquire with a standard decoder.The results give new insight into how iBCI movements currently differ from able-bodied movements and suggest that restoring a Fitts' law-like relationship to iBCI movements may require non-linear decoding strategies.

  View details for DOI 10.1088/1741-2552/aa5990

  View details for PubMedID 28177925

• The need for calcium imaging in nonhuman primates: New motor neuroscience and brain-machine interfaces EXPERIMENTAL NEUROLOGY O'Shea, D. J., Tiautmann, E., Chandrasekaran, C., Stavisky, S., Kao, J. C., Sahani, M., Ryu, S., Deisseroth, K., Shenoy, K. V. 2017; 287: 437-451

  Abstract

  A central goal of neuroscience is to understand how populations of neurons coordinate and cooperate in order to give rise to perception, cognition, and action. Nonhuman primates (NHPs) are an attractive model with which to understand these mechanisms in humans, primarily due to the strong homology of their brains and the cognitively sophisticated behaviors they can be trained to perform. Using electrode recordings, the activity of one to a few hundred individual neurons may be measured electrically, which has enabled many scientific findings and the development of brain-machine interfaces. Despite these successes, electrophysiology samples sparsely from neural populations and provides little information about the genetic identity and spatial micro-organization of recorded neurons. These limitations have spurred the development of all-optical methods for neural circuit interrogation. Fluorescent calcium signals serve as a reporter of neuronal responses, and when combined with post-mortem optical clearing techniques such as CLARITY, provide dense recordings of neuronal populations, spatially organized and annotated with genetic and anatomical information. Here, we advocate that this methodology, which has been of tremendous utility in smaller animal models, can and should be developed for use with NHPs. We review here several of the key opportunities and challenges for calcium-based optical imaging in NHPs. We focus on motor neuroscience and brain-machine interface design as representative domains of opportunity within the larger field of NHP neuroscience.

  View details for DOI 10.1016/j.expneurol.2016.08.003

  View details for Web of Science ID 000391158800002

  View details for PubMedCentralID PMC5154795

• Making brain-machine interfaces robust to future neural variability NATURE COMMUNICATIONS Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2016; 7

  Abstract

  A major hurdle to clinical translation of brain-machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change. We tested whether a decoder could be made more robust to future neural variability by training it to handle a variety of recording conditions sampled from months of previously collected data as well as synthetic training data perturbations. We developed a new multiplicative recurrent neural network BMI decoder that successfully learned a large variety of neural-to-kinematic mappings and became more robust with larger training data sets. Here we demonstrate that when tested with a non-human primate preclinical BMI model, this decoder is robust under conditions that disabled a state-of-the-art Kalman filter-based decoder. These results validate a new BMI strategy in which accumulated data history are effectively harnessed, and may facilitate reliable BMI use by reducing decoder retraining downtime.

  View details for DOI 10.1038/ncomms13749

  View details for Web of Science ID 000389627100001

  View details for PubMedID 27958268

  View details for PubMedCentralID PMC5159828

• The Importance of Planning in Motor Learning. Neuron O'Shea, D. J., Shenoy, K. V. 2016; 92 (4): 669-671

  Abstract

  The addition of differentiating follow-through motions can facilitate simultaneous learning of multiple motor skills that would otherwise interfere with each other. In this issue of Neuron, Sheahan and colleagues (2016) demonstrate that it is the preparation, not execution, of different follow-through movements that separates motor memories and reduces interference.

  View details for DOI 10.1016/j.neuron.2016.11.003

  View details for PubMedID 27883896

• Tensor Analysis Reveals Distinct Population Structure that Parallels the Different Computational Roles of Areas M1 and V1. PLoS computational biology Seely, J. S., Kaufman, M. T., Ryu, S. I., Shenoy, K. V., Cunningham, J. P., Churchland, M. M. 2016; 12 (11)

  Abstract

  Cortical firing rates frequently display elaborate and heterogeneous temporal structure. One often wishes to compute quantitative summaries of such structure-a basic example is the frequency spectrum-and compare with model-based predictions. The advent of large-scale population recordings affords the opportunity to do so in new ways, with the hope of distinguishing between potential explanations for why responses vary with time. We introduce a method that assesses a basic but previously unexplored form of population-level structure: when data contain responses across multiple neurons, conditions, and times, they are naturally expressed as a third-order tensor. We examined tensor structure for multiple datasets from primary visual cortex (V1) and primary motor cortex (M1). All V1 datasets were 'simplest' (there were relatively few degrees of freedom) along the neuron mode, while all M1 datasets were simplest along the condition mode. These differences could not be inferred from surface-level response features. Formal considerations suggest why tensor structure might differ across modes. For idealized linear models, structure is simplest across the neuron mode when responses reflect external variables, and simplest across the condition mode when responses reflect population dynamics. This same pattern was present for existing models that seek to explain motor cortex responses. Critically, only dynamical models displayed tensor structure that agreed with the empirical M1 data. These results illustrate that tensor structure is a basic feature of the data. For M1 the tensor structure was compatible with only a subset of existing models.

  View details for DOI 10.1371/journal.pcbi.1005164

  View details for PubMedID 27814353

  View details for PubMedCentralID PMC5096707

• The need for calcium imaging in nonhuman primates: New motor neuroscience and brain-machine interfaces. Experimental neurology O'Shea, D. J., Trautmann, E., Chandrasekaran, C., Stavisky, S., Kao, J. C., Sahani, M., Ryu, S., Deisseroth, K., Shenoy, K. V. 2016

  Abstract

  A central goal of neuroscience is to understand how populations of neurons coordinate and cooperate in order to give rise to perception, cognition, and action. Nonhuman primates (NHPs) are an attractive model with which to understand these mechanisms in humans, primarily due to the strong homology of their brains and the cognitively sophisticated behaviors they can be trained to perform. Using electrode recordings, the activity of one to a few hundred individual neurons may be measured electrically, which has enabled many scientific findings and the development of brain-machine interfaces. Despite these successes, electrophysiology samples sparsely from neural populations and provides little information about the genetic identity and spatial micro-organization of recorded neurons. These limitations have spurred the development of all-optical methods for neural circuit interrogation. Fluorescent calcium signals serve as a reporter of neuronal responses, and when combined with post-mortem optical clearing techniques such as CLARITY, provide dense recordings of neuronal populations, spatially organized and annotated with genetic and anatomical information. Here, we advocate that this methodology, which has been of tremendous utility in smaller animal models, can and should be developed for use with NHPs. We review here several of the key opportunities and challenges for calcium-based optical imaging in NHPs. We focus on motor neuroscience and brain-machine interface design as representative domains of opportunity within the larger field of NHP neuroscience.

  View details for DOI 10.1016/j.expneurol.2016.08.003

  View details for PubMedID 27511294

  View details for PubMedCentralID PMC5154795

• The Largest Response Component in the Motor Cortex Reflects Movement Timing but Not Movement Type. eNeuro Kaufman, M. T., Seely, J. S., Sussillo, D., Ryu, S. I., Shenoy, K. V., Churchland, M. M. 2016; 3 (4)

  Abstract

  Neural activity in monkey motor cortex (M1) and dorsal premotor cortex (PMd) can reflect a chosen movement well before that movement begins. The pattern of neural activity then changes profoundly just before movement onset. We considered the prediction, derived from formal considerations, that the transition from preparation to movement might be accompanied by a large overall change in the neural state that reflects when movement is made rather than which movement is made. Specifically, we examined "components" of the population response: time-varying patterns of activity from which each neuron's response is approximately composed. Amid the response complexity of individual M1 and PMd neurons, we identified robust response components that were "condition-invariant": their magnitude and time course were nearly identical regardless of reach direction or path. These condition-invariant response components occupied dimensions orthogonal to those occupied by the "tuned" response components. The largest condition-invariant component was much larger than any of the tuned components; i.e., it explained more of the structure in individual-neuron responses. This condition-invariant response component underwent a rapid change before movement onset. The timing of that change predicted most of the trial-by-trial variance in reaction time. Thus, although individual M1 and PMd neurons essentially always reflected which movement was made, the largest component of the population response reflected movement timing rather than movement type.

  View details for PubMedID 27761519

• A high-performance neural prosthesis incorporating discrete state selection with hidden Markov models. IEEE transactions on bio-medical engineering Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2016: -?

  Abstract

  Communication neural prostheses aim to restore efficient communication to people with motor neurological injury or disease by decoding neural activity into control signals. These control signals are both analog (e.g., the velocity of a computer mouse) and discrete (e.g., clicking an icon with a computer mouse) in nature. Effective, high-performing, and intuitive-to-use communication prostheses should be capable of decoding both analog and discrete state variables seamlessly. However, to date, the highest-performing autonomous communication prostheses rely on precise analog decoding and typically do not incorporate high-performance discrete decoding. In this report, we incorporated a hidden Markov model (HMM) into an intracortical communication prosthesis to enable accurate and fast discrete state decoding in parallel with analog decoding. In closed-loop experiments with nonhuman primates implanted with multielectrode arrays, we demonstrate that incorporating an HMM into a neural prosthesis can increase state-of-the-art achieved bitrate by 13.9% and 4.2% in two monkeys ( ). We found that the transition model of the HMM is critical to achieving this performance increase. Further, we found that using an HMM resulted in the highest achieved peak performance we have ever observed for these monkeys, achieving peak bitrates of 6.5, 5.7, and 4.7 bps in Monkeys J, R, and L, respectively. Finally, we found that this neural prosthesis was robustly controllable for the duration of entire experimental sessions. These results demonstrate that high-performance discrete decoding can be beneficially combined with analog decoding to achieve new state-of-the-art levels of performance.

  View details for PubMedID 27337709

• Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Science translational medicine Jarosiewicz, B., Sarma, A. A., Bacher, D., Masse, N. Y., Simeral, J. D., Sorice, B., Oakley, E. M., Blabe, C., Pandarinath, C., Gilja, V., Cash, S. S., Eskandar, E. N., Friehs, G., Henderson, J. M., Shenoy, K. V., Donoghue, J. P., Hochberg, L. R. 2015; 7 (313): 313ra179-?

  Abstract

  Brain-computer interfaces (BCIs) promise to restore independence for people with severe motor disabilities by translating decoded neural activity directly into the control of a computer. However, recorded neural signals are not stationary (that is, can change over time), degrading the quality of decoding. Requiring users to pause what they are doing whenever signals change to perform decoder recalibration routines is time-consuming and impractical for everyday use of BCIs. We demonstrate that signal nonstationarity in an intracortical BCI can be mitigated automatically in software, enabling long periods (hours to days) of self-paced point-and-click typing by people with tetraplegia, without degradation in neural control. Three key innovations were included in our approach: tracking the statistics of the neural activity during self-timed pauses in neural control, velocity bias correction during neural control, and periodically recalibrating the decoder using data acquired during typing by mapping neural activity to movement intentions that are inferred retrospectively based on the user's self-selected targets. These methods, which can be extended to a variety of neurally controlled applications, advance the potential for intracortical BCIs to help restore independent communication and assistive device control for people with paralysis.

  View details for DOI 10.1126/scitranslmed.aac7328

  View details for PubMedID 26560357

• Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Science translational medicine Jarosiewicz, B., Sarma, A. A., Bacher, D., Masse, N. Y., Simeral, J. D., Sorice, B., Oakley, E. M., Blabe, C., Pandarinath, C., Gilja, V., Cash, S. S., Eskandar, E. N., Friehs, G., Henderson, J. M., Shenoy, K. V., Donoghue, J. P., Hochberg, L. R. 2015; 7 (313): 313ra179-?

  View details for DOI 10.1126/scitranslmed.aac7328

  View details for PubMedID 26560357

• Clinical translation of a high-performance neural prosthesis. Nature medicine Gilja, V., Pandarinath, C., Blabe, C. H., Nuyujukian, P., Simeral, J. D., Sarma, A. A., Sorice, B. L., Perge, J. A., Jarosiewicz, B., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2015; 21 (10): 1142-1145

  Abstract

  Neural prostheses have the potential to improve the quality of life of individuals with paralysis by directly mapping neural activity to limb- and computer-control signals. We translated a neural prosthetic system previously developed in animal model studies for use by two individuals with amyotrophic lateral sclerosis who had intracortical microelectrode arrays placed in motor cortex. Measured more than 1 year after implant, the neural cursor-control system showed the highest published performance achieved by a person to date, more than double that of previous pilot clinical trial participants.

  View details for DOI 10.1038/nm.3953

  View details for PubMedID 26413781

• Clinical translation of a high-performance neural prosthesis NATURE MEDICINE Gilja, V., Pandarinath, C., Blabe, C. H., Nuyujukian, P., Simeral, J. D., Sarma, A. A., Sorice, B. L., Perge, J. A., Jarosiewicz, B., Hochberg, L. R., Shenoy, K. V., Henderson, J. M. 2015; 21 (10): 1142-?

  Abstract

  Neural prostheses have the potential to improve the quality of life of individuals with paralysis by directly mapping neural activity to limb- and computer-control signals. We translated a neural prosthetic system previously developed in animal model studies for use by two individuals with amyotrophic lateral sclerosis who had intracortical microelectrode arrays placed in motor cortex. Measured more than 1 year after implant, the neural cursor-control system showed the highest published performance achieved by a person to date, more than double that of previous pilot clinical trial participants.

  View details for DOI 10.1038/nm.3953

  View details for Web of Science ID 000362355400015

  View details for PubMedID 26413781

• Optogenetics: 10 years after ChR2 in neurons-views from the community NATURE NEUROSCIENCE Adamantidis, A., Arber, S., Bains, J. S., Bamberg, E., Bonci, A., Buzsaki, G., Cardin, J. A., Costa, R. M., Dan, Y., Goda, Y., Graybiel, A. M., Haeusser, M., Hegemann, P., Huguenard, J. R., Insel, T. R., Janak, P. H., Johnston, D., Josselyn, S. A., Koch, C., Kreitzer, A. C., Luescher, C., Malenka, R. C., Miesenboeck, G., Nagel, G., Roska, B., Schnitzer, M. J., Shenoy, K. V., Soltesz, I., Sternson, S. M., Tsien, R. W., Tsien, R. Y., Turrigiano, G. G., Tye, K. M., Wilson, R. I. 2015; 18 (9): 1202–12

  View details for PubMedID 26308981

• Assessment of brain-machine interfaces from the perspective of people with paralysis. Journal of neural engineering Blabe, C. H., Gilja, V., Chestek, C. A., Shenoy, K. V., Anderson, K. D., Henderson, J. M. 2015; 12 (4): 043002-?

  Abstract

  One of the main goals of brain-machine interface (BMI) research is to restore function to people with paralysis. Currently, multiple BMI design features are being investigated, based on various input modalities (externally applied and surgically implantable sensors) and output modalities (e.g. control of computer systems, prosthetic arms, and functional electrical stimulation systems). While these technologies may eventually provide some level of benefit, they each carry associated burdens for end-users. We sought to assess the attitudes of people with paralysis toward using various technologies to achieve particular benefits, given the burdens currently associated with the use of each system.We designed and distributed a technology survey to determine the level of benefit necessary for people with tetraplegia due to spinal cord injury to consider using different technologies, given the burdens currently associated with them. The survey queried user preferences for 8 BMI technologies including electroencephalography, electrocorticography, and intracortical microelectrode arrays, as well as a commercially available eye tracking system for comparison. Participants used a 5-point scale to rate their likelihood to adopt these technologies for 13 potential control capabilities.Survey respondents were most likely to adopt BMI technology to restore some of their natural upper extremity function, including restoration of hand grasp and/or some degree of natural arm movement. High speed typing and control of a fast robot arm were also of interest to this population. Surgically implanted wireless technologies were twice as 'likely' to be adopted as their wired equivalents.Assessing end-user preferences is an essential prerequisite to the design and implementation of any assistive technology. The results of this survey suggest that people with tetraplegia would adopt an unobtrusive, autonomous BMI system for both restoration of upper extremity function and control of external devices such as communication interfaces.

  View details for DOI 10.1088/1741-2560/12/4/043002

  View details for PubMedID 26169880

• Auto-deleting brain machine interface: Error detection using spiking neural activity in the motor cortex. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference Even-Chen, N., Stavisky, S. D., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2015; 2015: 71-75

  Abstract

  Brain machine interfaces (BMIs) aim to assist people with paralysis by increasing their independence and ability to communicate, e.g., by using a cursor-based virtual keyboard. Current BMI clinical trials are hampered by modest performance that causes selection of wrong characters (errors) and thus reduces achieved typing rate. If it were possible to detect these errors without explicit knowledge of the task goal, this could be used to automatically "undo" wrong selections or even prevent upcoming wrong selections. We decoded imminent or recent errors during closed-loop BMI control from intracortical spiking neural activity. In our experiment, a non-human primate controlled a neurally-driven BMI cursor to acquire targets on a grid, which simulates a virtual keyboard. In offline analyses of this closed-loop BMI control data, we identified motor cortical neural signals indicative of task error occurrence. We were able to detect task outcomes (97% accuracy) and even predict upcoming task outcomes (86% accuracy) using neural activity alone. This novel strategy may help increase the performance and clinical viability of BMIs.

  View details for DOI 10.1109/EMBC.2015.7318303

  View details for PubMedID 26736203

• Assessment of brain-machine interfaces from the perspective of people with paralysis JOURNAL OF NEURAL ENGINEERING Blabe, C. H., Gilja, V., Chestek, C. A., Shenoy, K. V., Anderson, K. D., Henderson, J. M. 2015; 12 (4)

  Abstract

  One of the main goals of brain-machine interface (BMI) research is to restore function to people with paralysis. Currently, multiple BMI design features are being investigated, based on various input modalities (externally applied and surgically implantable sensors) and output modalities (e.g. control of computer systems, prosthetic arms, and functional electrical stimulation systems). While these technologies may eventually provide some level of benefit, they each carry associated burdens for end-users. We sought to assess the attitudes of people with paralysis toward using various technologies to achieve particular benefits, given the burdens currently associated with the use of each system.We designed and distributed a technology survey to determine the level of benefit necessary for people with tetraplegia due to spinal cord injury to consider using different technologies, given the burdens currently associated with them. The survey queried user preferences for 8 BMI technologies including electroencephalography, electrocorticography, and intracortical microelectrode arrays, as well as a commercially available eye tracking system for comparison. Participants used a 5-point scale to rate their likelihood to adopt these technologies for 13 potential control capabilities.Survey respondents were most likely to adopt BMI technology to restore some of their natural upper extremity function, including restoration of hand grasp and/or some degree of natural arm movement. High speed typing and control of a fast robot arm were also of interest to this population. Surgically implanted wireless technologies were twice as 'likely' to be adopted as their wired equivalents.Assessing end-user preferences is an essential prerequisite to the design and implementation of any assistive technology. The results of this survey suggest that people with tetraplegia would adopt an unobtrusive, autonomous BMI system for both restoration of upper extremity function and control of external devices such as communication interfaces.

  View details for DOI 10.1088/1741-2560/12/4/043002

  View details for Web of Science ID 000358178900002

• Single-trial dynamics of motor cortex and their applications to brain-machine interfaces NATURE COMMUNICATIONS Kao, J. C., Nuyujukian, P., Ryu, S. I., Churchland, M. M., Cunningham, J. P., Shenoy, K. V. 2015; 6

  Abstract

  Increasing evidence suggests that neural population responses have their own internal drive, or dynamics, that describe how the neural population evolves through time. An important prediction of neural dynamical models is that previously observed neural activity is informative of noisy yet-to-be-observed activity on single-trials, and may thus have a denoising effect. To investigate this prediction, we built and characterized dynamical models of single-trial motor cortical activity. We find these models capture salient dynamical features of the neural population and are informative of future neural activity on single trials. To assess how neural dynamics may beneficially denoise single-trial neural activity, we incorporate neural dynamics into a brain-machine interface (BMI). In online experiments, we find that a neural dynamical BMI achieves substantially higher performance than its non-dynamical counterpart. These results provide evidence that neural dynamics beneficially inform the temporal evolution of neural activity on single trials and may directly impact the performance of BMIs.

  View details for DOI 10.1038/ncomms8759

  View details for Web of Science ID 000358858800003

• A neural network that finds a naturalistic solution for the production of muscle activity NATURE NEUROSCIENCE Sussillo, D., Churchland, M. M., Kaufman, M. T., Shenoy, K. V. 2015; 18 (7): 1025-?

  Abstract

  It remains an open question how neural responses in motor cortex relate to movement. We explored the hypothesis that motor cortex reflects dynamics appropriate for generating temporally patterned outgoing commands. To formalize this hypothesis, we trained recurrent neural networks to reproduce the muscle activity of reaching monkeys. Models had to infer dynamics that could transform simple inputs into temporally and spatially complex patterns of muscle activity. Analysis of trained models revealed that the natural dynamical solution was a low-dimensional oscillator that generated the necessary multiphasic commands. This solution closely resembled, at both the single-neuron and population levels, what was observed in neural recordings from the same monkeys. Notably, data and simulations agreed only when models were optimized to find simple solutions. An appealing interpretation is that the empirically observed dynamics of motor cortex may reflect a simple solution to the problem of generating temporally patterned descending commands.

  View details for DOI 10.1038/nn.4042

  View details for Web of Science ID 000356866200018

  View details for PubMedID 26075643

• Neural population dynamics in human motor cortex during movements in people with ALS ELIFE Pandarinath, C., Gilja, V., Blabe, C. H., Nuyujukian, P., Sarma, A. A., Sorice, B. L., Eskandar, E. N., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2015; 4

  Abstract

  The prevailing view of motor cortex holds that motor cortical neural activity represents muscle or movement parameters. However, recent studies in non-human primates have shown that neural activity does not simply represent muscle or movement parameters; instead, its temporal structure is well-described by a dynamical system where activity during movement evolves lawfully from an initial pre-movement state. In this study, we analyze neuronal ensemble activity in motor cortex in two clinical trial participants diagnosed with Amyotrophic Lateral Sclerosis (ALS). We find that activity in human motor cortex has similar dynamical structure to that of non-human primates, indicating that human motor cortex contains a similar underlying dynamical system for movement generation.

  View details for DOI 10.7554/eLife.07436

  View details for Web of Science ID 000356720100001

  View details for PubMedID 26099302

  View details for PubMedCentralID PMC4475900

• A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes. Journal of neural engineering Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2015; 12 (3): 036009-?

  Abstract

  Brain-machine interfaces (BMIs) seek to enable people with movement disabilities to directly control prosthetic systems with their neural activity. Current high performance BMIs are driven by action potentials (spikes), but access to this signal often diminishes as sensors degrade over time. Decoding local field potentials (LFPs) as an alternative or complementary BMI control signal may improve performance when there is a paucity of spike signals. To date only a small handful of LFP decoding methods have been tested online; there remains a need to test different LFP decoding approaches and improve LFP-driven performance. There has also not been a reported demonstration of a hybrid BMI that decodes kinematics from both LFP and spikes. Here we first evaluate a BMI driven by the local motor potential (LMP), a low-pass filtered time-domain LFP amplitude feature. We then combine decoding of both LMP and spikes to implement a hybrid BMI.Spikes and LFP were recorded from two macaques implanted with multielectrode arrays in primary and premotor cortex while they performed a reaching task. We then evaluated closed-loop BMI control using biomimetic decoders driven by LMP, spikes, or both signals together.LMP decoding enabled quick and accurate cursor control which surpassed previously reported LFP BMI performance. Hybrid decoding of both spikes and LMP improved performance when spikes signal quality was mediocre to poor.These findings show that LMP is an effective BMI control signal which requires minimal power to extract and can substitute for or augment impoverished spikes signals. Use of this signal may lengthen the useful lifespan of BMIs and is therefore an important step towards clinically viable BMIs.

  View details for DOI 10.1088/1741-2560/12/3/036009

  View details for PubMedID 25946198

  View details for PubMedCentralID PMC4457459

• A high performing brain-machine interface driven by low-frequency local field potentials alone and together with spikes JOURNAL OF NEURAL ENGINEERING Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2015; 12 (3)

  Abstract

  Brain-machine interfaces (BMIs) seek to enable people with movement disabilities to directly control prosthetic systems with their neural activity. Current high performance BMIs are driven by action potentials (spikes), but access to this signal often diminishes as sensors degrade over time. Decoding local field potentials (LFPs) as an alternative or complementary BMI control signal may improve performance when there is a paucity of spike signals. To date only a small handful of LFP decoding methods have been tested online; there remains a need to test different LFP decoding approaches and improve LFP-driven performance. There has also not been a reported demonstration of a hybrid BMI that decodes kinematics from both LFP and spikes. Here we first evaluate a BMI driven by the local motor potential (LMP), a low-pass filtered time-domain LFP amplitude feature. We then combine decoding of both LMP and spikes to implement a hybrid BMI.Spikes and LFP were recorded from two macaques implanted with multielectrode arrays in primary and premotor cortex while they performed a reaching task. We then evaluated closed-loop BMI control using biomimetic decoders driven by LMP, spikes, or both signals together.LMP decoding enabled quick and accurate cursor control which surpassed previously reported LFP BMI performance. Hybrid decoding of both spikes and LMP improved performance when spikes signal quality was mediocre to poor.These findings show that LMP is an effective BMI control signal which requires minimal power to extract and can substitute for or augment impoverished spikes signals. Use of this signal may lengthen the useful lifespan of BMIs and is therefore an important step towards clinically viable BMIs.

  View details for DOI 10.1088/1741-2560/12/3/036009

  View details for Web of Science ID 000354998600010

  View details for PubMedID 25946198

  View details for PubMedCentralID PMC4457459

• Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. Journal of neural engineering Christie, B. P., Tat, D. M., Irwin, Z. T., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Thompson, D. E., Chestek, C. A. 2015; 12 (1): 016009-?

  Abstract

  Objective. For intracortical brain-machine interfaces (BMIs), action potential voltage waveforms are often sorted to separate out individual neurons. If these neurons contain independent tuning information, this process could increase BMI performance. However, the sorting of action potentials ('spikes') requires high sampling rates and is computationally expensive. To explicitly define the difference between spike sorting and alternative methods, we quantified BMI decoder performance when using threshold-crossing events versus sorted action potentials. Approach. We used data sets from 58 experimental sessions from two rhesus macaques implanted with Utah arrays. Data were recorded while the animals performed a center-out reaching task with seven different angles. For spike sorting, neural signals were sorted into individual units by using a mixture of Gaussians to cluster the first four principal components of the waveforms. For thresholding events, spikes that simply crossed a set threshold were retained. We decoded the data offline using both a Naïve Bayes classifier for reaching direction and a linear regression to evaluate hand position. Main results. We found the highest performance for thresholding when placing a threshold between -3 and -4.5 × Vrms. Spike sorted data outperformed thresholded data for one animal but not the other. The mean Naïve Bayes classification accuracy for sorted data was 88.5% and changed by 5% on average when data were thresholded. The mean correlation coefficient for sorted data was 0.92, and changed by 0.015 on average when thresholded. Significance. For prosthetics applications, these results imply that when thresholding is used instead of spike sorting, only a small amount of performance may be lost. The utilization of threshold-crossing events may significantly extend the lifetime of a device because these events are often still detectable once single neurons are no longer isolated.

  View details for DOI 10.1088/1741-2560/12/1/016009

  View details for PubMedID 25504690

• Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. Journal of neural engineering Christie, B. P., Tat, D. M., Irwin, Z. T., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Thompson, D. E., Chestek, C. A. 2015; 12 (1): 016009-?

  Abstract

  Objective. For intracortical brain-machine interfaces (BMIs), action potential voltage waveforms are often sorted to separate out individual neurons. If these neurons contain independent tuning information, this process could increase BMI performance. However, the sorting of action potentials ('spikes') requires high sampling rates and is computationally expensive. To explicitly define the difference between spike sorting and alternative methods, we quantified BMI decoder performance when using threshold-crossing events versus sorted action potentials. Approach. We used data sets from 58 experimental sessions from two rhesus macaques implanted with Utah arrays. Data were recorded while the animals performed a center-out reaching task with seven different angles. For spike sorting, neural signals were sorted into individual units by using a mixture of Gaussians to cluster the first four principal components of the waveforms. For thresholding events, spikes that simply crossed a set threshold were retained. We decoded the data offline using both a Naïve Bayes classifier for reaching direction and a linear regression to evaluate hand position. Main results. We found the highest performance for thresholding when placing a threshold between -3 and -4.5 × Vrms. Spike sorted data outperformed thresholded data for one animal but not the other. The mean Naïve Bayes classification accuracy for sorted data was 88.5% and changed by 5% on average when data were thresholded. The mean correlation coefficient for sorted data was 0.92, and changed by 0.015 on average when thresholded. Significance. For prosthetics applications, these results imply that when thresholding is used instead of spike sorting, only a small amount of performance may be lost. The utilization of threshold-crossing events may significantly extend the lifetime of a device because these events are often still detectable once single neurons are no longer isolated.

  View details for DOI 10.1088/1741-2560/12/1/016009

  View details for PubMedID 25504690

• A High-Performance Keyboard Neural Prosthesis Enabled by Task Optimization IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Nuyujukian, P., Fan, J. M., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2015; 62 (1): 21-29

  Abstract

  Communication neural prostheses are an emerging class of medical devices that aim to restore efficient communication to people suffering from paralysis. These systems rely on an interface with the user, either via the use of a continuouslymoving cursor (e.g., mouse) or the discrete selection of symbols (e.g., keyboard). In developing these interfaces, many design choices have a significant impact on the performance of the system. The objective of this study was to explore the design choices of a continuously-moving cursor neural prosthesis and optimize the interface to maximize information theoretic performance. We swept interface parameters of two keyboard-like tasks to find task and subject specific optimal parameters as measured by achieved bitrate using two rhesus macaques implanted with multielectrode arrays. In this report, we present the highest performing free-paced neural prosthesis under any recording modality with sustainable communication rates of up to 3.5 bits per second (bps). These findings demonstrate that meaningful high performance can be achieved using an intracortical neural prosthesis, and that, when optimized, these systems may be appropriate for use as communication devices for those with physical disabilities.

  View details for DOI 10.1109/TBME.2014.2354697

  View details for Web of Science ID 000346765500003

  View details for PubMedID 25203982

• Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex. eLife Kaufman, M. T., Churchland, M. M., Ryu, S. I., Shenoy, K. V. 2015; 4

  Abstract

  When choosing actions, we can act decisively, vacillate, or suffer momentary indecision. Studying how individual decisions unfold requires moment-by-moment readouts of brain state. Here we provide such a view from dorsal premotor and primary motor cortex. Two monkeys performed a novel decision task while we recorded from many neurons simultaneously. We found that a decoder trained using 'forced choices' (one target viable) was highly reliable when applied to 'free choices'. However, during free choices internal events formed three categories. Typically, neural activity was consistent with rapid, unwavering choices. Sometimes, though, we observed presumed 'changes of mind': the neural state initially reflected one choice before changing to reflect the final choice. Finally, we observed momentary 'indecision': delay forming any clear motor plan. Further, moments of neural indecision accompanied moments of behavioral indecision. Together, these results reveal the rich and diverse set of internal events long suspected to occur during free choice.

  View details for DOI 10.7554/eLife.04677

  View details for PubMedID 25942352

  View details for PubMedCentralID PMC4415122

• Neural population dynamics in human motor cortex during movements in people with ALS. eLife Pandarinath, C., Gilja, V., Blabe, C. H., Nuyujukian, P., Sarma, A. A., Sorice, B. L., Eskandar, E. N., Hochberg, L. R., Henderson, J. M., Shenoy, K. V. 2015; 4

  Abstract

  The prevailing view of motor cortex holds that motor cortical neural activity represents muscle or movement parameters. However, recent studies in non-human primates have shown that neural activity does not simply represent muscle or movement parameters; instead, its temporal structure is well-described by a dynamical system where activity during movement evolves lawfully from an initial pre-movement state. In this study, we analyze neuronal ensemble activity in motor cortex in two clinical trial participants diagnosed with Amyotrophic Lateral Sclerosis (ALS). We find that activity in human motor cortex has similar dynamical structure to that of non-human primates, indicating that human motor cortex contains a similar underlying dynamical system for movement generation.

  View details for DOI 10.7554/eLife.07436

  View details for PubMedID 26099302

• Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nature communications Kao, J. C., Nuyujukian, P., Ryu, S. I., Churchland, M. M., Cunningham, J. P., Shenoy, K. V. 2015; 6: 7759-?

  Abstract

  Increasing evidence suggests that neural population responses have their own internal drive, or dynamics, that describe how the neural population evolves through time. An important prediction of neural dynamical models is that previously observed neural activity is informative of noisy yet-to-be-observed activity on single-trials, and may thus have a denoising effect. To investigate this prediction, we built and characterized dynamical models of single-trial motor cortical activity. We find these models capture salient dynamical features of the neural population and are informative of future neural activity on single trials. To assess how neural dynamics may beneficially denoise single-trial neural activity, we incorporate neural dynamics into a brain-machine interface (BMI). In online experiments, we find that a neural dynamical BMI achieves substantially higher performance than its non-dynamical counterpart. These results provide evidence that neural dynamics beneficially inform the temporal evolution of neural activity on single trials and may directly impact the performance of BMIs.

  View details for DOI 10.1038/ncomms8759

  View details for PubMedID 26220660

• Performance sustaining intracortical neural prostheses. Journal of neural engineering Nuyujukian, P., Kao, J. C., Fan, J. M., Stavisky, S. D., Ryu, S. I., Shenoy, K. V. 2014; 11 (6): 066003-?

  Abstract

  Objective. Neural prostheses, or brain-machine interfaces, aim to restore efficient communication and movement ability to those suffering from paralysis. A major challenge these systems face is robust performance, particularly with aging signal sources. The aim in this study was to develop a neural prosthesis that could sustain high performance in spite of signal instability while still minimizing retraining time. Approach. We trained two rhesus macaques implanted with intracortical microelectrode arrays 1-4 years prior to this study to acquire targets with a neurally-controlled cursor. We measured their performance via achieved bitrate (bits per second, bps). This task was repeated over contiguous days to evaluate the sustained performance across time. Main results. We found that in the monkey with a younger (i.e., two year old) implant and better signal quality, a fixed decoder could sustain performance for a month at a rate of 4 bps, the highest achieved communication rate reported to date. This fixed decoder was evaluated across 22 months and experienced a performance decline at a rate of 0.24 bps yr(-1). In the monkey with the older (i.e., 3.5 year old) implant and poorer signal quality, a fixed decoder could not sustain performance for more than a few days. Nevertheless, performance in this monkey was maintained for two weeks without requiring additional online retraining time by utilizing prior days' experimental data. Upon analysis of the changes in channel tuning, we found that this stability appeared partially attributable to the cancelling-out of neural tuning fluctuations when projected to two-dimensional cursor movements. Significance. The findings in this study (1) document the highest-performing communication neural prosthesis in monkeys, (2) confirm and extend prior reports of the stability of fixed decoders, and (3) demonstrate a protocol for system stability under conditions where fixed decoders would otherwise fail. These improvements to decoder stability are important for minimizing training time and should make neural prostheses more practical to use.

  View details for DOI 10.1088/1741-2560/11/6/066003

  View details for PubMedID 25307561

• Performance sustaining intracortical neural prostheses JOURNAL OF NEURAL ENGINEERING Nuyujukian, P., Kao, J. C., Fan, J. M., Stavisky, S. D., Ryu, S. I., Shenoy, K. V. 2014; 11 (6)

  Abstract

  Objective. Neural prostheses, or brain-machine interfaces, aim to restore efficient communication and movement ability to those suffering from paralysis. A major challenge these systems face is robust performance, particularly with aging signal sources. The aim in this study was to develop a neural prosthesis that could sustain high performance in spite of signal instability while still minimizing retraining time. Approach. We trained two rhesus macaques implanted with intracortical microelectrode arrays 1-4 years prior to this study to acquire targets with a neurally-controlled cursor. We measured their performance via achieved bitrate (bits per second, bps). This task was repeated over contiguous days to evaluate the sustained performance across time. Main results. We found that in the monkey with a younger (i.e., two year old) implant and better signal quality, a fixed decoder could sustain performance for a month at a rate of 4 bps, the highest achieved communication rate reported to date. This fixed decoder was evaluated across 22 months and experienced a performance decline at a rate of 0.24 bps yr(-1). In the monkey with the older (i.e., 3.5 year old) implant and poorer signal quality, a fixed decoder could not sustain performance for more than a few days. Nevertheless, performance in this monkey was maintained for two weeks without requiring additional online retraining time by utilizing prior days' experimental data. Upon analysis of the changes in channel tuning, we found that this stability appeared partially attributable to the cancelling-out of neural tuning fluctuations when projected to two-dimensional cursor movements. Significance. The findings in this study (1) document the highest-performing communication neural prosthesis in monkeys, (2) confirm and extend prior reports of the stability of fixed decoders, and (3) demonstrate a protocol for system stability under conditions where fixed decoders would otherwise fail. These improvements to decoder stability are important for minimizing training time and should make neural prostheses more practical to use.

  View details for DOI 10.1088/1741-2560/11/6/066003

  View details for Web of Science ID 000345647700009

  View details for PubMedID 25307561

• Combining Decoder Design and Neural Adaptation in Brain-Machine Interfaces NEURON Shenoy, K. V., Carmena, J. M. 2014; 84 (4): 665-680

  Abstract

  Brain-machine interfaces (BMIs) aim to help people with paralysis by decoding movement-related neural signals into control signals for guiding computer cursors, prosthetic arms, and other assistive devices. Despite compelling laboratory experiments and ongoing FDA pilot clinical trials, system performance, robustness, and generalization remain challenges. We provide a perspective on how two complementary lines of investigation, that have focused on decoder design and neural adaptation largely separately, could be brought together to advance BMIs. This BMI paradigm should also yield new scientific insights into the function and dysfunction of the nervous system.

  View details for DOI 10.1016/j.neuron.2014.08.038

  View details for Web of Science ID 000345424900005

  View details for PubMedID 25459407

• A freely-moving monkey treadmill model JOURNAL OF NEURAL ENGINEERING Foster, J. D., Nuyujukian, P., Freifeld, O., Gao, H., Walker, R., Ryu, S. I., Meng, T. H., Murmann, B., Black, M. J., Shenoy, K. V. 2014; 11 (4)

  Abstract

  Objective. Motor neuroscience and brain-machine interface (BMI) design is based on examining how the brain controls voluntary movement, typically by recording neural activity and behavior from animal models. Recording technologies used with these animal models have traditionally limited the range of behaviors that can be studied, and thus the generality of science and engineering research. We aim to design a freely-moving animal model using neural and behavioral recording technologies that do not constrain movement. Approach. We have established a freely-moving rhesus monkey model employing technology that transmits neural activity from an intracortical array using a head-mounted device and records behavior through computer vision using markerless motion capture. We demonstrate the flexibility and utility of this new monkey model, including the first recordings from motor cortex while rhesus monkeys walk quadrupedally on a treadmill. Main results. Using this monkey model, we show that multi-unit threshold-crossing neural activity encodes the phase of walking and that the average firing rate of the threshold crossings covaries with the speed of individual steps. On a population level, we find that neural state-space trajectories of walking at different speeds have similar rotational dynamics in some dimensions that evolve at the step rate of walking, yet robustly separate by speed in other state-space dimensions. Significance. Freely-moving animal models may allow neuroscientists to examine a wider range of behaviors and can provide a flexible experimental paradigm for examining the neural mechanisms that underlie movement generation across behaviors and environments. For BMIs, freely-moving animal models have the potential to aid prosthetic design by examining how neural encoding changes with posture, environment and other real-world context changes. Understanding this new realm of behavior in more naturalistic settings is essential for overall progress of basic motor neuroscience and for the successful translation of BMIs to people with paralysis.

  View details for DOI 10.1088/1741-2560/11/4/046020

  View details for Web of Science ID 000340046500020

• A freely-moving monkey treadmill model. Journal of neural engineering D Foster, J., Nuyujukian, P., Freifeld, O., Gao, H., Walker, R., I Ryu, S., H Meng, T., Murmann, B., J Black, M., V Shenoy, K. 2014; 11 (4): 046020-?

  Abstract

  Objective. Motor neuroscience and brain-machine interface (BMI) design is based on examining how the brain controls voluntary movement, typically by recording neural activity and behavior from animal models. Recording technologies used with these animal models have traditionally limited the range of behaviors that can be studied, and thus the generality of science and engineering research. We aim to design a freely-moving animal model using neural and behavioral recording technologies that do not constrain movement. Approach. We have established a freely-moving rhesus monkey model employing technology that transmits neural activity from an intracortical array using a head-mounted device and records behavior through computer vision using markerless motion capture. We demonstrate the flexibility and utility of this new monkey model, including the first recordings from motor cortex while rhesus monkeys walk quadrupedally on a treadmill. Main results. Using this monkey model, we show that multi-unit threshold-crossing neural activity encodes the phase of walking and that the average firing rate of the threshold crossings covaries with the speed of individual steps. On a population level, we find that neural state-space trajectories of walking at different speeds have similar rotational dynamics in some dimensions that evolve at the step rate of walking, yet robustly separate by speed in other state-space dimensions. Significance. Freely-moving animal models may allow neuroscientists to examine a wider range of behaviors and can provide a flexible experimental paradigm for examining the neural mechanisms that underlie movement generation across behaviors and environments. For BMIs, freely-moving animal models have the potential to aid prosthetic design by examining how neural encoding changes with posture, environment and other real-world context changes. Understanding this new realm of behavior in more naturalistic settings is essential for overall progress of basic motor neuroscience and for the successful translation of BMIs to people with paralysis.

  View details for DOI 10.1088/1741-2560/11/4/046020

  View details for PubMedID 24995476

• Information Systems Opportunities in Brain-Machine Interface Decoders PROCEEDINGS OF THE IEEE Kao, J. C., Stavisky, S. D., Sussillo, D., Nuyujukian, P., Shenoy, K. V. 2014; 102 (5): 666-682

  View details for DOI 10.1109/JPROC.2014.2307357

  View details for Web of Science ID 000336915700004

• Self-recalibrating classifiers for intracortical brain-computer interfaces. Journal of neural engineering Bishop, W., Chestek, C. C., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2014; 11 (2): 026001-?

  Abstract

  Objective. Intracortical brain-computer interface (BCI) decoders are typically retrained daily to maintain stable performance. Self-recalibrating decoders aim to remove the burden this may present in the clinic by training themselves autonomously during normal use but have only been developed for continuous control. Here we address the problem for discrete decoding (classifiers). Approach. We recorded threshold crossings from 96-electrode arrays implanted in the motor cortex of two rhesus macaques performing center-out reaches in 7 directions over 41 and 36 separate days spanning 48 and 58 days in total for offline analysis. Main results. We show that for the purposes of developing a self-recalibrating classifier, tuning parameters can be considered as fixed within days and that parameters on the same electrode move up and down together between days. Further, drift is constrained across time, which is reflected in the performance of a standard classifier which does not progressively worsen if it is not retrained daily, though overall performance is reduced by more than 10% compared to a daily retrained classifier. Two novel self-recalibrating classifiers produce a [Formula: see text] increase in classification accuracy over that achieved by the non-retrained classifier to nearly recover the performance of the daily retrained classifier. Significance. We believe that the development of classifiers that require no daily retraining will accelerate the clinical translation of BCI systems. Future work should test these results in a closed-loop setting.

  View details for DOI 10.1088/1741-2560/11/2/026001

  View details for PubMedID 24503597

• Self-recalibrating classifiers for intracortical brain-computer interfaces. Journal of neural engineering Bishop, W., Chestek, C. C., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2014; 11 (2): 026001-?

  Abstract

  Objective. Intracortical brain-computer interface (BCI) decoders are typically retrained daily to maintain stable performance. Self-recalibrating decoders aim to remove the burden this may present in the clinic by training themselves autonomously during normal use but have only been developed for continuous control. Here we address the problem for discrete decoding (classifiers). Approach. We recorded threshold crossings from 96-electrode arrays implanted in the motor cortex of two rhesus macaques performing center-out reaches in 7 directions over 41 and 36 separate days spanning 48 and 58 days in total for offline analysis. Main results. We show that for the purposes of developing a self-recalibrating classifier, tuning parameters can be considered as fixed within days and that parameters on the same electrode move up and down together between days. Further, drift is constrained across time, which is reflected in the performance of a standard classifier which does not progressively worsen if it is not retrained daily, though overall performance is reduced by more than 10% compared to a daily retrained classifier. Two novel self-recalibrating classifiers produce a [Formula: see text] increase in classification accuracy over that achieved by the non-retrained classifier to nearly recover the performance of the daily retrained classifier. Significance. We believe that the development of classifiers that require no daily retraining will accelerate the clinical translation of BCI systems. Future work should test these results in a closed-loop setting.

  View details for DOI 10.1088/1741-2560/11/2/026001

  View details for PubMedID 24503597

• Cortical activity in the null space: permitting preparation without movement. Nature neuroscience Kaufman, M. T., Churchland, M. M., Ryu, S. I., Shenoy, K. V. 2014; 17 (3): 440-448

  Abstract

  Neural circuits must perform computations and then selectively output the results to other circuits. Yet synapses do not change radically at millisecond timescales. A key question then is: how is communication between neural circuits controlled? In motor control, brain areas directly involved in driving movement are active well before movement begins. Muscle activity is some readout of neural activity, yet it remains largely unchanged during preparation. Here we find that during preparation, while the monkey holds still, changes in motor cortical activity cancel out at the level of these population readouts. Motor cortex can thereby prepare the movement without prematurely causing it. Further, we found evidence that this mechanism also operates in dorsal premotor cortex, largely accounting for how preparatory activity is attenuated in primary motor cortex. Selective use of 'output-null' vs. 'output-potent' patterns of activity may thus help control communication to the muscles and between these brain areas.

  View details for DOI 10.1038/nn.3643

  View details for PubMedID 24487233

• Cortical activity in the null space: permitting preparation without movement. Nature neuroscience Kaufman, M. T., Churchland, M. M., Ryu, S. I., Shenoy, K. V. 2014; 17 (3): 440-448

  Abstract

  Neural circuits must perform computations and then selectively output the results to other circuits. Yet synapses do not change radically at millisecond timescales. A key question then is: how is communication between neural circuits controlled? In motor control, brain areas directly involved in driving movement are active well before movement begins. Muscle activity is some readout of neural activity, yet it remains largely unchanged during preparation. Here we find that during preparation, while the monkey holds still, changes in motor cortical activity cancel out at the level of these population readouts. Motor cortex can thereby prepare the movement without prematurely causing it. Further, we found evidence that this mechanism also operates in dorsal premotor cortex, largely accounting for how preparatory activity is attenuated in primary motor cortex. Selective use of 'output-null' vs. 'output-potent' patterns of activity may thus help control communication to the muscles and between these brain areas.

  View details for DOI 10.1038/nn.3643

  View details for PubMedID 24487233

• Intention estimation in brain-machine interfaces. Journal of neural engineering Fan, J. M., Nuyujukian, P., Kao, J. C., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2014; 11 (1): 016004-?

  Abstract

  The objective of this work was to quantitatively investigate the mechanisms underlying the performance gains of the recently reported 'recalibrated feedback intention-trained Kalman Filter' (ReFIT-KF).This was accomplished by designing variants of the ReFIT-KF algorithm and evaluating training and online data to understand the neural basis of this improvement. We focused on assessing the contribution of two training set innovations of the ReFIT-KF algorithm: intention estimation and the two-stage training paradigm.Within the two-stage training paradigm, we found that intention estimation independently increased target acquisition rates by 37% and 59%, respectively, across two monkeys implanted with multiunit intracortical arrays. Intention estimation improved performance by enhancing the tuning properties and the mutual information between the kinematic and neural training data. Furthermore, intention estimation led to fewer shifts in channel tuning between the training set and online control, suggesting that less adaptation was required during online control. Retraining the decoder with online BMI training data also reduced shifts in tuning, suggesting a benefit of training a decoder in the same behavioral context; however, retraining also led to slower online decode velocities. Finally, we demonstrated that one- and two-stage training paradigms performed comparably when intention estimation is applied.These findings highlight the utility of intention estimation in reducing the need for adaptive strategies and improving the online performance of BMIs, helping to guide future BMI design decisions.

  View details for PubMedID 24654266

• Intention estimation in brain-machine interfaces. Journal of neural engineering Fan, J. M., Nuyujukian, P., Kao, J. C., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2014; 11 (1): 016004-?

  Abstract

  The objective of this work was to quantitatively investigate the mechanisms underlying the performance gains of the recently reported 'recalibrated feedback intention-trained Kalman Filter' (ReFIT-KF).This was accomplished by designing variants of the ReFIT-KF algorithm and evaluating training and online data to understand the neural basis of this improvement. We focused on assessing the contribution of two training set innovations of the ReFIT-KF algorithm: intention estimation and the two-stage training paradigm.Within the two-stage training paradigm, we found that intention estimation independently increased target acquisition rates by 37% and 59%, respectively, across two monkeys implanted with multiunit intracortical arrays. Intention estimation improved performance by enhancing the tuning properties and the mutual information between the kinematic and neural training data. Furthermore, intention estimation led to fewer shifts in channel tuning between the training set and online control, suggesting that less adaptation was required during online control. Retraining the decoder with online BMI training data also reduced shifts in tuning, suggesting a benefit of training a decoder in the same behavioral context; however, retraining also led to slower online decode velocities. Finally, we demonstrated that one- and two-stage training paradigms performed comparably when intention estimation is applied.These findings highlight the utility of intention estimation in reducing the need for adaptive strategies and improving the online performance of BMIs, helping to guide future BMI design decisions.

  View details for PubMedID 24654266

• Neural Dynamics of Reaching following Incorrect or Absent Motor Preparation NEURON Ames, K. C., Ryu, S. I., Shenoy, K. V. 2014; 81 (2): 438-451

  Abstract

  Moving is thought to take separate preparation and execution steps. During preparation, neural activity in primary motor and dorsal premotor cortices achieves a state specific to an upcoming action but movements are not performed until the execution phase. We investigated whether this preparatory state (more precisely, prepare-and-hold state) is required for movement execution using two complementary experiments. We compared monkeys' neural activity during delayed and nondelayed reaches and in a delayed reaching task in which the target switched locations on a small percentage of trials. Neural population activity bypassed the prepare-and-hold state both in the absence of a delay and if the wrong reach was prepared. However, the initial neural response to the target was similar across behavioral conditions. This suggests that the prepare-and-hold state can be bypassed if needed, but there is a short-latency preparatory step that is performed prior to movement even without a delay.

  View details for DOI 10.1016/j.neuron.2013.12.003

  View details for Web of Science ID 000330420700020

  View details for PubMedID 24462104

  View details for PubMedCentralID PMC3936035

• Cortical activity in the null space: permitting preparation without movement Nature Neuroscience Kaufman, M., T. 2014
• Hybrid decoding of both spikes and low-frequency local field potentials for brain-machine interfaces. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2014; 2014: 3041-3044

  Abstract

  The best-performing brain-machine interfaces (BMIs) to date decode movement intention from intracortically recorded spikes, but these signals may be lost over time. A way to increase the useful lifespan of BMIs is to make more comprehensive use of available neural signals. Recent studies have demonstrated that the local field potential (LFP), a potentially more robust signal, can also be used to control a BMI. However, LFP-driven performance has fallen short of the best spikes-driven performance. Here we report a biomimetic BMI driven by low-frequency LFP that enabled a rhesus monkey to acquire and hold randomly placed targets with 99% success rate. Although LFP-driven performance was still worse than when decoding spikes, to the best of our knowledge this represents the highest-performing LFP-based BMI. We also demonstrate a new hybrid BMI that decodes cursor velocity using both spikes and LFP. This hybrid decoder improved performance over spikes-only decoding. Our results suggest that LFP can complement spikes when spikes are available or provide an alternative control signal if spikes are absent.

  View details for DOI 10.1109/EMBC.2014.6944264

  View details for PubMedID 25570632

• Intention estimation in brain-machine interfaces Journal of Neural Engineering Fan, J., M., Nuyujukian, P., Kao, J., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2014; 11:016004
• DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. Journal of neural engineering Cowley, B. R., Kaufman, M. T., Butler, Z. S., Churchland, M. M., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2013; 10 (6): 066012-?

  Abstract

  Objective. Analyzing and interpreting the activity of a heterogeneous population of neurons can be challenging, especially as the number of neurons, experimental trials, and experimental conditions increases. One approach is to extract a set of latent variables that succinctly captures the prominent co-fluctuation patterns across the neural population. A key problem is that the number of latent variables needed to adequately describe the population activity is often greater than 3, thereby preventing direct visualization of the latent space. By visualizing a small number of 2-d projections of the latent space or each latent variable individually, it is easy to miss salient features of the population activity. Approach. To address this limitation, we developed a Matlab graphical user interface (called DataHigh) that allows the user to quickly and smoothly navigate through a continuum of different 2-d projections of the latent space. We also implemented a suite of additional visualization tools (including playing out population activity timecourses as a movie and displaying summary statistics, such as covariance ellipses and average timecourses) and an optional tool for performing dimensionality reduction. Main results. To demonstrate the utility and versatility of DataHigh, we used it to analyze single-trial spike count and single-trial timecourse population activity recorded using a multi-electrode array, as well as trial-averaged population activity recorded using single electrodes. Significance. DataHigh was developed to fulfil a need for visualization in exploratory neural data analysis, which can provide intuition that is critical for building scientific hypotheses and models of population activity.

  View details for DOI 10.1088/1741-2560/10/6/066012

  View details for PubMedID 24216250

• DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. Journal of neural engineering Cowley, B. R., Kaufman, M. T., Butler, Z. S., Churchland, M. M., Ryu, S. I., Shenoy, K. V., Yu, B. M. 2013; 10 (6): 066012-?

  Abstract

  Objective. Analyzing and interpreting the activity of a heterogeneous population of neurons can be challenging, especially as the number of neurons, experimental trials, and experimental conditions increases. One approach is to extract a set of latent variables that succinctly captures the prominent co-fluctuation patterns across the neural population. A key problem is that the number of latent variables needed to adequately describe the population activity is often greater than 3, thereby preventing direct visualization of the latent space. By visualizing a small number of 2-d projections of the latent space or each latent variable individually, it is easy to miss salient features of the population activity. Approach. To address this limitation, we developed a Matlab graphical user interface (called DataHigh) that allows the user to quickly and smoothly navigate through a continuum of different 2-d projections of the latent space. We also implemented a suite of additional visualization tools (including playing out population activity timecourses as a movie and displaying summary statistics, such as covariance ellipses and average timecourses) and an optional tool for performing dimensionality reduction. Main results. To demonstrate the utility and versatility of DataHigh, we used it to analyze single-trial spike count and single-trial timecourse population activity recorded using a multi-electrode array, as well as trial-averaged population activity recorded using single electrodes. Significance. DataHigh was developed to fulfil a need for visualization in exploratory neural data analysis, which can provide intuition that is critical for building scientific hypotheses and models of population activity.

  View details for DOI 10.1088/1741-2560/10/6/066012

  View details for PubMedID 24216250

• Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature Mante, V., Sussillo, D., Shenoy, K. V., Newsome, W. T. 2013; 503 (7474): 78-84

  Abstract

  Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of task-relevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding context-dependent computations.

  View details for DOI 10.1038/nature12742

  View details for PubMedID 24201281

• Context-dependent computation by recurrent dynamics in prefrontal cortex NATURE Mante, V., Sussillo, D., Shenoy, K. V., Newsome, W. T. 2013; 503 (7474): 78-?

  Abstract

  Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of task-relevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding context-dependent computations.

  View details for DOI 10.1038/nature12742

  View details for Web of Science ID 000326585600035

  View details for PubMedID 24201281

• A coaxial optrode as multifunction write-read probe for optogenetic studies in non-human primates. Journal of neuroscience methods Ozden, I., Wang, J., Lu, Y., May, T., Lee, J., Goo, W., O'Shea, D. J., Kalanithi, P., Diester, I., Diagne, M., Deisseroth, K., Shenoy, K. V., Nurmikko, A. V. 2013; 219 (1): 142-154

  Abstract

  Advances in optogenetics have led to first reports of expression of light-gated ion-channels in non-human primates (NHPs). However, a major obstacle preventing effective application of optogenetics in NHPs and translation to optogenetic therapeutics is the absence of compatible multifunction optoelectronic probes for (1) precision light delivery, (2) low-interference electrophysiology, (3) protein fluorescence detection, and (4) repeated insertion with minimal brain trauma.Here we describe a novel brain probe device, a "coaxial optrode", designed to minimize brain tissue damage while microfabricated to perform simultaneous electrophysiology, light delivery and fluorescence measurements in the NHP brain. The device consists of a tapered, gold-coated optical fiber inserted in a polyamide tube. A portion of the gold coating is exposed at the fiber tip to allow electrophysiological recordings in addition to light delivery/collection at the tip.Coaxial optrode performance was demonstrated by experiments in rodents and NHPs, and characterized by computational models. The device mapped opsin expression in the brain and achieved precisely targeted optical stimulation and electrophysiology with minimal cortical damage.Overall, combined electrical, optical and mechanical features of the coaxial optrode allowed a performance for NHP studies which was not possible with previously existing devices.Coaxial optrode is currently being used in two NHP laboratories as a major tool to study brain function by inducing light modulated neural activity and behavior. By virtue of its design, the coaxial optrode can be extended for use as a chronic implant and multisite neural stimulation/recording.

  View details for DOI 10.1016/j.jneumeth.2013.06.011

  View details for PubMedID 23867081

  View details for PubMedCentralID PMC3789534

• The roles of monkey M1 neuron classes in movement preparation and execution JOURNAL OF NEUROPHYSIOLOGY Kaufman, M. T., Churchland, M. M., Shenoy, K. V. 2013; 110 (4): 817-825

  Abstract

  The motor cortices exhibit substantial activity while preparing movements, yet the arm remains still during preparation. We investigated whether a subpopulation of presumed inhibitory neurons in primary motor cortex (M1) might be involved in "gating" motor output during preparation, while permitting output during movement. This hypothesis predicts a release of inhibition just before movement onset. In data from M1 of two monkeys, we did not find evidence for this hypothesis: few neurons exhibited a clear pause during movement, and these were at the tail end of a broad distribution. We then identified a subpopulation likely to be enriched for inhibitory interneurons, using their waveform shapes. We found that the firing rates of this subpopulation tended to increase during movement instead of decreasing as predicted by the M1 gating model. No clear subset that might implement an inhibitory gate was observed. Together with previous evidence against upstream inhibitory mechanisms in premotor cortex, this provides evidence against an inhibitory "gate" for motor output in cortex. Instead, it appears that some other mechanism must likely exist.

  View details for DOI 10.1152/jn.00892.2011

  View details for Web of Science ID 000323208800003

  View details for PubMedID 23699057

  View details for PubMedCentralID PMC3742981

• 194 High Performance Computer Cursor Control Using Neuronal Ensemble Recordings From the Motor Cortex of a Person With ALS. Neurosurgery Henderson, J. M., Gilja, V., Pandarinath, C., Blabe, C., Hochberg, L. R., Shenoy, K. V. 2013; 60: 184-?

  Abstract

  Chronically implanted brain-computer interface systems have been demonstrated in several human research participants, with encouraging early results. A major aim of the current project is to provide improved speed and accuracy of computer cursor control for people with paralysis.A 50-year-old woman with Amyotrophic Lateral Sclerosis (ALS) and weakness of all 4 limbs (but with some retained upper extremity function) underwent implantation of an array of 100 silicon microelectrodes into the 'hand knob' area of the precentral gyrus as part of a multi-site pilot clinical trial (Braingate2, IDE). Beginning 1 month following implantation, twice-weekly recording sessions were carried out in the participant's home. A circular cursor and several targets were displayed on a computer monitor. The participant performed a 'center-out' cursor task by moving her finger on a trackpad to acquire the targets while neural activity was recorded. This neural activity was correlated with finger movement to produce a velocity-based Kalman filter, which was in turn used to derive on-screen cursor movement from neural activity. Under neural control, the participant acquired 1 of either 4 or 8 peripheral targets, placed between 150 and 225 pixels from a central target. Each block consisted of 160 consecutive trials. Targets were acquired by touching the target with the neurally controlled cursor, with or without a required dwell time. All targets had a diameter of 100 pixels: Accuracy and acquisition time varied across 36 blocks, with more recent sessions tending toward higher performance. Best performance in the 8 target task with 250 msec dwell was 92% accuracy, with average acquisition time of 1.89 ± 1.09 seconds.Our research participant was able to acquire targets using neural control with high speed and accuracy. Optimizations are being explored to increase performance further, with the eventual goal of providing cursor control approaching that achievable by able-bodied computer users.

  View details for DOI 10.1227/01.neu.0000432784.58847.74

  View details for PubMedID 23839461

• Cortical control of arm movements: a dynamical systems perspective. Annual review of neuroscience Shenoy, K. V., Sahani, M., Churchland, M. M. 2013; 36: 337-359

  Abstract

  Our ability to move is central to everyday life. Investigating the neural control of movement in general, and the cortical control of volitional arm movements in particular, has been a major research focus in recent decades. Studies have involved primarily either attempts to account for single-neuron responses in terms of tuning for movement parameters or attempts to decode movement parameters from populations of tuned neurons. Even though this focus on encoding and decoding has led to many seminal advances, it has not produced an agreed-upon conceptual framework. Interest in understanding the underlying neural dynamics has recently increased, leading to questions such as how does the current population response determine the future population response, and to what purpose? We review how a dynamical systems perspective may help us understand why neural activity evolves the way it does, how neural activity relates to movement parameters, and how a unified conceptual framework may result. Expected final online publication date for the Annual Review of Neuroscience Volume 36 is July 08, 2013. Please see http://www.annualreviews.org/catalog/pubdates.aspx for revised estimates.

  View details for DOI 10.1146/annurev-neuro-062111-150509

  View details for PubMedID 23725001

• Investigating the role of firing-rate normalization and dimensionality reduction in brain-machine interface robustness. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Kao, J. C., Nuyujukian, P., Stavisky, S., Ryu, S. I., Ganguli, S., Shenoy, K. V. 2013; 2013: 293-298

  Abstract

  The intraday robustness of brain-machine interfaces (BMIs) is important to their clinical viability. In particular, BMIs must be robust to intraday perturbations in neuron firing rates, which may arise from several factors including recording loss and external noise. Using a state-of-the-art decode algorithm, the Recalibrated Feedback Intention Trained Kalman filter (ReFIT-KF) [1] we introduce two novel modifications: (1) a normalization of the firing rates, and (2) a reduction of the dimensionality of the data via principal component analysis (PCA). We demonstrate in online studies that a ReFIT-KF equipped with normalization and PCA (NPC-ReFIT-KF) (1) achieves comparable performance to a standard ReFIT-KF when at least 60% of the neural variance is captured, and (2) is more robust to the undetected loss of channels. We present intuition as to how both modifications may increase the robustness of BMIs, and investigate the contribution of each modification to robustness. These advances, which lead to a decoder achieving state-of-the-art performance with improved robustness, are important for the clinical viability of BMI systems.

  View details for DOI 10.1109/EMBC.2013.6609495

  View details for PubMedID 24109682

• Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces. Journal of neural engineering Dethier, J., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Boahen, K. 2013; 10 (3): 036008-?

  Abstract

  Objective. Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. Approach. One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). Main results. Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. Significance. These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.

  View details for DOI 10.1088/1741-2560/10/3/036008

  View details for PubMedID 23574919

• Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces. Journal of neural engineering Dethier, J., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Boahen, K. 2013; 10 (3): 036008-?

  Abstract

  Objective. Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. Approach. One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). Main results. Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. Significance. These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.

  View details for DOI 10.1088/1741-2560/10/3/036008

  View details for PubMedID 23574919

• Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas. Journal of neural engineering Chestek, C. A., Gilja, V., Blabe, C. H., Foster, B. L., Shenoy, K. V., Parvizi, J., Henderson, J. M. 2013; 10 (2): 026002-?

  Abstract

  Brain-machine interface systems translate recorded neural signals into command signals for assistive technology. In individuals with upper limb amputation or cervical spinal cord injury, the restoration of a useful hand grasp could significantly improve daily function. We sought to determine if electrocorticographic (ECoG) signals contain sufficient information to select among multiple hand postures for a prosthetic hand, orthotic, or functional electrical stimulation system.We recorded ECoG signals from subdural macro- and microelectrodes implanted in motor areas of three participants who were undergoing inpatient monitoring for diagnosis and treatment of intractable epilepsy. Participants performed five distinct isometric hand postures, as well as four distinct finger movements. Several control experiments were attempted in order to remove sensory information from the classification results. Online experiments were performed with two participants.Classification rates were 68%, 84% and 81% for correct identification of 5 isometric hand postures offline. Using 3 potential controls for removing sensory signals, error rates were approximately doubled on average (2.1×). A similar increase in errors (2.6×) was noted when the participant was asked to make simultaneous wrist movements along with the hand postures. In online experiments, fist versus rest was successfully classified on 97% of trials; the classification output drove a prosthetic hand. Online classification performance for a larger number of hand postures remained above chance, but substantially below offline performance. In addition, the long integration windows used would preclude the use of decoded signals for control of a BCI system.These results suggest that ECoG is a plausible source of command signals for prosthetic grasp selection. Overall, avenues remain for improvement through better electrode designs and placement, better participant training, and characterization of non-stationarities such that ECoG could be a viable signal source for grasp control for amputees or individuals with paralysis.

  View details for DOI 10.1088/1741-2560/10/2/026002

  View details for PubMedID 23369953

• Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas JOURNAL OF NEURAL ENGINEERING Chestek, C. A., Gilja, V., Blabe, C. H., Foster, B. L., Shenoy, K. V., Parvizi, J., Henderson, J. M. 2013; 10 (2)

  Abstract

  Brain-machine interface systems translate recorded neural signals into command signals for assistive technology. In individuals with upper limb amputation or cervical spinal cord injury, the restoration of a useful hand grasp could significantly improve daily function. We sought to determine if electrocorticographic (ECoG) signals contain sufficient information to select among multiple hand postures for a prosthetic hand, orthotic, or functional electrical stimulation system.We recorded ECoG signals from subdural macro- and microelectrodes implanted in motor areas of three participants who were undergoing inpatient monitoring for diagnosis and treatment of intractable epilepsy. Participants performed five distinct isometric hand postures, as well as four distinct finger movements. Several control experiments were attempted in order to remove sensory information from the classification results. Online experiments were performed with two participants.Classification rates were 68%, 84% and 81% for correct identification of 5 isometric hand postures offline. Using 3 potential controls for removing sensory signals, error rates were approximately doubled on average (2.1×). A similar increase in errors (2.6×) was noted when the participant was asked to make simultaneous wrist movements along with the hand postures. In online experiments, fist versus rest was successfully classified on 97% of trials; the classification output drove a prosthetic hand. Online classification performance for a larger number of hand postures remained above chance, but substantially below offline performance. In addition, the long integration windows used would preclude the use of decoded signals for control of a BCI system.These results suggest that ECoG is a plausible source of command signals for prosthetic grasp selection. Overall, avenues remain for improvement through better electrode designs and placement, better participant training, and characterization of non-stationarities such that ECoG could be a viable signal source for grasp control for amputees or individuals with paralysis.

  View details for DOI 10.1088/1741-2560/10/2/026002

  View details for Web of Science ID 000316728700003

  View details for PubMedID 23369953

  View details for PubMedCentralID PMC3670711

• A recurrent neural network that produces EMG from rhythmic dynamics. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Sussillo, D., Churchland, M. M., Kaufman, M. T., Shenoy, K. V. 2013: III-67
• Quantifying representational and dynamical structure in large neural datasets. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Seely, J., Kaufman, M. T., Kohn, A., Smith, M., Movshon, A., Priebe, N., Shenoy, Krishna, V. 2013
• Selective integration of sensory evidence by recurrent dynamics in prefrontal cortex. Nature. Mante, V., Sussillo, D., Shenoy, K. V., Newsome, W. T. 2013; 503: 78-8, 45-47
• Characterization of dynamical activity in motor cortex. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Elsayed, G., Kaufman, M. T., Ryu, S. I., Shenoy, K. V., Churchland, M. M., Cunningham, J. P. 2013
• Neural dynamics following optogenetic disruption of motor preparation. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) O'Shea, D., Goo, W., Kalanithi, P., Diester, I., Ramakrishnan, C., Deisseroth, K., Shenoy, Krishna, V. 2013
• High performance computer cursor control using neuronal ensemble recordings from the motor cortex of a person with ALS. Neurosurgery. Henderson, J. M., Gilja, V., Pandarinath, C., Blabe, C., Hochberg, L. R., Shenoy, K. V. 2013; 1:184: 60
• DataHigh: Graphical user interface for visualizing and interacting with high-dimensional neural activity. Journal of Neural Engineering. Cowley, B. R., Kaufman, M. T., Butler, Z. S., Churchland, M. M., Ryu, S. I., Shenoy, K. V. 2013; 10:066012
• Dimensionality, dynamics, and correlations in the motor cortical substrate for reaching. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Gao, P., rautmann, E., Yu, B. M., Santhanam, G., Ryu, S. I., Shenoy, K. V. 2013
• A high-performance neural prosthesis enabled by control algorithm design NATURE NEUROSCIENCE Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M., Churchland, M. M., Kaufman, M. T., Kao, J. C., Ryu, S. I., Shenoy, K. V. 2012; 15 (12): 1752-1757

  Abstract

  Neural prostheses translate neural activity from the brain into control signals for guiding prosthetic devices, such as computer cursors and robotic limbs, and thus offer individuals with disabilities greater interaction with the world. However, relatively low performance remains a critical barrier to successful clinical translation; current neural prostheses are considerably slower, with less accurate control, than the native arm. Here we present a new control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF) that incorporates assumptions about the nature of closed-loop neural prosthetic control. When tested in rhesus monkeys implanted with motor cortical electrode arrays, the ReFIT-KF algorithm outperformed existing neural prosthetic algorithms in all measured domains and halved target acquisition time. This control algorithm permits sustained, uninterrupted use for hours and generalizes to more challenging tasks without retraining. Using this algorithm, we demonstrate repeatable high performance for years after implantation in two monkeys, thereby increasing the clinical viability of neural prostheses.

  View details for DOI 10.1038/nn.3265

  View details for Web of Science ID 000311706700023

  View details for PubMedID 23160043

  View details for PubMedCentralID PMC3638087

• Neural population dynamics during reaching NATURE Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Foster, J. D., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. 2012; 487 (7405): 51-?

  Abstract

  Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.

  View details for DOI 10.1038/nature11129

  View details for Web of Science ID 000305982900048

  View details for PubMedID 22722855

  View details for PubMedCentralID PMC3393826

• An L (1)-regularized logistic model for detecting short-term neuronal interactions JOURNAL OF COMPUTATIONAL NEUROSCIENCE Zhao, M., Batista, A., Cunningham, J. P., Chestek, C., Rivera-Alvidrez, Z., Kalmar, R., Ryu, S., Shenoy, K., Iyengar, S. 2012; 32 (3): 479-497

  Abstract

  Interactions among neurons are a key component of neural signal processing. Rich neural data sets potentially containing evidence of interactions can now be collected readily in the laboratory, but existing analysis methods are often not sufficiently sensitive and specific to reveal these interactions. Generalized linear models offer a platform for analyzing multi-electrode recordings of neuronal spike train data. Here we suggest an L(1)-regularized logistic regression model (L(1)L method) to detect short-term (order of 3 ms) neuronal interactions. We estimate the parameters in this model using a coordinate descent algorithm, and determine the optimal tuning parameter using a Bayesian Information Criterion. Simulation studies show that in general the L(1)L method has better sensitivities and specificities than those of the traditional shuffle-corrected cross-correlogram (covariogram) method. The L(1)L method is able to detect excitatory interactions with both high sensitivity and specificity with reasonably large recordings, even when the magnitude of the interactions is small; similar results hold for inhibition given sufficiently high baseline firing rates. Our study also suggests that the false positives can be further removed by thresholding, because their magnitudes are typically smaller than true interactions. Simulations also show that the L(1)L method is somewhat robust to partially observed networks. We apply the method to multi-electrode recordings collected in the monkey dorsal premotor cortex (PMd) while the animal prepares to make reaching arm movements. The results show that some neurons interact differently depending on task conditions. The stronger interactions detected with our L(1)L method were also visible using the covariogram method.

  View details for DOI 10.1007/s10827-011-0365-5

  View details for Web of Science ID 000303589500007

  View details for PubMedID 22038503

• A recurrent neural network for closed-loop intracortical brain-machine interface decoders JOURNAL OF NEURAL ENGINEERING Sussillo, D., Nuyujukian, P., Fan, J. M., Kao, J. C., Stavisky, S. D., Ryu, S., Shenoy, K. 2012; 9 (2)

  Abstract

  Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships in time series data with complex temporal dependences. In this paper, we explore the ability of a simplified type of RNN, one with limited modifications to the internal weights called an echostate network (ESN), to effectively and continuously decode monkey reaches during a standard center-out reach task using a cortical brain-machine interface (BMI) in a closed loop. We demonstrate that the RNN, an ESN implementation termed a FORCE decoder (from first order reduced and controlled error learning), learns the task quickly and significantly outperforms the current state-of-the-art method, the velocity Kalman filter (VKF), using the measure of target acquire time. We also demonstrate that the FORCE decoder generalizes to a more difficult task by successfully operating the BMI in a randomized point-to-point task. The FORCE decoder is also robust as measured by the success rate over extended sessions. Finally, we show that decoded cursor dynamics are more like naturalistic hand movements than those of the VKF. Taken together, these results suggest that RNNs in general, and the FORCE decoder in particular, are powerful tools for BMI decoder applications.

  View details for DOI 10.1088/1741-2560/9/2/026027

  View details for Web of Science ID 000302144100027

  View details for PubMedID 22427488

  View details for PubMedCentralID PMC3638090

• HermesE: A 96-Channel Full Data Rate Direct Neural Interface in 0.13 mu m CMOS IEEE JOURNAL OF SOLID-STATE CIRCUITS Gao, H., Walker, R. M., Nuyujukian, P., Makinwa, K. A., Shenoy, K. V., Murmann, B., Meng, T. H. 2012; 47 (4): 1043-1055

  View details for DOI 10.1109/JSSC.2012.2185338

  View details for Web of Science ID 000302494700023

• Brain Enabled by Next-Generation Neurotechnology: Using Multiscale and Multimodal Models IEEE PULSE Shenoy, K. V., Nurmikko, A. V. 2012; 3 (2): 31-36

  Abstract

  As many articles in this issue of IEEE Pulse demonstrate, interfacing directly with the brain presents several fundamental challenges. These challenges reside at multiple levels and span many disciplines, ranging from the need to understand brain states at the level of neural circuits to creating technological innovations to facilitate new therapeutic options. The goal of our multiuniversity research team, composed of researchers from Stanford University, Brown University, the University of California at San Francisco (UCSF), and the University College London (UCL), is to substantially elevate the fundamental understanding of brain information processing and its relationship with sensation, behavior, and injury. Our team was assembled to provide expertise ranging from neuroscience to neuroengineering and to neurological and psychiatric clinical guidance, all of which are critical to the overarching research goal. By employing a suite of innovative experimental, computational, and theoretical approaches, the Defense Advanced Research Projects Agency (DARPA) Reorganization and Plasticity to Accelerate Injury Recovery (REPAIR) team has set its sights on learning how the brain and its microcircuitry react (e.g., to sudden physiological changes) and what can be done to encourage recovery from such (reversible) injury. In this article, we summarize some of the team's technical goals, approaches, and early illustrative results.

  View details for DOI 10.1109/MPUL.2011.2181021

  View details for Web of Science ID 000307806600009

  View details for PubMedID 22481743

• A brain machine interface control algorithm designed from a feedback control perspective 34th Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M., Ryu, S. I., Shenoy, K. V. IEEE. 2012: 1318–1322

  Abstract

  We present a novel brain machine interface (BMI) control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF). The design of ReFIT-KF is motivated from a feedback control perspective applied to existing BMI control algorithms. The result is two design innovations that alter the modeling assumptions made by these algorithms and the methods by which these algorithms are trained. In online neural control experiments recording from a 96-electrode array implanted in M1 of a macaque monkey, the ReFIT-KF control algorithm demonstrates large performance improvements over the current state of the art velocity Kalman filter, reducing target acquisition time by a factor of two, while maintaining a 500 ms hold period, thereby increasing the clinical viability of BMI systems.

  View details for Web of Science ID 000313296501144

  View details for PubMedID 23366141

• Concurrent integration and gating of sensory information with orthogonal mixed representations. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Mante, V., Sussillo, D., Shenoy, K. V., Newsome, W. T. 2012: II-58
• Long-term decoding stability without retraining for intracortical brain computer Interface. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Bishop, W., Nuyujukian, P., Chestek, C. A., Gilja, V., Ryu, S. I., Shenoy, K. V. 2012: III-40
• Dimensionality in motor cortex: differences between models and experiment. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Seely, J., Kaufman, M. T., Ryu, S. I., Cunningham, J. P., Shenoy, K. V., Churchland, M. M. 2012: II-67
• 2010 DARPA neural engineering, science, and technology forum [Guest Editorial]. IEEE EMBS. Schnitzer, J. J. 2012; 0.131944444444444
• Brain models enabled by next-generation neurotechnology. Pulse Magazine, IEEE Engineering in Medicine and Biology Society. Shenoy, K. V., Nurmikko, A. V. 2012; 3: 31-36.
• Neural dynamics of reaching following incomplete or incorrect planning. Frontiers in Neuroscience. Ames, K. C., Ryu, S. I., Shenoy, K. V. 2012: T-5.
• Neural Prosthetics In Encyclopedia of Motor Control Shenoy, K. V., Chestek, C. A. edited by Wolpert, D. Scholarpedia.. 2012: 1

  View details for DOI doi:10.4249/scholarpedia.

• A recurrent neural network that produces EMG from thythmic dynamics. Translational and Computational Motor Control (TCMC) pre-meeting to Society for Neuroscience annual meeting, New Orleans, LA. Sussillo, D., Churchland, M. M., Kaufman, M. T., Shenoy, K. V. 2012
• Identifying the neural initiation of a movement. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Petreska, B., Kaufman, M. T., Churchland, M. M., Ryu, S. I., Shenoy, K. V., Sahani, M. 2012: I-66
• A high-performance, robust brain-machine interface without retraining. Frontiers in Neuroscience. Nuyujukian, P., Kao, J., Fan, J. M., Stavisky, S., Ryu, S. I., Shenoy, K. V. 2012: III-65
• A framework for relating neural activity to freely moving behavior 34th Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Foster, J. D., Nuyujukian, P., Freifeld, O., Ryu, S. I., Black, M. J., Shenoy, K. V. IEEE. 2012: 2736–2739

  Abstract

  Two research communities, motor systems neuroscience and motor prosthetics, examine the relationship between neural activity in the motor cortex and movement. The former community aims to understand how the brain controls and generates movement; the latter community focuses on how to decode neural activity as control signals for a prosthetic cursor or limb. Both have made progress toward understanding the relationship between neural activity in the motor cortex and behavior. However, these findings are tested using animal models in an environment that constrains behavior to simple, limited movements. These experiments show that, in constrained settings, simple reaching motions can be decoded from small populations of spiking neurons. It is unclear whether these findings hold for more complex, full-body behaviors in unconstrained settings. Here we present the results of freely-moving behavioral experiments from a monkey with simultaneous intracortical recording. We investigated neural firing rates while the monkey performed various tasks such as walking on a treadmill, reaching for food, and sitting idly. We show that even in such an unconstrained and varied context, neural firing rates are well tuned to behavior, supporting findings of basic neuroscience. Further, we demonstrate that the various behavioral tasks can be reliably classified with over 95% accuracy, illustrating the viability of decoding techniques despite significant variation and environmental distractions associated with unconstrained behavior. Such encouraging results hint at potential utility of the freely-moving experimental paradigm.

  View details for Web of Science ID 000313296502238

  View details for PubMedID 23366491

• DataHigh: Graphical user interface for visualizing and interacting with high-dimensional neural activity 34th Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Cowley, B. R., Kaufman, M. T., Churchland, M. M., Ryu, S. I., Shenoy, K. V., Yu, B. M. IEEE. 2012: 4607–4610

  Abstract

  The activity of tens to hundreds of neurons can be succinctly summarized by a smaller number of latent variables extracted using dimensionality reduction methods. These latent variables define a reduced-dimensional space in which we can study how population activity varies over time, across trials, and across experimental conditions. Ideally, we would like to visualize the population activity directly in the reduced-dimensional space, whose optimal dimensionality (as determined from the data) is typically greater than 3. However, direct plotting can only provide a 2D or 3D view. To address this limitation, we developed a Matlab graphical user interface (GUI) that allows the user to quickly navigate through a continuum of different 2D projections of the reduced-dimensional space. To demonstrate the utility and versatility of this GUI, we applied it to visualize population activity recorded in premotor and motor cortices during reaching tasks. Examples include single-trial population activity recorded using a multi-electrode array, as well as trial-averaged population activity recorded sequentially using single electrodes. Because any single 2D projection may provide a misleading impression of the data, being able to see a large number of 2D projections is critical for intuition-and hypothesis-building during exploratory data analysis. The GUI includes a suite of additional interactive tools, including playing out population activity timecourses as a movie and displaying summary statistics, such as covariance ellipses and average timecourses. The use of visualization tools like the GUI developed here, in tandem with dimensionality reduction methods, has the potential to further our understanding of neural population activity.

  View details for Web of Science ID 000313296504203

  View details for PubMedID 23366954

  View details for PubMedCentralID PMC3745770

• Single-Trial Neural Correlates of Arm Movement Preparation NEURON Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., Sahani, M., Shenoy, K. V. 2011; 71 (3): 555-564

  Abstract

  The process by which neural circuitry in the brain plans and executes movements is not well understood. Until recently, most available data were limited either to single-neuron electrophysiological recordings or to measures of aggregate field or metabolism. Neither approach reveals how individual neurons' activities are coordinated within the population, and thus inferences about how the neural circuit forms a motor plan for an upcoming movement have been indirect. Here we build on recent advances in the measurement and description of population activity to frame and test an "initial condition hypothesis" of arm movement preparation and initiation. This hypothesis leads to a model in which the timing of movements may be predicted on each trial using neurons' moment-by-moment firing rates and rates of change of those rates. Using simultaneous microelectrode array recordings from premotor cortex of monkeys performing delayed-reach movements, we compare such single-trial predictions to those of other theories. We show that our model can explain approximately 4-fold more arm-movement reaction-time variance than the best alternative method. Thus, the initial condition hypothesis elucidates a view of the relationship between single-trial preparatory neural population dynamics and single-trial behavior.

  View details for DOI 10.1016/j.neuron.2011.05.047

  View details for Web of Science ID 000293991700017

  View details for PubMedID 21835350

  View details for PubMedCentralID PMC3155684

• Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex 39th Neural Interfaces Conference (NIC2010) Chestek, C. A., Gilja, V., Nuyujukian, P., Foster, J. D., Fan, J. M., Kaufman, M. T., Churchland, M. M., Rivera-Alvidrez, Z., Cunningham, J. P., Ryu, S. I., Shenoy, K. V. IOP PUBLISHING LTD. 2011

  Abstract

  Cortically-controlled prosthetic systems aim to help disabled patients by translating neural signals from the brain into control signals for guiding prosthetic devices. Recent reports have demonstrated reasonably high levels of performance and control of computer cursors and prosthetic limbs, but to achieve true clinical viability, the long-term operation of these systems must be better understood. In particular, the quality and stability of the electrically-recorded neural signals require further characterization. Here, we quantify action potential changes and offline neural decoder performance over 382 days of recording from four intracortical arrays in three animals. Action potential amplitude decreased by 2.4% per month on average over the course of 9.4, 10.4, and 31.7 months in three animals. During most time periods, decoder performance was not well correlated with action potential amplitude (p > 0.05 for three of four arrays). In two arrays from one animal, action potential amplitude declined by an average of 37% over the first 2 months after implant. However, when using simple threshold-crossing events rather than well-isolated action potentials, no corresponding performance loss was observed during this time using an offline decoder. One of these arrays was effectively used for online prosthetic experiments over the following year. Substantial short-term variations in waveforms were quantified using a wireless system for contiguous recording in one animal, and compared within and between days for all three animals. Overall, this study suggests that action potential amplitude declines more slowly than previously supposed, and performance can be maintained over the course of multiple years when decoding from threshold-crossing events rather than isolated action potentials. This suggests that neural prosthetic systems may provide high performance over multiple years in human clinical trials.

  View details for DOI 10.1088/1741-2560/8/4/045005

  View details for Web of Science ID 000292962800009

  View details for PubMedID 21775782

  View details for PubMedCentralID PMC3644617

• Challenges and Opportunities for Next-Generation Intracortically Based Neural Prostheses IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Gilja, V., Chestek, C. A., Diester, I., Henderson, J. M., Deisseroth, K., Shenoy, K. V. 2011; 58 (7): 1891-1899

  Abstract

  Neural prosthetic systems aim to help disabled patients by translating neural signals from the brain into control signals for guiding computer cursors, prosthetic arms, and other assistive devices. Intracortical electrode arrays measure action potentials and local field potentials from individual neurons, or small populations of neurons, in the motor cortices and can provide considerable information for controlling prostheses. Despite several compelling proof-of-concept laboratory animal experiments and an initial human clinical trial, at least three key challenges remain which, if left unaddressed, may hamper the translation of these systems into widespread clinical use. We review these challenges: achieving able-bodied levels of performance across tasks and across environments, achieving robustness across multiple decades, and restoring able-bodied quality proprioception and somatosensation. We also describe some emerging opportunities for meeting these challenges. If these challenges can be largely or fully met, intracortically based neural prostheses may achieve true clinical viability and help increasing numbers of disabled patients.

  View details for DOI 10.1109/TBME.2011.2107553

  View details for PubMedID 21257365

• Toward Clinically Viable Brain-Machine Interfaces 66th Annual Meeting of the Society-of-Biological-Psychiatry Shenoy, K. V. ELSEVIER SCIENCE INC. 2011: 193S–193S

  View details for Web of Science ID 000290641800614

• A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces JOURNAL OF NEUROPHYSIOLOGY Cunningham, J. P., Nuyujukian, P., Gilja, V., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2011; 105 (4): 1932-1949

  Abstract

  Neural prosthetic systems seek to improve the lives of severely disabled people by decoding neural activity into useful behavioral commands. These systems and their decoding algorithms are typically developed "offline," using neural activity previously gathered from a healthy animal, and the decoded movement is then compared with the true movement that accompanied the recorded neural activity. However, this offline design and testing may neglect important features of a real prosthesis, most notably the critical role of feedback control, which enables the user to adjust neural activity while using the prosthesis. We hypothesize that understanding and optimally designing high-performance decoders require an experimental platform where humans are in closed-loop with the various candidate decode systems and algorithms. It remains unexplored the extent to which the subject can, for a particular decode system, algorithm, or parameter, engage feedback and other strategies to improve decode performance. Closed-loop testing may suggest different choices than offline analyses. Here we ask if a healthy human subject, using a closed-loop neural prosthesis driven by synthetic neural activity, can inform system design. We use this online prosthesis simulator (OPS) to optimize "online" decode performance based on a key parameter of a current state-of-the-art decode algorithm, the bin width of a Kalman filter. First, we show that offline and online analyses indeed suggest different parameter choices. Previous literature and our offline analyses agree that neural activity should be analyzed in bins of 100- to 300-ms width. OPS analysis, which incorporates feedback control, suggests that much shorter bin widths (25-50 ms) yield higher decode performance. Second, we confirm this surprising finding using a closed-loop rhesus monkey prosthetic system. These findings illustrate the type of discovery made possible by the OPS, and so we hypothesize that this novel testing approach will help in the design of prosthetic systems that will translate well to human patients.

  View details for DOI 10.1152/jn.00503.2010

  View details for Web of Science ID 000289620500044

  View details for PubMedID 20943945

  View details for PubMedCentralID PMC3075301

• Combining Wireless Neural Recording and Video Capture for the Analysis of Natural Gait. International IEEE/EMBS Conference on Neural Engineering : [proceedings]. International IEEE EMBS Conference on Neural Engineering Foster, J. D., Freifeld, O., Nuyujukian, P., Ryu, S. I., Black, M. J., Shenoy, K. V. 2011; 2011: 613–16

  Abstract

  Neural control of movement is typically studied in constrained environments where there is a reduced set of possible behaviors. This constraint may unintentionally limit the applicability of findings to the generalized case of unconstrained behavior. We hypothesize that examining the unconstrained state across multiple behavioral contexts will lead to new insights into the neural control of movement and help advance the design of neural prosthetic decode algorithms. However, to pursue electrophysiological studies in such a manner requires a more flexible framework for experimentation. We propose that head-mounted neural recording systems with wireless data transmission, combined with markerless computer-vision based motion tracking, will enable new, less constrained experiments. As a proof-of-concept, we recorded and wirelessly transmitted broadband neural data from 32 electrodes in premotor cortex while acquiring single-camera video of a rhesus macaque walking on a treadmill. We demonstrate the ability to extract behavioral kinematics using an automated computer vision algorithm without use of markers and to predict kinematics from the neural data. Together these advances suggest that a new class of "freely moving monkey" experiments should be possible and should help broaden our understanding of the neural control of movement.

  View details for PubMedID 26019730

• Adaptive Resolution ADC Array for an Implantable Neural Sensor IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS O'Driscoll, S., Shenoy, K. V., Meng, T. H. 2011; 5 (2): 120-130

  Abstract

  This paper describes an analog-to-digital converter (ADC) array for an implantable neural sensor which digitizes neural signals sensed by a microelectrode array. The ADC array consists of 96 variable resolution ADC base cells. The resolution of each ADC cell in the array is varied according to neural data content of the signal from the corresponding electrode. The resolution adaptation algorithm is essentially to periodically recalibrate the required resolution and this is done without requiring any additional ADC cells. The adaptation implementation and results are described. The base ADC cell is implemented using a successive approximation charge redistribution architecture. The choice of architecture and circuit design are presented. The base ADC has been implemented in 0.13 μm CMOS as a 100 kS/s SAR ADC whose resolution can be varied from 3 to 8 bits with corresponding power consumption of 0.23 μW to 0.90 μW achieving an ENOB of 7.8 at the 8-bit setting. The energy per conversion step figure of merit is 48 fJ/step at the 8-bit setting. Resolution adaptation reduces power consumption by a factor of 2.3 for typical motor neuron signals while maintaining an effective 7.8-bit resolution across all channels.

  View details for DOI 10.1109/TBCAS.2011.2145418

  View details for Web of Science ID 000290534500004

• An optogenetic toolbox designed for primates NATURE NEUROSCIENCE Diester, I., Kaufman, M. T., Mogri, M., Pashaie, R., Goo, W., Yizhar, O., Ramakrishnan, C., Deisseroth, K., Shenoy, K. V. 2011; 14 (3): 387-397

  Abstract

  Optogenetics is a technique for controlling subpopulations of neurons in the intact brain using light. This technique has the potential to enhance basic systems neuroscience research and to inform the mechanisms and treatment of brain injury and disease. Before launching large-scale primate studies, the method needs to be further characterized and adapted for use in the primate brain. We assessed the safety and efficiency of two viral vector systems (lentivirus and adeno-associated virus), two human promoters (human synapsin (hSyn) and human thymocyte-1 (hThy-1)) and three excitatory and inhibitory mammalian codon-optimized opsins (channelrhodopsin-2, enhanced Natronomonas pharaonis halorhodopsin and the step-function opsin), which we characterized electrophysiologically, histologically and behaviorally in rhesus monkeys (Macaca mulatta). We also introduced a new device for measuring in vivo fluorescence over time, allowing minimally invasive assessment of construct expression in the intact brain. We present a set of optogenetic tools designed for optogenetic experiments in the non-human primate brain.

  View details for DOI 10.1038/nn.2749

  View details for Web of Science ID 000287650100021

  View details for PubMedID 21278729

  View details for PubMedCentralID PMC3150193

• A Brain-Machine Interface Operating with a Real-Time Spiking Neural Network Control Algorithm. Advances in neural information processing systems Dethier, J., Nuyujukian, P., Eliasmith, C., Stewart, T., Elassaad, S. A., Shenoy, K. V., Boahen, K. 2011; 2011: 2213-2221

  Abstract

  Motor prostheses aim to restore function to disabled patients. Despite compelling proof of concept systems, barriers to clinical translation remain. One challenge is to develop a low-power, fully-implantable system that dissipates only minimal power so as not to damage tissue. To this end, we implemented a Kalman-filter based decoder via a spiking neural network (SNN) and tested it in brain-machine interface (BMI) experiments with a rhesus monkey. The Kalman filter was trained to predict the arm's velocity and mapped on to the SNN using the Neural Engineering Framework (NEF). A 2,000-neuron embedded Matlab SNN implementation runs in real-time and its closed-loop performance is quite comparable to that of the standard Kalman filter. The success of this closed-loop decoder holds promise for hardware SNN implementations of statistical signal processing algorithms on neuromorphic chips, which may offer power savings necessary to overcome a major obstacle to the successful clinical translation of neural motor prostheses.

  View details for PubMedID 25309106

• A 96-channel full data rate direct neural interface in 0.13 um CMOS. Walker, R. M., Gao, H., Nuyujukian, P., Makinwa, K., Shenoy, K. V., Meng, T. H. 2011
• A brain-machine interface operating with a real-time spiking neural network control algorithm. Advances in Neural Information Processing Systems (NIPS) Dethier, J., Nuyujukian, P., Elassaad, S., Stewart, T., Eliasmith, C., Shenoy, K. V. edited by Shawe-Taylor, J., Zemel, R., S., Bartlett, P. MIT Press Cambridge, MA.. 2011: 1
• Spiking Neural Network Decoder for Brain-Machine Interfaces. International IEEE/EMBS Conference on Neural Engineering : [proceedings]. International IEEE EMBS Conference on Neural Engineering Dethier, J., Gilja, V., Nuyujukian, P., Elassaad, S. A., Shenoy, K. V., Boahen, K. 2011

  Abstract

  We used a spiking neural network (SNN) to decode neural data recorded from a 96-electrode array in premotor/motor cortex while a rhesus monkey performed a point-to-point reaching arm movement task. We mapped a Kalman-filter neural prosthetic decode algorithm developed to predict the arm's velocity on to the SNN using the Neural Engineering Framework and simulated it using Nengo, a freely available software package. A 20,000-neuron network matched the standard decoder's prediction to within 0.03% (normalized by maximum arm velocity). A 1,600-neuron version of this network was within 0.27%, and run in real-time on a 3GHz PC. These results demonstrate that a SNN can implement a statistical signal processing algorithm widely used as the decoder in high-performance neural prostheses (Kalman filter), and achieve similar results with just a few thousand neurons. Hardware SNN implementations-neuromorphic chips-may offer power savings, essential for realizing fully-implantable cortically controlled prostheses.

  View details for PubMedID 24352611

• The role of horizontal long-range connections in shaping the dynamics of multi-electrode array data. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Lerchner, A., Shenoy, K. V., Sahani, M. 2011: I-27.
• A dynamical systems view of motor preparation: Implications for neural prosthetic system design.Chapter 3 in Andrea Shenoy, K. V., Kaufman, M. T., Sahani, M., Churchland, M. M. edited by Green, M., Chapman, C., Elaine, Kalaska, F., John Amsterdam: The Netherlands.. 2011: 33–58
• Firing rate oscillations underlie motor cortex responses during reaching in monkey. Frontiers in Neuroscience. Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Ryu, S., Shenoy, K. V. 2011: III-32.
• Previews: New insights into motor cortex. Neuron. Graziano, M. S. 2011; 71: 387-388
• Dynamical segmentation of single trials from population neural data. Advances in Neural Information Processing Systems (NIPS) Petreska, B., Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S. I., Shenoy, K. V. edited by Shawe-Taylor, J., Zemel, R., S., Bartlett, P. MIT Press. 2011: 1
• Modelling low-dimensional dynamics in recorded spiking populations. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Macke, J., Busing, L., Cunningham, J. P., Yu, B. M., Shenoy, K. V., Sahani, M. 2011: I-34.
• Detecting changes in neural dynamics within single trials. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Petreska, B., Cunningham, J. P., Santhanam, G., Yu, B. M., Ryu, S. I., Shenoy, K. V. 2011: I-33.
• Cortical preparatory activity avoids causing movement by remaining in a muscle-neutral space. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Kaufman, M. T., Churchland, M. M., Shenoy, K. V. 2011: II-61.
• Empirical models of spiking in neural populations. Advances in Neural Information Processing Systems (NIPS) Macke, J., Buesing, L., Cunningham, J. P., Yu, B. M., Shenoy, K. V., Sahani, M. edited by Shawe-Taylor, J., Zemel, R., S., Bartlett, P. MIT Press Cambridge, MA.. 2011: 1
• A dynamical systems view of motor preparation: Implications for neural prosthetic system design ENHANCING PERFORMANCE FOR ACTION AND PERCEPTION: MULTISENSORY INTEGRATION, NEUROPLASTICITY AND NEUROPROSTHETICS, PT II Shenoy, K. V., Kaufman, M. T., Sahani, M., Churchland, M. M. 2011; 192: 33-58

  Abstract

  Neural prosthetic systems aim to help disabled patients suffering from a range of neurological injuries and disease by using neural activity from the brain to directly control assistive devices. This approach in effect bypasses the dysfunctional neural circuitry, such as an injured spinal cord. To do so, neural prostheses depend critically on a scientific understanding of the neural activity that drives them. We review here several recent studies aimed at understanding the neural processes in premotor cortex that precede arm movements and lead to the initiation of movement. These studies were motivated by hypotheses and predictions conceived of within a dynamical systems perspective. This perspective concentrates on describing the neural state using as few degrees of freedom as possible and on inferring the rules that govern the motion of that neural state. Although quite general, this perspective has led to a number of specific predictions that have been addressed experimentally. It is hoped that the resulting picture of the dynamical role of preparatory and movement-related neural activity will be particularly helpful to the development of neural prostheses, which can themselves be viewed as dynamical systems under the control of the larger dynamical system to which they are attached.

  View details for DOI 10.1016/B978-0-444-53355-5.00003-8

  View details for Web of Science ID 000310992900004

  View details for PubMedID 21763517

  View details for PubMedCentralID PMC3665515

• Monkey Models for Brain-Machine Interfaces: The Need for Maintaining Diversity 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Nuyujukian, P., Fan, J. M., Gilja, V., Kalanithi, P. S., Chestek, C. A., Shenoy, K. V. IEEE. 2011: 1301–1305

  Abstract

  Brain-machine interfaces (BMIs) aim to help disabled patients by translating neural signals from the brain into control signals for guiding prosthetic arms, computer cursors, and other assistive devices. Animal models are central to the development of these systems and have helped enable the successful translation of the first generation of BMIs. As we move toward next-generation systems, we face the question of which animal models will aid broader patient populations and achieve even higher performance, robustness, and functionality. We review here four general types of rhesus monkey models employed in BMI research, and describe two additional, complementary models. Given the physiological diversity of neurological injury and disease, we suggest a need to maintain the current diversity of animal models and to explore additional alternatives, as each mimic different aspects of injury or disease.

  View details for Web of Science ID 000298810001110

  View details for PubMedID 22254555

• Extracting rotational structure from motor cortical data. Frontiers in Neuroscience.Conference Abstract: Computational and Systems Neuroscience (COSYNE) Cunningham, J. P., Churchland, M. M., Kaufman, M. T., Shenoy, K. V. 2011: II-33.
• Combining Wireless Neural Recording and Video Capture for the Analysis of Natural Gait 5th International IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Conference on Neural Engineering (NER) Foster, J. D., Freifeld, O., Nuyujukian, P., Ryu, S. I., Black, M. J., Shenoy, K. V. IEEE. 2011: 613–616

  View details for Web of Science ID 000298735800147

• Spiking Neural Network Decoder for Brain-Machine Interfaces 5th International IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Conference on Neural Engineering (NER) Dethier, J., Gilja, V., Nuyujukian, P., Elassaad, S. A., Shenoy, K. V., Boahen, K. IEEE. 2011: 396–399

  View details for Web of Science ID 000298735800094

• Cortical Preparatory Activity: Representation of Movement or First Cog in a Dynamical Machine? NEURON Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Ryu, S. I., Shenoy, K. V. 2010; 68 (3): 387-400

  Abstract

  The motor cortices are active during both movement and movement preparation. A common assumption is that preparatory activity constitutes a subthreshold form of movement activity: a neuron active during rightward movements becomes modestly active during preparation of a rightward movement. We asked whether this pattern of activity is, in fact, observed. We found that it was not: at the level of a single neuron, preparatory tuning was weakly correlated with movement-period tuning. Yet, somewhat paradoxically, preparatory tuning could be captured by a preferred direction in an abstract "space" that described the population-level pattern of movement activity. In fact, this relationship accounted for preparatory responses better than did traditional tuning models. These results are expected if preparatory activity provides the initial state of a dynamical system whose evolution produces movement activity. Our results thus suggest that preparatory activity may not represent specific factors, and may instead play a more mechanistic role.

  View details for DOI 10.1016/j.neuron.2010.09.015

  View details for Web of Science ID 000284255800009

  View details for PubMedID 21040842

  View details for PubMedCentralID PMC2991102

• Autonomous head-mounted electrophysiology systems for freely behaving primates CURRENT OPINION IN NEUROBIOLOGY Gilja, V., Chestek, C. A., Nuyujukian, P., Foster, J., Shenoy, K. V. 2010; 20 (5): 676-686

  Abstract

  Recent technological advances have led to new light-weight battery-operated systems for electrophysiology. Such systems are head mounted, run for days without experimenter intervention, and can record and stimulate from single or multiple electrodes implanted in a freely behaving primate. Here we discuss existing systems, studies that use them, and how they can augment traditional, physically restrained, 'in-rig' electrophysiology. With existing technical capabilities, these systems can acquire multiple signal classes, such as spikes, local field potential, and electromyography signals, and can stimulate based on real-time processing of recorded signals. Moving forward, this class of technologies, along with advances in neural signal processing and behavioral monitoring, have the potential to dramatically expand the scope and scale of electrophysiological studies.

  View details for DOI 10.1016/j.conb.2010.06.007

  View details for Web of Science ID 000283481100022

  View details for PubMedID 20655733

  View details for PubMedCentralID PMC3401169

• Roles of Monkey Premotor Neuron Classes in Movement Preparation and Execution JOURNAL OF NEUROPHYSIOLOGY Kaufman, M. T., Churchland, M. M., Santhanam, G., Yu, B. M., Afshar, A., Ryu, S. I., Shenoy, K. V. 2010; 104 (2): 799-810

  Abstract

  Dorsal premotor cortex (PMd) is known to be involved in the planning and execution of reaching movements. However, it is not understood how PMd plan activity-often present in the very same neurons that respond during movement-is prevented from itself producing movement. We investigated whether inhibitory interneurons might "gate" output from PMd, by maintaining high levels of inhibition during planning and reducing inhibition during execution. Recently developed methods permit distinguishing interneurons from pyramidal neurons using extracellular recordings. We extend these methods here for use with chronically implanted multi-electrode arrays. We then applied these methods to single- and multi-electrode recordings in PMd of two monkeys performing delayed-reach tasks. Responses of putative interneurons were not generally in agreement with the hypothesis that they act to gate output from the area: in particular it was not the case that interneurons tended to reduce their firing rates around the time of movement. In fact, interneurons increased their rates more than putative pyramidal neurons during both the planning and movement epochs. The two classes of neurons also differed in a number of other ways, including greater modulation across conditions for interneurons, and interneurons more frequently exhibiting increases in firing rate during movement planning and execution. These findings provide novel information about the greater responsiveness of putative PMd interneurons in motor planning and execution and suggest that we may need to consider new possibilities for how planning activity is structured such that it does not itself produce movement.

  View details for DOI 10.1152/jn.00231.2009

  View details for Web of Science ID 000280932400023

  View details for PubMedID 20538784

  View details for PubMedCentralID PMC2934936

• HermesD: A High-Rate Long-Range Wireless Transmission System for Simultaneous Multichannel Neural Recording Applications IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS Miranda, H., Gilja, V., Chestek, C. A., Shenoy, K. V., Meng, T. H. 2010; 4 (3): 181-191

  Abstract

  HermesD is a high-rate, low-power wireless transmission system to aid research in neural prosthetic systems for motor disabilities and basic motor neuroscience. It is the third generation of our "Hermes systems" aimed at recording and transmitting neural activity from brain-implanted electrode arrays. This system supports the simultaneous transmission of 32 channels of broadband data sampled at 30 ks/s, 12 b/sample, using frequency-shift keying modulation on a carrier frequency adjustable from 3.7 to 4.1 GHz, with a link range extending over 20 m. The channel rate is 24 Mb/s and the bit stream includes synchronization and error detection mechanisms. The power consumption, approximately 142 mW, is low enough to allow the system to operate continuously for 33 h, using two 3.6-V/1200-mAh Li-SOCl2 batteries. The transmitter was designed using off-the-shelf components and is assembled in a stack of three 28 mm ? 28-mm boards that fit in a 38 mm ? 38 mm ? 51-mm aluminum enclosure, a significant size reduction over the initial version of HermesD. A 7-dBi circularly polarized patch antenna is used as the transmitter antenna, while on the receiver side, a 13-dBi circular horn antenna is employed. The advantages of using circularly polarized waves are analyzed and confirmed by indoor measurements. The receiver is a stand-alone device composed of several submodules and is interfaced to a computer for data acquisition and processing. It is based on the superheterodyne architecture and includes automatic frequency control that keeps it optimally tuned to the transmitter frequency. The HermesD communications performance is shown through bit-error rate measurements and eye-diagram plots. The sensitivity of the receiver is -83 dBm for a bit-error probability of 10(-9). Experimental recordings from a rhesus monkey conducting multiple tasks show a signal quality comparable to commercial acquisition systems, both in the low-frequency (local field potentials) and upper-frequency bands (action potentials) of the neural signals. This system can be easily scaled up in terms of the number of channels and data rate to accommodate future generations of Hermes systems.

  View details for DOI 10.1109/TBCAS.2010.2044573

  View details for Web of Science ID 000283121300006

• Stimulus onset quenches neural variability: a widespread cortical phenomenon NATURE NEUROSCIENCE Churchland, M. M., Yu, B. M., Cunningham, J. P., Sugrue, L. P., Cohen, M. R., Corrado, G. S., Newsome, W. T., Clark, A. M., Hosseini, P., Scott, B. B., Bradley, D. C., Smith, M. A., Kohn, A., Movshon, J. A., Armstrong, K. M., Moore, T., Chang, S. W., Snyder, L. H., Lisberger, S. G., Priebe, N. J., Finn, I. M., Ferster, D., Ryu, S. I., Santhanam, G., Sahani, M., Shenoy, K. V. 2010; 13 (3): 369-U25

  Abstract

  Neural responses are typically characterized by computing the mean firing rate, but response variability can exist across trials. Many studies have examined the effect of a stimulus on the mean response, but few have examined the effect on response variability. We measured neural variability in 13 extracellularly recorded datasets and one intracellularly recorded dataset from seven areas spanning the four cortical lobes in monkeys and cats. In every case, stimulus onset caused a decline in neural variability. This occurred even when the stimulus produced little change in mean firing rate. The variability decline was observed in membrane potential recordings, in the spiking of individual neurons and in correlated spiking variability measured with implanted 96-electrode arrays. The variability decline was observed for all stimuli tested, regardless of whether the animal was awake, behaving or anaesthetized. This widespread variability decline suggests a rather general property of cortex, that its state is stabilized by an input.

  View details for DOI 10.1038/nn.2501

  View details for Web of Science ID 000274860100020

  View details for PubMedID 20173745

  View details for PubMedCentralID PMC2828350

• Low-Dimensional Neural Features Predict Muscle EMG Signals 32nd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBC 10) Rivera-Alvidrez, Z., Kalmar, R. S., Ryu, S. I., Shenoy, K. V. IEEE. 2010: 6027–6033

  Abstract

  Understanding the relationship between neural activity in motor cortex and muscle activity during movements is important both for basic science and for the design of neural prostheses. While there has been significant work in decoding muscle EMG from neural data, decoders often require many parameters which make the analysis susceptible to overfitting, which reduces generalizability and makes the results difficult to interpret. To address this issue, we recorded simultaneous neural activity from the motor cortices (M1/PMd) of rhesus monkeys performing an arm-reaching task while recording EMG from arm muscles. In this work, we focused on relating the mean neural activity (averaged across reach trials to one target) to the corresponding mean EMG. We reduced the dimensionality of the neural data and found that the curvature of the low-dimensional (low-D) neural activity could be used as a signature of muscle activity. Using this signature, and without directly fitting EMG data to the neural activity, we derived neural axes based on reaches to only one reach target (< 5% of the data) that could explain EMG for reaches across multiple targets (average R(2) = 0.65). Our results suggest that cortical population activity is tightly related to muscle EMG measurements, predicting a lag between cortical activity and movement generation of 47.5 ms. Furthermore, our ability to predict EMG features across different movements suggests that there are fundamental axes or directions in the low-D neural space along which the neural population activity moves to activate particular muscles.

  View details for Web of Science ID 000287964006107

  View details for PubMedID 21097116

• The roles of monkey premotor neuron classes in movement preparation and execution. Journal of Neurophysiology. Kaufman, M. T., Churchland, M. M., Santhanam, G., Yu, B. M., Afshar, A., Ryu, S. I., Shenoy, Krishna, V. 2010; 104: 799-810.
• A high-performance cortically-controlled motor prosthesis enabled by a feedback control perspective. Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Ryu, S. I., Shenoy, Krishna, V. 2010
• An online, closed-loop testing platform for neural prosthetic systems. Cunningham, J. P., Nuyujukian, P., Gilja, V., Chestek, C. A., Ryu, S. I., Shenoy, K. V. 2010
• Waveform stability and neural decoder performance across 7 weeks. Chestek, C. A., Gilja, V., Nuyujukian, P., Foster, J. D., Kaufman, M. T., Ryu, S. I., Shenoy, Krishna, V. 2010
• Low dimensional neural features predict specific muscle EMG signals. Rivera-Alvidrez, Z., Kalmar, R., Ryu, S. I., Shenoy, K. V. 2010
• Ensemble activity underlying movement preparation in prearcuate cortex. Frontiers in Neuroscience. Conference Abstract: Computational and Systems Neuroscience (COSYNE) Kalmar, R., Reppas, J., Ryu, S. I., Shenoy, K. V., Newsome, W. T. 2010

  View details for DOI 10.3389/conf.fnins.2010.03.00228

• Toward human cortical prostheses: Addressing the performance barrier to clinical reality. Abstract #164. Congress of Neurological Surgeons Annual Meeting Abstracts Ryu, S. I., Gilja, V., Nuyujukian, P., Chestek, C. A., Yu, B. M., Shenoy, K. V. 2010
• Neural decoding for motor and communication prostheses. Chapter in Statistical Signal Processing for Neuroscience Yu, B. M., Santhanam, G., Sahani, M., Shenoy, K. V. edited by Elsevier. Elsevier. 2010: 219–263.
• Editorial overview -- Special section on New Technologies. Current Opinion in Neurobiology. Schuman, E., Zhuang, X. 2010; 20: 608-609.
• Preparatory tuning in premotor cortex relates most closely to the pop- ulation movement-epoch response. Frontiers in Neuroscience. Conference Abstract: Computational and Systems Neuroscience (COSYNE) Churchland, M. M., Kaufman, M. T., Cunningham, J. P., Shenoy, K. V. 2010

  View details for DOI 10.3389/conf.fnins.2010.03.00253

• High-performance continuous neural cursor control enabled by a feedback control perspective. Frontiers in Neuroscience. Frontiers in Neuroscience. Conference Abstract: Computational and Systems Neuroscience (COSYNE), Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Ryu, S. I., Shenoy, Krishna, V. 2010

  View details for DOI 10.3389/conf.fnins.2010.03.00249

• Motor systems CURRENT OPINION IN NEUROBIOLOGY El Manira, A., Shenoy, K. 2009; 19 (6): 570-571

  View details for DOI 10.1016/j.conb.2009.10.015

  View details for Web of Science ID 000273864300002

  View details for PubMedID 19897357

• Methods for estimating neural firing rates, and their application to brain-machine interfaces NEURAL NETWORKS Cunningham, J. P., Gilja, V., Ryu, S. I., Shenoy, K. V. 2009; 22 (9): 1235-1246

  Abstract

  Neural spike trains present analytical challenges due to their noisy, spiking nature. Many studies of neuroscientific and neural prosthetic importance rely on a smoothed, denoised estimate of a spike train's underlying firing rate. Numerous methods for estimating neural firing rates have been developed in recent years, but to date no systematic comparison has been made between them. In this study, we review both classic and current firing rate estimation techniques. We compare the advantages and drawbacks of these methods. Then, in an effort to understand their relevance to the field of neural prostheses, we also apply these estimators to experimentally gathered neural data from a prosthetic arm-reaching paradigm. Using these estimates of firing rate, we apply standard prosthetic decoding algorithms to compare the performance of the different firing rate estimators, and, perhaps surprisingly, we find minimal differences. This study serves as a review of available spike train smoothers and a first quantitative comparison of their performance for brain-machine interfaces.

  View details for DOI 10.1016/j.neunet.2009.02.004

  View details for Web of Science ID 000272073800005

  View details for PubMedID 19349143

  View details for PubMedCentralID PMC2783748

• Factor-Analysis Methods for Higher-Performance Neural Prostheses JOURNAL OF NEUROPHYSIOLOGY Santhanam, G., Yu, B. M., Gilja, V., Ryu, S. I., Afshar, A., Sahani, M., Shenoy, K. V. 2009; 102 (2): 1315-1330

  Abstract

  Neural prostheses aim to provide treatment options for individuals with nervous-system disease or injury. It is necessary, however, to increase the performance of such systems before they can be clinically viable for patients with motor dysfunction. One performance limitation is the presence of correlated trial-to-trial variability that can cause neural responses to wax and wane in concert as the subject is, for example, more attentive or more fatigued. If a system does not properly account for this variability, it may mistakenly interpret such variability as an entirely different intention by the subject. We report here the design and characterization of factor-analysis (FA)-based decoding algorithms that can contend with this confound. We characterize the decoders (classifiers) on experimental data where monkeys performed both a real reach task and a prosthetic cursor task while we recorded from 96 electrodes implanted in dorsal premotor cortex. The decoder attempts to infer the underlying factors that comodulate the neurons' responses and can use this information to substantially lower error rates (one of eight reach endpoint predictions) by 150 ms, although still advantageous at shorter timescales, that Gaussian-based algorithms performed better than the analogous Poisson-based algorithms and that the FA algorithm is robust even with a limited amount of training data. We propose that FA-based methods are effective in modeling correlated trial-to-trial neural variability and can be used to substantially increase overall prosthetic system performance.

  View details for DOI 10.1152/jn.00097.2009

  View details for Web of Science ID 000269500400061

  View details for PubMedID 19297518

  View details for PubMedCentralID PMC2724333

• Wireless Neural Recording With Single Low-Power Integrated Circuit IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Harrison, R. R., Kier, R. J., Chestek, C. A., Gilja, V., Nuyujukian, P., Ryu, S., Greger, B., Solzbacher, F., Shenoy, K. V. 2009; 17 (4): 322-329

  Abstract

  We present benchtop and in vivo experimental results from an integrated circuit designed for wireless implantable neural recording applications. The chip, which was fabricated in a commercially available 0.6- mum 2P3M BiCMOS process, contains 100 amplifiers, a 10-bit analog-to-digital converter (ADC), 100 threshold-based spike detectors, and a 902-928 MHz frequency-shift-keying (FSK) transmitter. Neural signals from a selected amplifier are sampled by the ADC at 15.7 kSps and telemetered over the FSK wireless data link. Power, clock, and command signals are sent to the chip wirelessly over a 2.765-MHz inductive (coil-to-coil) link. The chip is capable of operating with only two off-chip components: a power/command receiving coil and a 100-nF capacitor.

  View details for DOI 10.1109/TNSRE.2009.2023298

  View details for Web of Science ID 000268900300003

  View details for PubMedID 19497825

  View details for PubMedCentralID PMC2941647

• HermesC: Low-Power Wireless Neural Recording System for Freely Moving Primates IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Chestek, C. A., Gilja, V., Nuyujukian, P., Kier, R. J., Solzbacher, F., Ryu, S. I., Harrison, R. R., Shenoy, K. V. 2009; 17 (4): 330-338

  Abstract

  Neural prosthetic systems have the potential to restore lost functionality to amputees or patients suffering from neurological injury or disease. Current systems have primarily been designed for immobile patients, such as tetraplegics functioning in a rather static, carefully tailored environment. However, an active patient such as amputee in a normal dynamic, everyday environment may be quite different in terms of the neural control of movement. In order to study motor control in a more unconstrained natural setting, we seek to develop an animal model of freely moving humans. Therefore, we have developed and tested HermesC-INI3, a system for recording and wirelessly transmitting neural data from electrode arrays implanted in rhesus macaques who are freely moving. This system is based on the integrated neural interface (INI3) microchip which amplifies, digitizes, and transmits neural data across a approximately 900 MHz wireless channel. The wireless transmission has a range of approximately 4 m in free space. All together this device consumes 15.8 mA and 63.2 mW. On a single 2 A-hr battery pack, this device runs contiguously for approximately six days. The smaller size and power consumption of the custom IC allows for a smaller package (51 x 38 x 38 mm (3)) than previous primate systems. The HermesC-INI3 system was used to record and telemeter one channel of broadband neural data at 15.7 kSps from a monkey performing routine daily activities in the home cage.

  View details for DOI 10.1109/TNSRE.2009.2023293

  View details for Web of Science ID 000268900300004

  View details for PubMedID 19497829

• Gaussian-Process Factor Analysis for Low-Dimensional Single-Trial Analysis of Neural Population Activity JOURNAL OF NEUROPHYSIOLOGY Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S. I., Shenoy, K. V., Sahani, M. 2009; 102 (1): 614-635

  Abstract

  We consider the problem of extracting smooth, low-dimensional neural trajectories that summarize the activity recorded simultaneously from many neurons on individual experimental trials. Beyond the benefit of visualizing the high-dimensional, noisy spiking activity in a compact form, such trajectories can offer insight into the dynamics of the neural circuitry underlying the recorded activity. Current methods for extracting neural trajectories involve a two-stage process: the spike trains are first smoothed over time, then a static dimensionality-reduction technique is applied. We first describe extensions of the two-stage methods that allow the degree of smoothing to be chosen in a principled way and that account for spiking variability, which may vary both across neurons and across time. We then present a novel method for extracting neural trajectories-Gaussian-process factor analysis (GPFA)-which unifies the smoothing and dimensionality-reduction operations in a common probabilistic framework. We applied these methods to the activity of 61 neurons recorded simultaneously in macaque premotor and motor cortices during reach planning and execution. By adopting a goodness-of-fit metric that measures how well the activity of each neuron can be predicted by all other recorded neurons, we found that the proposed extensions improved the predictive ability of the two-stage methods. The predictive ability was further improved by going to GPFA. From the extracted trajectories, we directly observed a convergence in neural state during motor planning, an effect that was shown indirectly by previous studies. We then show how such methods can be a powerful tool for relating the spiking activity across a neural population to the subject's behavior on a single-trial basis. Finally, to assess how well the proposed methods characterize neural population activity when the underlying time course is known, we performed simulations that revealed that GPFA performed tens of percent better than the best two-stage method.

  View details for DOI 10.1152/jn.90941.2008

  View details for Web of Science ID 000267446000056

  View details for PubMedID 19357332

  View details for PubMedCentralID PMC2712272

• Human cortical prostheses: lost in translation? NEUROSURGICAL FOCUS Ryu, S. I., Shenoy, K. V. 2009; 27 (1)

  Abstract

  Direct brain control of a prosthetic system is the subject of much popular and scientific news. Neural technology and science have advanced to the point that proof-of-concept systems exist for cortically-controlled prostheses in rats, monkeys, and even humans. However, realizing the dream of making such technology available to everyone is still far off. Fortunately today there is great public and scientific interest in making this happen, but it will only occur when the functional benefits of such systems outweigh the risks. In this article, the authors briefly summarize the state of the art and then highlight many issues that will directly limit clinical translation, including system durability, system performance, and patient risk. Despite the challenges, scientists and clinicians are in the desirable position of having both public and fiscal support to begin addressing these issues directly. The ultimate challenge now is to determine definitively whether these prosthetic systems will become clinical reality or forever unrealized.

  View details for DOI 10.3171/2009.4.FOCUS0987

  View details for Web of Science ID 000268583300005

  View details for PubMedID 19569893

  View details for PubMedCentralID PMC3614414

• Neural Prosthetic Systems: Current Problems and Future Directions Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Chestek, C. A., Cunningham, J. P., Gilja, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. IEEE. 2009: 3369–3375

  Abstract

  By decoding neural activity into useful behavioral commands, neural prosthetic systems seek to improve the lives of severely disabled human patients. Motor decoding algorithms, which map neural spiking data to control parameters of a device such as a prosthetic arm, have received particular attention in the literature. Here, we highlight several outstanding problems that exist in most current approaches to decode algorithm design. These include two problems that we argue will unlikely result in further dramatic increases in performance, specifically spike sorting and spiking models. We also discuss three issues that have been less examined in the literature, and we argue that addressing these issues may result in dramatic future increases in performance. These include: non-stationarity of recorded waveforms, limitations of a linear mappings between neural activity and movement kinematics, and the low signal to noise ratio of the neural data. We demonstrate these problems with data from 39 experimental sessions with a non-human primate performing reaches and with recent literature. In all, this study suggests that research in cortically-controlled prosthetic systems may require reprioritization to achieve performance that is acceptable for a clinically viable human system.

  View details for Web of Science ID 000280543602209

  View details for PubMedID 19963796

• Guest Editorial -- Special section on wireless neural interfaces. IEEE TNSRE. Judy, J. W., Markovic, D. 2009; 17: 309-311.
• Human cortical prostheses: Lost in translation? Neurosurgical Focus Ryu, S. I., Shenoy, K. V. edited by guest, P. P. 2009
• Guest Editorial -- Special section on wireless neural interfaces. IEEE TNSRE Judy, J. W., Markovic, D. 2009; 17: 309-311.
• Stimulus onset quenches neural variability:a widespread cortical phenomenon. Frontiers in Systems Neuroscience. Conference Abstract: Computational and systems neuroscience. Churchland, M. M., Yu, B. M., Cunningham, J. C., Sugrue, L., Cohen, M., Corrado, G., Shenoy, Krishna, V. 2009

  View details for DOI 10.3389/conf.neuro.06.2009.03.295.

• Gaussian-process factor analysis for low-d single-trial analysis of neural population activity. Frontiers in Systems Neuroscience. Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S., Shenoy, K., Sahani, M. 2009

  View details for DOI 10.3389/conf.neuro.06.2009.03.153.

• A high-rate long-range wireless transmission system for multichannel neural recording applications IEEE International Symposium on Circuits and Systems (ISCAS 2009) Miranda, H., Gilja, V., Chestek, C., Shenoy, K. V., Meng, T. H. IEEE. 2009: 1265–1268

  View details for Web of Science ID 000275929800324

• Toward Optimal Target Placement for Neural Prosthetic Devices JOURNAL OF NEUROPHYSIOLOGY Cunningham, J. P., Yu, B. M., Gilja, V., Ryu, S. I., Shenoy, K. V. 2008; 100 (6): 3445-3457

  Abstract

  Neural prosthetic systems have been designed to estimate continuous reach trajectories (motor prostheses) and to predict discrete reach targets (communication prostheses). In the latter case, reach targets are typically decoded from neural spiking activity during an instructed delay period before the reach begins. Such systems use targets placed in radially symmetric geometries independent of the tuning properties of the neurons available. Here we seek to automate the target placement process and increase decode accuracy in communication prostheses by selecting target locations based on the neural population at hand. Motor prostheses that incorporate intended target information could also benefit from this consideration. We present an optimal target placement algorithm that approximately maximizes decode accuracy with respect to target locations. In simulated neural spiking data fit from two monkeys, the optimal target placement algorithm yielded statistically significant improvements up to 8 and 9% for two and sixteen targets, respectively. For four and eight targets, gains were more modest, as the target layouts found by the algorithm closely resembled the canonical layouts. We trained a monkey in this paradigm and tested the algorithm with experimental neural data to confirm some of the results found in simulation. In all, the algorithm can serve not only to create new target layouts that outperform canonical layouts, but it can also confirm or help select among multiple canonical layouts. The optimal target placement algorithm developed here is the first algorithm of its kind, and it should both improve decode accuracy and help automate target placement for neural prostheses.

  View details for DOI 10.1152/jn.90833.2008

  View details for Web of Science ID 000261449400040

  View details for PubMedID 18829845

  View details for PubMedCentralID PMC2604856

• Detecting neural-state transitions using hidden Markov models for motor cortical prostheses JOURNAL OF NEUROPHYSIOLOGY Kemere, C., Santhanam, G., Yu, B. M., Afshar, A., Ryu, S. I., Meng, T. H., Shenoy, K. V. 2008; 100 (4): 2441-2452

  Abstract

  Neural prosthetic interfaces use neural activity related to the planning and perimovement epochs of arm reaching to afford brain-directed control of external devices. Previous research has primarily centered on accurately decoding movement intention from either plan or perimovement activity, but has assumed that temporal boundaries between these epochs are known to the decoding system. In this work, we develop a technique to automatically differentiate between baseline, plan, and perimovement epochs of neural activity. Specifically, we use a generative model of neural activity to capture how neural activity varies between these three epochs. Our approach is based on a hidden Markov model (HMM), in which the latent variable (state) corresponds to the epoch of neural activity, coupled with a state-dependent Poisson firing model. Using an HMM, we demonstrate that the time of transition from baseline to plan epochs, a transition in neural activity that is not accompanied by any external behavior changes, can be detected using a threshold on the a posteriori HMM state probabilities. Following detection of the plan epoch, we show that the intended target of a center-out movement can be detected about as accurately as that by a maximum-likelihood estimator using a window of known plan activity. In addition, we demonstrate that our HMM can detect transitions in neural activity corresponding to targets not found in training data. Thus the HMM technique for automatically detecting transitions between epochs of neural activity enables prosthetic interfaces that can operate autonomously.

  View details for DOI 10.1152/jn.00924.2007

  View details for Web of Science ID 000259967000063

  View details for PubMedID 18614757

  View details for PubMedCentralID PMC2576226

• Cortical neural prosthesis performance improves when eye position is monitored IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Batista, A. P., Yu, B. M., Santhanam, G., Ryu, S. I., Afshar, A., Shenoy, K. V. 2008; 16 (1): 24-31

  Abstract

  Neural prostheses that extract signals directly from cortical neurons have recently become feasible as assistive technologies for tetraplegic individuals. Significant effort toward improving the performance of these systems is now warranted. A simple technique that can improve prosthesis performance is to account for the direction of gaze in the operation of the prosthesis. This proposal stems from recent discoveries that the direction of gaze influences neural activity in several areas that are commonly targeted for electrode implantation in neural prosthetics. Here, we first demonstrate that neural prosthesis performance does improve when eye position is taken into account. We then show that eye position can be estimated directly from neural activity, and thus performance gains can be realized even without a device that tracks eye position.

  View details for DOI 10.1109/TNSRE.2007.906958

  View details for Web of Science ID 000253442400004

  View details for PubMedID 18303802

• Signal processing challenges for neural prostheses IEEE SIGNAL PROCESSING MAGAZINE Linderman, M. D., Santhanam, G., Kemere, C. T., Gilja, V., O'Driscoll, S., Yu, B. M., Afshar, A., Ryu, S. I., Shenoy, K. V., Meng, T. H. 2008; 25 (1): 18-28

  View details for DOI 10.1109/MSP.2007.909016

  View details for Web of Science ID 000251906900006

• Single-trial representation of uncertainty about reach goals in macaque PMd. Rivera, Z. A., Kalmar, R., Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., Shenoy, Krishna, V. 2008
• Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. Advances in Neural Information Processing Systems (NIPS Yu, B. M., Cunningham, J. P., Ryu, S. I., Shenoy, K. V., Sahani, M. MIT Press. 2008: 1
• Low-frequency noise characterization of near-IR VCSELs for functional brain imaging Conference on Photonic Therapeutics and Diagnostics IV Lee, T. T., Lim, P. G., Harris, J. S., Shenoy, K. V., Smith, S. J. SPIE-INT SOC OPTICAL ENGINEERING. 2008

  View details for DOI 10.1117/12.764143

  View details for Web of Science ID 000255314100050

• Fast gaussian process methods for point process intensity estimation Cunningham, J. P., Sahani, M., Shenoy, K. V. 2008
• Neural basis of reach preparation. Shenoy, K. V. 2008
• HermesC: RF wireless low-power neural recording system for freely behaving primates. Gilja, V., Chestek, C., Nuyujukian, P., Ryu, S. I., Kier, R., Solzbacher, F., Shenoy, Krishna, V. 2008
• Inferring neural firing rates from spike trains using Gaussian processes. Advances in Neural Information Processing Systems (NIPS) Cunningham, J., Yu, B. M., Shenoy, K. V., ahani, M. edited by J, P., D, K., Y, S. MIT Press Cambridge, MA.. 2008: 1
• Brain-computer interfaces [from the guest editors]. IEEE Signal Processing Magazine. Sajda, P., Muller, K. R., Shenoy, K. V. 2008; 25: 16-17.
• Neural decoding of movements: From linear to nonlinear trajectory models 14th International Conference on Neural Information Processing (ICONIP 2007) Yu, B. M., Cunningham, J. P., Shenoy, K. V., Sahani, M. SPRINGER-VERLAG BERLIN. 2008: 586–595

  View details for Web of Science ID 000257246100061

• An Efficient Approximation for the Real-Time Implementation of the Mixture of Trajectory Models Decoder IEEE Biomedical Circuits and Systems Conference - Intelligent Biomedical Systems Bishop, W., Yu, B. M., Santhanam, G., Afshar, A., Ryu, S. I., Shenoy, K. V. IEEE. 2008: 133–136

  View details for Web of Science ID 000264876700035

• A Wireless Neural Interface for Chronic Recording IEEE Biomedical Circuits and Systems Conference - Intelligent Biomedical Systems Harrison, R. R., Kier, R. J., Kim, S., Rieth, L., Warren, D. J., Ledbetter, N. M., Clark, G. A., Solzbacher, F., Chestek, C. A., Gilja, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. IEEE. 2008: 125–128

  View details for Web of Science ID 000264876700033

• A Factor-Analysis decoder for high-performance neural prostheses 33rd IEEE International Conference on Acoustics, Speech and Signal Processing Santhanam, G., Yu, B. M., Gilja, V., Ryu, S. I., Afshar, A., Sahani, M., Shenoy, K. V. IEEE. 2008: 5208–5211

  View details for Web of Science ID 000257456703286

• The Use of a Virtual Integration Environment for the Real-Time Implementation of Neural Decode Algorithms 30th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Bishop, W., Yu, B. M., Santhanam, G., Afshar, A., Ryu, S. I., Shenoy, K. V., Vogelstein, R. J., Beaty, J., Harshbarger, S. IEEE. 2008: 628–633

  Abstract

  We have developed a virtual integration environment (VIE) for the development of neural prosthetic systems. This paper, the second of two companion articles, describes the use of the VIE as a common platform for the implementation of neural decode algorithms. In this paper, a linear filter decode and a recursive Bayesian algorithm are implemented as separate signal analysis modules of the VIE for the real-time decode of end effector trajectory. The process of implementing each algorithm is described and the real-time behavior as well as computational cost for each algorithm is examined. This is the first report of the real-time implementation of the Mixture of Trajectory Models decode [10]. These real-time algorithms can be easily interfaced with pre-existing modules of the VIE to control simulated and real devices.

  View details for Web of Science ID 000262404500158

  View details for PubMedID 19162734

• Wireless neural signal acquisition with single low-power integrated circuit IEEE International Symposium on Circuits and Systems Harrison, R. R., Kier, R. J., Greger, B., Solzbacher, F., Chestek, C. A., Gija, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V. IEEE. 2008: 1748–1751

  View details for Web of Science ID 000258532101168

• HermesC: RF wireless low-power neural recording system for freely behaving primates IEEE International Symposium on Circuits and Systems Chestek, C. A., Gija, V., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., Kier, R. J., Solzbacher, F., Harrison, R. R. IEEE. 2008: 1752–1755

  View details for Web of Science ID 000258532101169

• HermesB: A continuous neural recording system for freely behaving primates IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Santhanam, G., Linderman, M. D., Gija, V., Afshar, A., Ryu, S. I., Meng, T. H., Shenoy, K. V. 2007; 54 (11): 2037-2050

  Abstract

  Chronically implanted electrode arrays have enabled a broad range of advances in basic electrophysiology and neural prosthetics. Those successes motivate new experiments, particularly, the development of prototype implantable prosthetic processors for continuous use in freely behaving subjects, both monkeys and humans. However, traditional experimental techniques require the subject to be restrained, limiting both the types and duration of experiments. In this paper, we present a dual-channel, battery-powered neural recording system with an integrated three-axis accelerometer for use with chronically implanted electrode arrays in freely behaving primates. The recording system called HermesB, is self-contained, autonomous, programmable, and capable of recording broadband neural (sampled at 30 kS/s) and acceleration data to a removable compact flash card for up to 48 h. We have collected long-duration data sets with HermesB from an adult macaque monkey which provide insight into time scales and free behaviors inaccessible under traditional experiments. Variations in action potential shape and root-mean square (RMS) noise are observed across a range of time scales. The peak-to-peak voltage of action potentials varied by up to 30% over a 24-h period including step changes in waveform amplitude (up to 25%) coincident with high acceleration movements of the head. These initial results suggest that spike-sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance. During physically active periods (defined by head-mounted accelerometer), significantly reduced 5-25-Hz local field potential (LFP) power and increased firing rate variability were observed. Using a threshold fit to LFP power, 93% of 403 5-min recording blocks were correctly classified as active or inactive, potentially providing an efficient tool for identifying different behavioral contexts in prosthetic applications. These results demonstrate the utility of the HermesB system and motivate using this type of system to advance neural prosthetics and electrophysiological experiments.

  View details for DOI 10.1109/TBME.2007.895753

  View details for Web of Science ID 000250449200014

  View details for PubMedID 18018699

• Single-neuron stability during repeated reaching in macaque premotor cortex JOURNAL OF NEUROSCIENCE Chestek, C. A., Batista, A. P., Santhanam, G., Yu, B. M., Afshar, A., Cunningham, J. P., Gilja, V., Ryu, S. I., Churchland, M. M., Shenoy, K. V. 2007; 27 (40): 10742-10750

  Abstract

  Some movements that animals and humans make are highly stereotyped, repeated with little variation. The patterns of neural activity associated with repeats of a movement may be highly similar, or the same movement may arise from different patterns of neural activity, if the brain exploits redundancies in the neural projections to muscles. We examined the stability of the relationship between neural activity and behavior. We asked whether the variability in neural activity that we observed during repeated reaching was consistent with a noisy but stable relationship, or with a changing relationship, between neural activity and behavior. Monkeys performed highly similar reaches under tight behavioral control, while many neurons in the dorsal aspect of premotor cortex and the primary motor cortex were simultaneously monitored for several hours. Neural activity was predominantly stable over time in all measured properties: firing rate, directional tuning, and contribution to a decoding model that predicted kinematics from neural activity. The small changes in neural activity that we did observe could be accounted for primarily by subtle changes in behavior. We conclude that the relationship between neural activity and practiced behavior is reasonably stable, at least on timescales of minutes up to 48 h. This finding has significant implications for the design of neural prosthetic systems because it suggests that device recalibration need not be overly frequent, It also has implications for studies of neural plasticity because a stable baseline permits identification of nonstationary shifts.

  View details for DOI 10.1523/JNEUROSCI.0959-07.2007

  View details for Web of Science ID 000249981400012

  View details for PubMedID 17913908

• Techniques for extracting single-trial activity patterns from large-scale neural recordings CURRENT OPINION IN NEUROBIOLOGY Churchland, M. M., Yu, B. M., Sahani, M., Shenoy, K. V. 2007; 17 (5): 609-618

  Abstract

  Large, chronically implanted arrays of microelectrodes are an increasingly common tool for recording from primate cortex and can provide extracellular recordings from many (order of 100) neurons. While the desire for cortically based motor prostheses has helped drive their development, such arrays also offer great potential to advance basic neuroscience research. Here we discuss the utility of array recording for the study of neural dynamics. Neural activity often has dynamics beyond that driven directly by the stimulus. While governed by those dynamics, neural responses may nevertheless unfold differently for nominally identical trials, rendering many traditional analysis methods ineffective. We review recent studies - some employing simultaneous recording, some not - indicating that such variability is indeed present both during movement generation and during the preceding premotor computations. In such cases, large-scale simultaneous recordings have the potential to provide an unprecedented view of neural dynamics at the level of single trials. However, this enterprise will depend not only on techniques for simultaneous recording but also on the use and further development of analysis techniques that can appropriately reduce the dimensionality of the data, and allow visualization of single-trial neural behavior.

  View details for DOI 10.1016/j.conb.2007.11.001

  View details for Web of Science ID 000252835100016

  View details for PubMedID 18093826

  View details for PubMedCentralID PMC2238690

• Free-paced high-performance brain-computer interfaces JOURNAL OF NEURAL ENGINEERING Achtman, N., Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., Shenoy, K. V. 2007; 4 (3): 336-347

  Abstract

  Neural prostheses aim to improve the quality of life of severely disabled patients by translating neural activity into control signals for guiding prosthetic devices or computer cursors. We recently demonstrated that plan activity from premotor cortex, which specifies the endpoint of the upcoming arm movement, can be used to swiftly and accurately guide computer cursors to the desired target locations. However, these systems currently require additional, non-neural information to specify when plan activity is present. We report here the design and performance of state estimator algorithms for automatically detecting the presence of plan activity using neural activity alone. Prosthesis performance was nearly as good when state estimation was used as when perfect plan timing information was provided separately ( approximately 5 percentage points lower, when using 200 ms of plan activity). These results strongly suggest that a completely neurally-driven high-performance brain-computer interface is possible.

  View details for DOI 10.1088/1741-2560/4/3/018

  View details for Web of Science ID 000250181600025

  View details for PubMedID 17873435

• Reference frames for reach planning in macaque dorsal premotor cortex JOURNAL OF NEUROPHYSIOLOGY Batista, A. P., Santhanam, G., Yu, B. M., Ryu, S. I., Afshar, A., Shenoy, K. V. 2007; 98 (2): 966-983

  Abstract

  When a human or animal reaches out to grasp an object, the brain rapidly computes a pattern of muscular contractions that can acquire the target. This computation involves a reference frame transformation because the target's position is initially available only in a visual reference frame, yet the required control signal is a set of commands to the musculature. One of the core brain areas involved in visually guided reaching is the dorsal aspect of the premotor cortex (PMd). Using chronically implanted electrode arrays in two Rhesus monkeys, we studied the contributions of PMd to the reference frame transformation for reaching. PMd neurons are influenced by the locations of reach targets relative to both the arm and the eyes. Some neurons encode reach goals using limb-centered reference frames, whereas others employ eye-centered reference fames. Some cells encode reach goals in a reference frame best described by the combined position of the eyes and hand. In addition to neurons like these where a reference frame could be identified, PMd also contains cells that are influenced by both the eye- and limb-centered locations of reach goals but for which a distinct reference frame could not be determined. We propose two interpretations for these neurons. First, they may encode reach goals using a reference frame we did not investigate, such as intrinsic reference frames. Second, they may not be adequately characterized by any reference frame.

  View details for DOI 10.1152/jn.00421.2006

  View details for Web of Science ID 000248601100036

  View details for PubMedID 17581846

• Temporal complexity and heterogeneity of single-neuron activity in premotor and motor cortex JOURNAL OF NEUROPHYSIOLOGY Churchland, M. M., Shenoy, K. V. 2007; 97 (6): 4235-4257

  Abstract

  The relationship between neural activity in motor cortex and movement is highly debated. Although many studies have examined the spatial tuning (e.g., for direction) of cortical responses, less attention has been paid to the temporal properties of individual neuron responses. We developed a novel task, employing two instructed speeds, that allows meaningful averaging of neural responses across reaches with nearly identical velocity profiles. Doing so preserves fine temporal structure and reveals considerable complexity and heterogeneity of response patterns in primary motor and premotor cortex. Tuning for direction was prominent, but the preferred direction was frequently inconstant with respect to time, instructed-speed, and/or reach distance. Response patterns were often temporally complex and multiphasic, and varied with direction and instructed speed in idiosyncratic ways. A wide variety of patterns was observed, and it was not uncommon for a neuron to exhibit a pattern shared by no other neuron in our dataset. Response patterns of individual neurons rarely, if ever, matched those of individual muscles. Indeed, the set of recorded responses spanned a much higher dimensional space than would be expected for a model in which neural responses relate to a moderate number of factors-dynamic, kinematic, or otherwise. Complex responses may provide a basis-set representing many parameters. Alternately, it may be necessary to discard the notion that responses exist to "represent" movement parameters. It has been argued that complex and heterogeneous responses are expected of a recurrent network that produces temporally patterned outputs, and the present results would seem to support this view.

  View details for DOI 10.1152/jn.00095.2007

  View details for Web of Science ID 000247938200038

  View details for PubMedID 17376854

• Mixture of trajectory models for neural decoding of goal-directed movements JOURNAL OF NEUROPHYSIOLOGY Yu, B. M., Kemere, C., Santhanam, G., Afshar, A., Ryu, S. I., Meng, T. H., Sahani, M., Shenoy, K. V. 2007; 97 (5): 3763-3780

  Abstract

  Probabilistic decoding techniques have been used successfully to infer time-evolving physical state, such as arm trajectory or the path of a foraging rat, from neural data. A vital element of such decoders is the trajectory model, expressing knowledge about the statistical regularities of the movements. Unfortunately, trajectory models that both 1) accurately describe the movement statistics and 2) admit decoders with relatively low computational demands can be hard to construct. Simple models are computationally inexpensive, but often inaccurate. More complex models may gain accuracy, but at the expense of higher computational cost, hindering their use for real-time decoding. Here, we present a new general approach to defining trajectory models that simultaneously meets both requirements. The core idea is to combine simple trajectory models, each accurate within a limited regime of movement, in a probabilistic mixture of trajectory models (MTM). We demonstrate the utility of the approach by using an MTM decoder to infer goal-directed reaching movements to multiple discrete goals from multi-electrode neural data recorded in monkey motor and premotor cortex. Compared with decoders using simpler trajectory models, the MTM decoder reduced the decoding error by 38 (48) percent in two monkeys using 98 (99) units, without a necessary increase in running time. When available, prior information about the identity of the upcoming reach goal can be incorporated in a principled way, further reducing the decoding error by 20 (11) percent. Taken together, these advances should allow prosthetic cursors or limbs to be moved more accurately toward intended reach goals.

  View details for DOI 10.1152/jn.00482.2006

  View details for Web of Science ID 000247933500055

  View details for PubMedID 17329627

• Hit and miss. Nature. Dell, H. 2007; 445:36.
• The timecourse of neural variability in visual area MT. Churchland, M. M., Bradley, D. C., Clark, A., Hosseini, P., Cohen, M. R., Newsome, W. T., Shenoy, Krishna, V. 2007
• Optimizing spike sorting for brain computer interfaces with non-stationary waveforms. Gilja, V., Santhanam, G., Linderman, M., Afshar, A., Ryu, S. I., Meng, T. H., Shenoy, Krishna, V. 2007
• Single-trial representation of uncertainty about reach goals in macaque PMd. Rivera, Z., Kalmar, R., Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., Shenoy, Krishna, V. 2007
• Potential role of neural preparatory activity in optimal control theory Shenoy, K. V., Churchland, M. M. 2007
• Extracting dynamical structure embedded in premotor cortical activity. Shenoy, K. V., Yu, B. M., Afshar, A., Churchland, M. M., Cunningham, J. P., Sahani, M. 2007
• Neural correlates of movement preparation. Churchland, M. M., Shenoy, K. V. 2007
• Inferring neural firing rates from spike trains using Gaussian processes. Cunningham, J., Yu, B. M., Sahani, M., Shenoy, K. V. 2007
• Delay of movement caused by disruption of cortical preparatory activity JOURNAL OF NEUROPHYSIOLOGY Churchland, M. M., Shenoy, K. V. 2007; 97 (1): 348-359

  Abstract

  We tested the hypothesis that delay-period activity in premotor cortex is essential to movement preparation. During a delayed-reach task, we used subthreshold intracortical microstimulation to disrupt putative "preparatory" activity. Microstimulation led to a highly specific increase in reach reaction time. Effects were largest when activity was disrupted around the time of the go cue. Earlier disruptions, which presumably allowed movement preparation time to recover, had only a weak impact. Furthermore, saccadic reaction time showed little or no increase. Finally, microstimulation of nearby primary motor cortex, even when slightly suprathreshold, had little effect on reach reaction time. These findings provide the first evidence, of a causal and temporally specific nature, that activity in premotor cortex is fundamental to movement preparation. Furthermore, although reaction times were increased, the movements themselves were essentially unperturbed. This supports the suggestion that movement preparation is an active and actively monitored process and that movement can be delayed until inaccuracies are repaired. These results are readily interpreted in the context of the recently developed optimal-subspace hypothesis.

  View details for DOI 10.1152/jn.00808.2006

  View details for Web of Science ID 000243532900033

  View details for PubMedID 17005608

• A central source of movement variability NEURON Churchland, M. M., Afshar, A., Shenoy, K. V. 2006; 52 (6): 1085-1096

  Abstract

  Movements are universally, sometimes frustratingly, variable. When such variability causes error, we typically assume that something went wrong during the movement. The same assumption is made by recent and influential models of motor control. These posit that the principal limit on repeatable performance is neuromuscular noise that corrupts movement as it occurs. An alternative hypothesis is that movement variability arises before movements begin, during motor preparation. We examined this possibility directly by recording the preparatory activity of single cortical neurons during a highly practiced reach task. Small variations in preparatory neural activity were predictive of small variations in the upcoming reach. Effect magnitudes were such that at least half of the observed movement variability likely had its source during motor preparation. Thus, even for a highly practiced task, the ability to repeatedly plan the same movement limits our ability to repeatedly execute the same movement.

  View details for DOI 10.1016/j.neuron.2006.10.034

  View details for PubMedID 17178410

  View details for PubMedCentralID PMC1941679

• Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach JOURNAL OF NEUROPHYSIOLOGY Churchland, M. M., Santhanam, G., Shenoy, K. V. 2006; 96 (6): 3130-3146

  Abstract

  Neurons in premotor and motor cortex show preparatory activity during an instructed-delay task. It has been suggested that such activity primarily reflects visuospatial aspects of the movement, such as target location or reach direction and extent. We asked whether a more dynamic feature, movement speed, is also reflected. Two monkeys were trained to reach at different speeds ("slow" or "fast," peak speed being approximately 50-100% higher for the latter) depending on target color. Targets were presented in seven directions and at two distances. Of 95 neurons with tuned delay-period activity, 95, 78, and 94% showed a significant influence of direction, distance, and instructed speed, respectively. Average peak modulations with respect to direction, distance and speed were 18, 10, and 11 spikes/s. Although robust, modulations of firing rate with target direction were not necessarily invariant: for 45% of neurons, the preferred direction depended significantly on target distance and/or instructed speed. We collected an additional dataset, examining in more detail the effect of target distance (5 distances from 3 to 12 cm in 2 directions). Of 41 neurons with tuned delay-period activity, 85, 83, and 98% showed a significant impact of direction, distance, and instructed speed. Statistical interactions between the effects of distance and instructed speed were common, but it was nevertheless clear that distance "tuning" was not in general a simple consequence of speed tuning. We conclude that delay-period preparatory activity robustly reflects a nonspatial aspect of the upcoming reach. However, it is unclear whether the recorded neural responses conform to any simple reference frame, intrinsic or extrinsic.

  View details for DOI 10.1152/jn.00307.2006

  View details for Web of Science ID 000242177800033

  View details for PubMedID 16855111

• A high-performance brain-computer interface NATURE Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., Shenoy, K. V. 2006; 442 (7099): 195-198

  Abstract

  Recent studies have demonstrated that monkeys and humans can use signals from the brain to guide computer cursors. Brain-computer interfaces (BCIs) may one day assist patients suffering from neurological injury or disease, but relatively low system performance remains a major obstacle. In fact, the speed and accuracy with which keys can be selected using BCIs is still far lower than for systems relying on eye movements. This is true whether BCIs use recordings from populations of individual neurons using invasive electrode techniques or electroencephalogram recordings using less- or non-invasive techniques. Here we present the design and demonstration, using electrode arrays implanted in monkey dorsal premotor cortex, of a manyfold higher performance BCI than previously reported. These results indicate that a fast and accurate key selection system, capable of operating with a range of keyboard sizes, is possible (up to 6.5 bits per second, or approximately 15 words per minute, with 96 electrodes). The highest information throughput is achieved with unprecedentedly brief neural recordings, even as recording quality degrades over time. These performance results and their implications for system design should substantially increase the clinical viability of BCIs in humans.

  View details for DOI 10.1038/nature04968

  View details for Web of Science ID 000238979700046

  View details for PubMedID 16838020

• Neural variability in premotor cortex provides a signature of motor preparation JOURNAL OF NEUROSCIENCE Churchland, M. M., Yu, B. M., Ryu, S. I., Santhanam, G., Shenoy, K. V. 2006; 26 (14): 3697-3712

  Abstract

  We present experiments and analyses designed to test the idea that firing rates in premotor cortex become optimized during motor preparation, approaching their ideal values over time. We measured the across-trial variability of neural responses in dorsal premotor cortex of three monkeys performing a delayed-reach task. Such variability was initially high, but declined after target onset, and was maintained at a rough plateau during the delay. An additional decline was observed after the go cue. Between target onset and movement onset, variability declined by an average of 34%. This decline in variability was observed even when mean firing rate changed little. We hypothesize that this effect is related to the progress of motor preparation. In this interpretation, firing rates are initially variable across trials but are brought, over time, to their "appropriate" values, becoming consistent in the process. Consistent with this hypothesis, reaction times were longer if the go cue was presented shortly after target onset, when variability was still high, and were shorter if the go cue was presented well after target onset, when variability had fallen to its plateau. A similar effect was observed for the natural variability in reaction time: longer (shorter) reaction times tended to occur on trials in which firing rates were more (less) variable. These results reveal a remarkable degree of temporal structure in the variability of cortical neurons. The relationship with reaction time argues that the changes in variability approximately track the progress of motor preparation.

  View details for DOI 10.1523/JNEUROSCI.3762-05.2006

  View details for Web of Science ID 000236552400012

  View details for PubMedID 16597724

• Neural recording stability of chronic electrode arrays in freely behaving primates. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Linderman, M. D., Gilja, V., Santhanam, G., Afshar, A., Ryu, S., Meng, T. H., Shenoy, K. V. 2006; 1: 4387-4391

  Abstract

  Chronically implanted electrode arrays have enabled a broad range of advances, particularly in the field of neural prosthetics. Those successes motivate development of prototype implantable prosthetic processors for long duration, continuous use in freely behaving subjects. However, traditional experimental protocols have provided limited information regarding the stability of the electrode arrays and their neural recordings. In this paper we present preliminary results derived from long duration neural recordings in a freely behaving primate which show variations in action potential shape and RMS noise across a range of time scales. These preliminary results suggest that spike sorting algorithms can no longer assume stable neural signals and will need to transition to adaptive signal processing methodologies to maximize performance.

  View details for PubMedID 17946626

• The activity of motor cortex neurons during reaches is temporally complex and exceedingly heterogeneous. Churchland, M. M., Shenoy, K. V. 2006
• Extracting dynamical structure embedded in neural activity. Neural Information Processing Systems (NIPS) Yu, B. M., Afshar, A., Santhanam, G., Ryu, S. I., Shenoy, K. V., Sahani, M. edited by Y, W., B, S., J, P. MIT Press, Cambridge, MA.. 2006: 1545–1552.
• Hidden Markov models for spatial and temporal estimation for prosthetic control. Abstract Viewer / Itinerary Planner. Kemere, C., Yu, B. M., Santhanam, G., Ryu, S. I., Afshar, A., Meng, T. H., Shenoy, Krishna, V. 2006
• Brain-Machine Interfaces introduction: Brain-machine interfaces promise to aid paralyzed patients by re-routing movement-related signals around damaged parts of the nervous system. A new study in Nature demonstrates a human with spinal injury manipulating a screen cursor and robotic devices by thought alone (Hochberg et al. Nature 442:164-171, 2006). Implanted electrodes in his motor cortex recorded neural activity, and translated it into movement commands. A second study, in monkeys, shows that brain-machine interfaces can operate at high speed, greatly increasing their clinical potential (Santhanam et al. Nature 442:195-198, 2006). This Nature Web Focus includes exclusive interviews and video footage of experiments, alongside papers that paved the way for these recent advances. Shenoy, K. V. 2006; 442: 164-171, 195-198
• Modulation of neuronal ensemble activity during movement planning in Parkinson's disease patients undergoing deep brain stimulation. Henderson, J. M., Afshar, A., Ryu, S. I., Hill, B. C., Bronte-Stewart, H. M., Shenoy, K. V. 2006
• Generating complex repeatable patterns of activity by gain modulating network neurons. Abstract Viewer / Itinerary Planner. Schaffer, E. S., Rajan, K., Churchland, M. M., Shenoy, K. V., Abbott, L. F. 2006
• Multiday electrophysiological recordings from freely behaving primates using an autonomous, multi-channel neural system. Abstract Viewer / Itinerary Planner. Gilja, V., Linderman, M. D., Santhanam, G., Afshar, A., Ryu, S. I., Meng, T. H., Shenoy, Krishna, V. 2006
• Neurons to Silicon: Implantable Prosthesis Processor. International Solid State Circuits Conference (ISSCC) O'Driscoll, S., Meng, T. H., Shenoy, K. V., Kemere, C. 2006: 552-553 & 672.
• Expectation propagation for inference in non-linear dynamical models with Poisson observations. Nonlinear Statistical Signal Processing Workshop Yu, B. M., Shenoy, K. V., Sahani, M. 2006
• Neurological disorders: Mind over machine. Nature Reviews Neuroscience Barton, S. 2006; 7: 682-683.
• Heterogeneous reference frames for reaching in macaque PMd. Batista, A. P., Santhanam, G., Yu, B., Ryu, S. I., Afshar, A., Shenoy, K. V. 2006
• A central source of movement variability Neuron. Churchland, M. M., Afshar, A., Shenoy, K. V. 2006; 52: 1085-1096.
• Acute implantation of high density microelectrode arrays for investigation of human cortex. Henderson, J. M., Afshar, A., Ryu, S. I., Hill, B. C., Bronte-Stewart, H. M., Shenoy, K. V. 2006
• Optimal target placement for neural communication prostheses. Abstract Viewer / Itinerary Planner. Atlanta Cunningham, J. P., Yu, B. M., Shenoy, K. V. 2006
• Influence of eye position on end-point decoding accuracy in dorsal premotor cortex. Abstract Viewer / Itinerary Planner. Batista, A. P., Yu, B. M., Santhanam, G., Ryu, S. I., Afshar, A., Shenoy, K. V. 2006
• The relationship between PMd neural activity and reaching behavior is stable in highly trained macaques. Abstract Viewer / Itinerary Planner. Chestek, C. A., Batista, A. P., Yu, B. M., Santhanam, G., Ryu, S. I., Afshar, A., Shenoy, Krishna, V. 2006
• Is this the bionic man Nature Shenoy, K. V. 2006; 442:109
• Neuroscience: Converting thoughts into action. Nature Scott, S. H. 2006; 442: 141-142.
• Preparing for speed. Focus on: Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. Journal of Neurophysiology Cisek, P. 2006; 96: 2842-2843.
• Integrated optical sensors for chronic, minimally-invasive imaging of brain function. Lee, T. T., O, L., Cang, J., Kaneko, M., Stryker, M. P., Smith, S. J., Shenoy, Krishna, V. 2006
• Optimal target placement for neural communication prostheses. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Cunningham, J. P., Yu, B. M., Shenoy, K. V. 2006; 1: 2912-2915

  Abstract

  Neural prosthetic systems have been designed to estimate continuous reach trajectories as well as discrete reach targets. In the latter case, reach targets are typically decoded from neural activity during an instructed delay period, before the reach begins. We have recently characterized the decoding speed and accuracy achievable by such a system. The results were obtained using canonical target layouts, independent of the tuning properties of the neurons available. Here we seek to increase decode accuracy by judiciously selecting the locations of the reach targets based on the characteristics of the neural population at hand. We present an optimal target placement algorithm that approximately maximizes decode accuracy with respect to target locations. Using maximum likelihood decoding, the optimal target placement algorithm yielded up to 11 and 12% improvement for two and sixteen targets, respectively. For four and eight targets, gains were more modest (5 and 3%, respectively) as the target layouts found by the algorithm closely resembled the canonical layouts. Thus, the algorithm can serve not only to find target layouts that outperform canonical layouts, but it can also confirm or help select among multiple canonical layouts. These results indicate that the optimal target placement algorithm is a valuable tool for designing high-performance prosthetic systems.

  View details for PubMedID 17945745

• Factor analysis with Poisson output. Technical Report NPSL-TR-06-1. Santhanam, G., Yu, B. M., Shenoy, K. V., Sahani, M. 2006
• Bionic brains become a reality Hopkin, M. 2006
• Neuroprosthetics: In search of the sixth sense. Nature Abbott, A. 2006; 442: 125-127.
• Neural rklecording stability of chronic electrode arrays in freely behaving primates 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Linderman, M. D., Gilja, V., Santhanam, G., Afshar, A., Ryu, S., Meng, T. H., Shenoy, K. V. IEEE. 2006: 3784–3788

  View details for Web of Science ID 000247284704099

• Optimal target placement for neural communication prostheses 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Cunningham, J. P., Yu, B. M., Shenoy, K. V. IEEE. 2006: 1063–1066

  View details for Web of Science ID 000247284701050

• Integrated semiconductor optical sensors for chronic, minimally-invasive imaging of brain function 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Lee, T. T., Levi, O., Cang, J., Kaneko, M., Stryker, M. P., Smith, S. J., Shenoy, K. V., Harris, J. S. IEEE. 2006: 2443–2446

  View details for Web of Science ID 000247284702192

• An autonomous, broadband, multi-channel neural recording system for freely behaving primates 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Linderman, M. D., Gilja, V., Santhanam, G., Afshar, A., Ryu, S., Meng, T. H., Shenoy, K. V. IEEE. 2006: 3780–3783

  View details for Web of Science ID 000247284704098

• Multiday electrophysiological recordings from freely behaving primates 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Gilja, V., Linderman, M. D., Santhanam, G., Afshar, A., Shenoy, K. V. IEEE. 2006: 5723–5726

  View details for Web of Science ID 000247284706158

• Multiday electrophysiological recordings from freely behaving primates. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Gilja, V., Linderman, M. D., Santhanam, G., Afshar, A., Ryu, S., Meng, T. H., Shenoy, K. V. 2006; 1: 5643-5646

  Abstract

  Continuous multiday broadband neural data provide a means for observing effects at fine timescales over long periods. In this paper we present analyses on such data sets to demonstrate neural correlates for physically active and inactive time periods, as defined by the response of a head-mounted accelerometer. During active periods, we found that 5-25 Hz local field potential (LFP) power was significantly reduced, firing rate variability increased, and firing rates have greater temporal correlation. Using a single threshold fit to LFP power, 93% of the 403 5 minute blocks tested were correctly classified as active or inactive (as labeled by thresholding each block's maximal accelerometer magnitude). These initial results motivate the use of such data sets for testing neural prosthetics systems and for finding the neural correlates of natural behaviors.

  View details for PubMedID 17947159

• An autonomous, broadband, multi-channel neural recording system for freely behaving primates. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Linderman, M. D., Gilja, V., Santhanam, G., Afshar, A., Ryu, S., Meng, T. H., Shenoy, K. V. 2006; 1: 1212-1215

  Abstract

  Successful laboratory proof-of-concept experiments with neural prosthetic systems motivate continued algorithm and hardware development. For these efforts to move beyond traditional fixed laboratory setups, new tools are needed to enable broadband, multi-channel, long duration neural recording from freely behaving primates. In this paper we present a dual-channel, battery powered, neural recording system with integrated 3-axis accelerometer for use with chronically implanted electrode arrays. The recording system, called HermesB, is self-contained, autonomous, programmable and capable of recording broadband neural and head acceleration data to a removable compact flash card for up to 48 hours.

  View details for PubMedID 17946450

• Increasing the performance of cortically-controlled prostheses. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Shenoy, K. V., Santhanam, G., Ryu, S. I., Afshar, A., Yu, B. M., Gilja, V., Linderman, M. D., Kalmar, R. S., Cunningham, J. P., Kemere, C. T., Batista, A. P., Churchland, M. M., Meng, T. H. 2006: 6652-6656

  Abstract

  Neural prostheses have received considerable attention due to their potential to dramatically improve the quality of life of severely disabled patients. Cortically-controlled prostheses are able to translate neural activity from cerebral cortex into control signals for guiding computer cursors or prosthetic limbs. Non-invasive and invasive electrode techniques can be used to measure neural activity, with the latter promising considerably higher levels of performance and therefore functionality to patients. We review here some of our recent experimental and computational work aimed at establishing a principled design methodology to increase electrode-based cortical prosthesis performance to near theoretical limits. Studies discussed include translating unprecedentedly brief periods of "plan" activity into high information rate (6.5 bits/s)control signals, improving decode algorithms and optimizing visual target locations for further performance increases, and recording from chronically implanted arrays in freely behaving monkeys to characterize neuron stability. Taken together, these results should substantially increase the clinical viability of cortical prostheses.

  View details for PubMedID 17959477

• Integrated semiconductor optical sensors for chronic, minimally-invasive imaging of brain function. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Lee, T. T., Levi, O., Cang, J., Kaneko, M., Stryker, M. P., Smith, S. J., Shenoy, K. V., Harris, J. S. 2006; 1: 1025-1028

  Abstract

  Intrinsic optical signal (IOS) imaging is a widely accepted technique for imaging brain activity. We propose an integrated device consisting of interleaved arrays of gallium arsenide (GaAs) based semiconductor light sources and detectors operating at telecommunications wavelengths in the near-infrared. Such a device will allow for long-term, minimally invasive monitoring of neural activity in freely behaving subjects, and will enable the use of structured illumination patterns to improve system performance. In this work we describe the proposed system and show that near-infrared IOS imaging at wavelengths compatible with semiconductor devices can produce physiologically significant images in mice, even through skull.

  View details for PubMedID 17946016

• Power feasibility of implantable digital spike sorting circuits for neural prosthetic systems IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Zumsteg, Z. S., Kemere, C., O'Driscoll, S., Santhanam, G., Ahmed, R. E., Shenoy, K. V., Meng, T. H. 2005; 13 (3): 272-279

  Abstract

  A new class of neural prosthetic systems aims to assist disabled patients by translating cortical neural activity into control signals for prosthetic devices. Based on the success of proof-of-concept systems in the laboratory, there is now considerable interest in increasing system performance and creating implantable electronics for use in clinical systems. A critical question that impacts system performance and the overall architecture of these systems is whether it is possible to identify the neural source of each action potential (spike sorting) in real-time and with low power. Low power is essential both for power supply considerations and heat dissipation in the brain. In this paper we report that state-of-the-art spike sorting algorithms are not only feasible using modern complementary metal oxide semiconductor very large scale integration processes, but may represent the best option for extracting large amounts of data in implantable neural prosthetic interfaces.

  View details for DOI 10.1109/TNSRE.2005.854307

  View details for Web of Science ID 000231969500004

  View details for PubMedID 16200751

• A high performance neurally-controlled cursor positioning system 2nd International IEEE/EMBS Conference on Neural Engineering Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., Shenoy, K. V. IEEE. 2005: 494–500

  View details for Web of Science ID 000229610400133

• Extracting dynamical structure embedded in neural activity. 2005 Abstract Viewer/Itinerary Planner. Sahani, M., Yu, B. M., Afshar, G., Santhanam, G., Ryu, S. I., Shenoy, K. V. 2005
• PMd delay activity during rapid sequential movement plans. 2005 Abstract Viewer/Itinerary Planner. Kalmar, R. S., Gilja, V., Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., Shenoy, Krishna, V. 2005
• Reconfigurable Neural-Prosthetics Processors. Toward Replacement Parts for the Brain Implantable Biomimetic Electronics as Neural Prostheses Mumbru, J., Shenoy, K. V., Panotopoulos, G., Ay, S., An, X., Mok, F. edited by Berger, T., Glanzman, D. MIT Press, Cambridge, MA.. 2005: 335–368.
• Extracting dynamical structure embedded in motor preparatory activity. Yu, B. M., Afshar, A., Shenoy, K. V., Sahani, M. 2005
• Trial-by-trial mean normalization improves plan period reach target decoding. 2005 Abstract Viewer/Itinerary Planner. Gilja, V., Kalmar, R. S., Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., Shenoy, Krishna, V. 2005
• Free-paced target estimation in a delayed reach task. 2005 Abstract Viewer/Itinerary Planner. Afshar, A., Achtman, N., Santhanam, G., Ryu, S. I., Yu, B. M., Shenoy, K. V. 2005
• Complex patterns of motor cortex activity during reaches at different speeds. 2005 Abstract Viewer/Itinerary Planner. Churchland, M. M., Shenoy, K. V. 2005
• Heterogeneous coordinate frames for reaching in macaque PMd. 2005 Abstract Viewer/Itinerary Planner. Batista, A. P., Santhanam, G., Yu, B. M., Ryu, S. I., Afshar, A., Shenoy, K. V. 2005
• Feedback-directed state transition for recursive Bayesian estimation of goal-directed trajectories. Yu, B. M., Santhanam, G., Ryu, S. I., Shenoy, K. V. 2005
• Neural variability in premotor cortex provides a signature of motor preparation. Churchland, M. M., Yu, B. M., Ryu, S., Santhanam, G., Shenoy, K. V. 2005
• Motor preparation and settling activity in PMd. Neural Control of Movement (NCM) Annual Meeting Churchland, M. M., Yu, B. M., Ryu, S. I., Santhanam, G., Shenoy, K. V. 2005
• Mixture of trajectory models for neural decoding of goal-directed movements. 2005 Abstract Viewer/Itinerary Planner. Yu, B. M., Kemere, C., Santhanam, G., Afshar, A., Ryu, S. I., Meng, T. H., Shenoy, Krishna, V. 2005
• Intra-cortical communication prosthesis design. 2005 Abstract Viewer/Itinerary Planner. Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A., Afshar, K. V. 2005
• Model-based neural decoding of reaching movements: A maximum likelihood approach IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Kemere, C., Shenoy, K. V., Meng, T. H. 2004; 51 (6): 925-932

  Abstract

  A new paradigm for decoding reaching movements from the signals of an ensemble of individual neurons is presented. This new method not only provides a novel theoretical basis for the task, but also results in a significant decrease in the error of reconstructed hand trajectories. By using a model of movement as a foundation for the decoding system, we show that the number of neurons required for reconstruction of the trajectories of point-to-point reaching movements in two dimensions can be halved. Additionally, using the presented framework, other forms of neural information, specifically neural "plan" activity, can be integrated into the trajectory decoding process. The decoding paradigm presented is tested in simulation using a database of experimentally gathered center-out reaches and corresponding neural data generated from synthetic models.

  View details for DOI 10.1109/TBME.2004.826675

  View details for Web of Science ID 000221578000008

  View details for PubMedID 15188860

• Improving neural prosthetic system performance by combining plan and peri-movement activity. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Yu, B. M., Ryu, S. I., Santhanam, G., Churchland, M. M., Shenoy, K. V. 2004; 6: 4516-4519

  Abstract

  While most neural prosthetic systems to date estimate arm movements based solely on the activity prior to reaching movements during a delay period (plan activity) or solely on the activity during reaching movements (peri-movement activity), we show that decode classification can be improved by 56% and 71% respectively by using both types of activity together. We recorded from the pre-motor cortex of a rhesus monkey performing a delayed-reach task to one of seven targets. We found that taking into account the time-varying structure in peri-movement activity further improved performance by 15%, while doing the same for plan activity did not improve performance. We also found low correlations in activity between pairs of simultaneously-recorded units and across time periods within a given trial condition. These results show that decode performance can be significantly improved by combining information from the plan and peri-movement periods, and that there is nearly no loss in performance when assuming independence between units and across tune periods within a given trial condition.

  View details for PubMedID 17271310

• Reaction time and the time-course of cortical pre-motor processing. Soc. for Neurosci. Churchland, M. M., Yu, B., Ryu, S. I., Santhanam, G., Shenoy, K. V. 2004
• Behavioral variability predicted from recorded plan activity. Churchland, M. M., Shenoy, K. 2004
• Coordinate frames for reaching in macaque dorsal premotor cortex (PMd). Soc. for Neurosci. Batista, A. P., Yu, B. M., Santhanam, G., Ryu, S. I., Shenoy, K. V. 2004
• Contribution of motor preparation and execution noise to goal-irrelevant movement variability. Soc. for Neurosci. Afshar, A., hurchland, M. M., Shenoy, K. V. 2004
• Settling recurrent networks underlie motor planning in the primate brain. Churchland, M. M., Yu, B. M., Ryu, S. I., Santhanam, G., Shenoy, K. V. 2004
• The speed at which reach movement plans can be decoded from the cortex and its implications for high performance neural prosthetic arm systems. Ryu, S. I., Santhanam, G., Yu, B. M., Shenoy, K. V. 2004
• Role of movement preparation in movement generation. Churchland, M. M., Yu, B. M., Ryu, S. I., Santhanam, G., Afshar, A., Shenoy, K. V. 2004
• Contribution of motor preparation and execution noise to goal-irrelevant movement variability. Afshar, A., Churchland, M. M., Shenoy, K. V. 2004
• Reconstruction of arm trajectories from plan and peri-movement motor cortical activity. Kemere, C., Santhanam, G., Ryu, S. I., Yu, B. M., Meng, T. H., Shenoy, K. V. 2004
• Improving neural prosthetic system performance by combining plan and peri-movement activity. Soc. for Neurosci. Yu, B. M., Ryu, S. I., Santhanam, G., Churchland, M. M., Shenoy, K. V. 2004
• Changes in reaction time induced by microstimulation in PMd. Soc. for Neurosci. Shenoy, K. V., Churchland, M. M. 2004
• High speed neural prosthetic icon positioning. Soc. for Neurosci. Ryu, S. I., Santhanam, G., Yu, B. M., Shenoy, K. V. 2004
• Premotor cortex plan activity used to decode upcoming reach speed for high-performance neural prosthetic system design. Ryu, S. I., Yu, B. M., Churchland, M. M., Shenoy, K. V. 2004
• High information transmission rates in a neural prosthetic system. Soc. for Neurosci. Santhanam, G., Ryu, S. I., Yu, B. M., Shenoy, K. V. 2004
• Reconstruction of arm trajectories from plan and peri-movement motor cortical activity. Soc. for Neurosci. Kemere, C., Santhanam, G., Ryu, S. I., Yu, B. M., Meng, T. H., Shenoy, K. V. 2004
• Validation of adaptive threshold spike detector for neural recording 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Watkins, P. T., Santhanam, G., Shenoy, K. V., Harrison, R. R. IEEE. 2004: 4079–4082

  View details for Web of Science ID 000225461801064

• Validation of adaptive threshold spike detector for neural recording. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Watkins, P. T., Santhanam, G., Shenoy, K. V., Harrison, R. R. 2004; 6: 4079-4082

  Abstract

  We compare the performance of algorithms for automatic spike detection in neural recording applications. Each algorithm sets a threshold based on an estimate of the background noise level. The adaptive spike detection algorithm is suitable for implementation in analog VLSI; results from a proof-of-concept chip using neural data are presented. We also present simulation results of algorithm performance on neural data and compare it to other methods of threshold level adjustment based on the root-mean-square (rms) voltage measured over a finite window. We show that the adaptive spike detection algorithm measures the background noise level accurately despite the presence of large-amplitude action potentials and multi-unit hash. Simulation results enable us to optimize the algorithm parameters, leading to an improved spike detector circuit that is currently being developed.

  View details for PubMedID 17271196

• Power feasibility of implantable digital spike-sorting circuits for neural prosthetic systems 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Zumsteg, Z. S., Ahmed, R. E., Santhanam, G., Shenoy, K. V., Meng, T. H. IEEE. 2004: 4237–4240

  View details for Web of Science ID 000225461801107

• Local field potential measurement with low-power analog integrated circuit. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Harrison, R. R., Santhanam, G., Shenoy, K. V. 2004; 6: 4067-4070

  Abstract

  Local field potentials (LFPs) in the brain are an important source of information for basic research and clinical (i.e., neuroprosthetic) applications. The energy contained in certain bands of LFPs in the 10-100 Hz range has been shown to correlate with specific arm movement parameters in nonhuman primates. In the near future, implantable devices will need to transmit neural information from hundreds of microelectrodes, and transcutaneous data transfer will become a significant bottleneck. Here we present a low-power, fully-integrated circuit that performs on-site data reduction by isolating LFPs and measuring their signal energy. The resulting analog VLSI circuit consumes 586 microm x 79 microm of silicon area and dissipates only 5 nanowatts of power. We show that the chip performs similarly to state-of-the-art signal processing algorithms.

  View details for PubMedID 17271193

• Power feasibility of implantable digital spike-sorting circuits for neural prosthetic systems. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Zumsteg, Z. S., Ahmed, R. E., Santhanam, G., Shenoy, K. V., Meng, T. H. 2004; 6: 4237-4240

  Abstract

  A new class of neural prosthetic systems aims to assist disabled patients by translating cortical neural activity into control signals for prosthetic devices. Based on the success of proof-of-concept systems in the laboratory, there is now considerable interest in increasing system performance and creating implantable electronics for use in clinical systems. A critical question that impacts system performance and the overall architecture of these systems is whether it is possible to identify the neural source of each action potential (spike sorting) in real-time and with low power. Low power is essential both for power supply considerations and heat dissipation in the brain. In this paper we report that several state-of-the-art spike sorting algorithms implemented in modern CMOS VLSI processes are expected to be power realistic.

  View details for PubMedID 17271239

• Improving neural prosthetic system performance for a fixed number of neurons. Yu, B. M., Ryu, S., Churchland, M. M., Shenoy, K. V. 2004
• Local field potential measurement with low-power analog integrated circuit 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Harrison, R. R., Santhanam, G., Shenoy, K. V. IEEE. 2004: 4067–4070

  View details for Web of Science ID 000225461801061

• Improving neural prosthetic system performance by combining plan and peri-movement activity 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Yu, B. A., Ryu, S. I., Santhanam, G., Churchland, M. M., Shenoy, K. V. IEEE. 2004: 4516–4519

  Abstract

  While most neural prosthetic systems to date estimate arm movements based solely on the activity prior to reaching movements during a delay period (plan activity) or solely on the activity during reaching movements (peri-movement activity), we show that decode classification can be improved by 56% and 71% respectively by using both types of activity together. We recorded from the pre-motor cortex of a rhesus monkey performing a delayed-reach task to one of seven targets. We found that taking into account the time-varying structure in peri-movement activity further improved performance by 15%, while doing the same for plan activity did not improve performance. We also found low correlations in activity between pairs of simultaneously-recorded units and across time periods within a given trial condition. These results show that decode performance can be significantly improved by combining information from the plan and peri-movement periods, and that there is nearly no loss in performance when assuming independence between units and across tune periods within a given trial condition.

  View details for Web of Science ID 000225461801180

• Model-based decoding of reaching movements for prosthetic systems 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Kemere, C., Santhanam, G., Yu, B. M., Ryu, S., Meng, T., Shenoy, K. V. IEEE. 2004: 4524–4528

  Abstract

  Model-based decoding of neural activity for neuroprosthetic systems has been shown, in simulation, to provide significant gain over traditional linear filter approaches. We tested the model-based decoding approach with real neural and behavioral data and found a 18% reduction in trajectory reconstruction error compared with a linear filter. This corresponds to a 40% reduction in the number of neurons required for equivalent performance. The model-based approach further permits the combination of target-tuned plan activity with movement activity. The addition of plan activity reduced reconstruction error by 23% relative to the linear filter, corresponding to 55% reduction in the number of neurons required. Taken together, these results indicate that a decoding algorithm employing a prior model of reaching kinematics can substantially improve trajectory estimates, thereby improving prosthetic system performance.

  View details for Web of Science ID 000225461801182

• Model-based decoding of reaching movements for prosthetic systems. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Kemere, C., Santhanam, G., Yu, B. M., Ryu, S., Meng, T., Shenoy, K. V. 2004; 6: 4524-4528

  Abstract

  Model-based decoding of neural activity for neuroprosthetic systems has been shown, in simulation, to provide significant gain over traditional linear filter approaches. We tested the model-based decoding approach with real neural and behavioral data and found a 18% reduction in trajectory reconstruction error compared with a linear filter. This corresponds to a 40% reduction in the number of neurons required for equivalent performance. The model-based approach further permits the combination of target-tuned plan activity with movement activity. The addition of plan activity reduced reconstruction error by 23% relative to the linear filter, corresponding to 55% reduction in the number of neurons required. Taken together, these results indicate that a decoding algorithm employing a prior model of reaching kinematics can substantially improve trajectory estimates, thereby improving prosthetic system performance.

  View details for PubMedID 17271312

• An extensible infrastructure for fully automated spike sorting during online experiments. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Santhanam, G., Sahani, M., Ryu, S., Shenoy, K. 2004; 6: 4380-4384

  Abstract

  When recording extracellular neural activity, it is often necessary to distinguish action potentials arising from distinct cells near the electrode tip, a process commonly referred to as "spike sorting." In a number of experiments, notably those that involve direct neuroprosthetic control of an effector, this cell-by-cell classification of the incoming signal must be achieved in real time. Several commercial offerings are available for this task, but all of these require some manual supervision per electrode, making each scheme cumbersome with large electrode counts. We present a new infrastructure that leverages existing unsupervised algorithms to sort and subsequently implement the resulting signal classification rules for each electrode using a commercially available Cerebus neural signal processor. We demonstrate an implementation of this infrastructure to classify signals from a cortical electrode array, using a probabilistic clustering algorithm (described elsewhere). The data were collected from a rhesus monkey performing a delayed center-out reach task. We used both sorted and unsorted (thresholded) action potentials from an array implanted in pre-motor cortex to "predict" the reach target, a common decoding operation in neuroprosthetic research. The use of sorted spikes led to an improvement in decoding accuracy of between 3.6 and 6.4%.

  View details for PubMedID 17271276

• An extensible infrastructure for fully automated spike sorting during online experiments 26th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Santhanam, G., Sahani, M., Ryu, S. I., Shenoy, K. V. IEEE. 2004: 4380–4384

  Abstract

  When recording extracellular neural activity, it is often necessary to distinguish action potentials arising from distinct cells near the electrode tip, a process commonly referred to as "spike sorting." In a number of experiments, notably those that involve direct neuroprosthetic control of an effector, this cell-by-cell classification of the incoming signal must be achieved in real time. Several commercial offerings are available for this task, but all of these require some manual supervision per electrode, making each scheme cumbersome with large electrode counts. We present a new infrastructure that leverages existing unsupervised algorithms to sort and subsequently implement the resulting signal classification rules for each electrode using a commercially available Cerebus neural signal processor. We demonstrate an implementation of this infrastructure to classify signals from a cortical electrode array, using a probabilistic clustering algorithm (described elsewhere). The data were collected from a rhesus monkey performing a delayed center-out reach task. We used both sorted and unsorted (thresholded) action potentials from an array implanted in pre-motor cortex to "predict" the reach target, a common decoding operation in neuroprosthetic research. The use of sorted spikes led to an improvement in decoding accuracy of between 3.6 and 6.4%.

  View details for Web of Science ID 000225461801145

• Neural prosthetic control signals from plan activity NEUROREPORT Shenoy, K. V., Meeker, D., Cao, S. Y., Kureshi, S. A., Pesaran, B., Buneo, C. A., Batista, A. R., Mitra, P. P., Burdick, J. W., Andersen, R. A. 2003; 14 (4): 591-596

  Abstract

  The prospect of assisting disabled patients by translating neural activity from the brain into control signals for prosthetic devices, has flourished in recent years. Current systems rely on neural activity present during natural arm movements. We propose here that neural activity present before or even without natural arm movements can provide an important, and potentially advantageous, source of control signals. To demonstrate how control signals can be derived from such plan activity we performed a computational study with neural activity previously recorded from the posterior parietal cortex of rhesus monkeys planning arm movements. We employed maximum likelihood decoders to estimate movement direction and to drive finite state machines governing when to move. Performance exceeded 90% with as few as 40 neurons.

  View details for DOI 10.1097/01.wnr.0000063250.41814.39

  View details for Web of Science ID 000182554200013

  View details for PubMedID 12657892

• Influence of movement speed on plan activity in monkey pre-motor cortex and implications for high-performance neural prosthetic system design 25th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Shenoy, K. V., Churchland, M. M., Santhanam, G., Yu, B. M., Ryu, S. I. IEEE. 2003: 1897–1900

  View details for Web of Science ID 000189395300496

• Movement speed alters distance tuning of plan activity in monkey pre-motor cortex. Soc. for Neurosci. Churchland, M. M., Shenoy, K. V. 2003
• Local field potential activity varies with reach distance, direction, and speed in monkey pre-motor cortex Soc. for Neurosci. G., Santhanam, Churchland, M. M., Sahani, M., Shenoy, K. V. 2003
• Neural prosthetic control signals from plan activity. NeuroReport Shenoy, K. V., Meeker, D., Cao, S., Kureshi, S. A., Pesaran, B., Mitra, P. 2003; 14: 591-596.
• Methods for estimating neural step sequences in neural prosthetic applications 1st International IEEE/EMBS Conference on Neural Engineering Santhanam, G., Shenoy, K. V. IEEE. 2003: 344–347

  View details for Web of Science ID 000182404500094

• Pursuit speed compensation in cortical area MSTd JOURNAL OF NEUROPHYSIOLOGY Shenoy, K. V., Crowell, J. A., Andersen, R. A. 2002; 88 (5): 2630-2647

  Abstract

  When we move forward the visual images on our retinas expand. Humans rely on the focus, or center, of this expansion to estimate their direction of self-motion or heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth pursuit eye movements add visual motion to the expanding retinal image and displace the focus of expansion. In spite of this, humans accurately judge their heading during pursuit eye movements even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd); neurons in this area are tuned to the retinal position of the focus and they modify their tuning to partially compensate for the focus shift caused by pursuit. However, the question remains whether these neurons shift focus tuning more at faster pursuit speeds, to compensate for the larger focus shifts created by faster pursuit. To investigate this question, we recorded from 40 MSTd neurons while monkeys made pursuit eye movements at a range of speeds across simulated self- or object motion displays. We found that most MSTd neurons modify their focus tuning more at faster pursuit speeds, consistent with the idea that they encode heading and other motion parameters regardless of pursuit speed. Across the population, the median rate of compensation increase with pursuit speed was 51% as great as required for perfect compensation. We recorded from the same neurons in a simulated pursuit condition, in which gaze was fixed but the entire display counter-rotated to produce the same retinal image as during real pursuit. This condition yielded the result that retinal cues contribute to pursuit compensation; the rate of compensation increase was 30% of that required for accurate encoding of heading. The difference between these two conditions was significant (P < 0.05), indicating that extraretinal cues also contribute significantly. We found a systematic antialignment between preferred pursuit and preferred visual motion directions. Neurons may use this antialignment to combine retinal and extraretinal compensatory cues. These results indicate that many MSTd neurons compensate for pursuit velocity, pursuit direction as previously reported and pursuit speed, and further implicate MSTd as a critical stage in the computation of egomotion.

  View details for DOI 10.1152/jn.00002.2001

  View details for Web of Science ID 000179080900042

  View details for PubMedID 12424299

• Decoding of plan and peri-movement neural signals in prosthetic systems IEEE Workshop on Signal Processing Systems (SIPS 02) Kemere, C. T., Santhanam, G., Yu, B. M., Shenoy, K. V., Meng, T. H. IEEE. 2002: 276–283

  View details for Web of Science ID 000179203300047

• Response of MSTd neurons to simulated 3D orientation of rotating planes JOURNAL OF NEUROPHYSIOLOGY Sugihara, H., Murakami, I., Shenoy, K. V., Andersen, R. A., Komatsu, H. 2002; 87 (1): 273-285

  Abstract

  We studied whether the dorsal division of the medial superior temporal area (MSTd) in the macaque has activity related to structure-from-motion (SFM) processing. As the simplest form of three-dimensional (3D) structure, we chose a planar stimulus and examined the relation between the neural responses and the simulated 3D orientation of the plane defined by motion cues. We recorded from 114 MSTd neurons while monkeys were performing a visual fixation task. These neurons responded to a basic set of optic flow patterns such as translation, expansion/contraction, and rotation. Responses of these neurons to rotating plane stimuli were examined to see whether the MSTd neurons exhibited selectivity to the tilt and slant that characterize the 3D orientation of the plane. We found that most MSTd neurons tested (97 of 114) responded to the plane stimuli, and many neurons (65 of 97) exhibited selectivity to tilt and/or slant. Of 97 neurons, 18% (17/97) were selective only to tilt, 24% (23/97) only to slant, and 26% (25/97) to both. Control experiments rejected the possibility that the selectivity could be explained solely by the sensitivity to local stimulus components such as local translation, local speed, and local speed gradients. These results suggest that MSTd neurons are sensitive to stimulus features specific to the perceived 3D orientation of the rotating plane stimuli and suggest that area MSTd is involved in SFM processing.

  View details for Web of Science ID 000173155700025

  View details for PubMedID 11784749

• Pursuit-Speed Compensation in Cortical Area MSTd. Journal of Neurophysiology. Shenoy, K. V., Crowell, J., Andersen, R. A. 2002; 88: 2630-2647.
• Response of MSTd neurons to simulated 3D-orientation of rotating planes. Journal of Neurophysiology Sugihara, H., Murakami, I., Shenoy, K. V., Andersen, R. A., Komatsu, H. 2002; 87: 273-285.
• Cognitive control signals for prosthetic systems. Soc. For Neurosci. Meeker, D., Shenoy, K. V., Cao, S., Pesaran, B., Scherberger, H., Jarvis, M. 2001; 27
• Neural mechanisms for self-motion perception in area MST. International review of neurobiology Andersen, R. A., Shenoy, K. V., Crowell, J. A., BRADLEY, D. C. 2000; 44: 219-233

  View details for PubMedID 10605648

• Neural mechanisms for self-motion perception in area MST. International Review of Neurobiology Andersen, R. A., Shenoy, K. V., Crowell, J. A., Bradley, D. C. Academic Press.. 2000: 219–233.
• Toward Adaptive Control of Neural Prosthetics by Parietal Cortex. Neural Information and Coding Workshop. Meeker, D., Shenoy, K. V., Kureshi, S., Cao, S., Burdick, J., Pesaran, B. 2000
• Influence of gaze rotation on the visual response of primate MSTd neurons JOURNAL OF NEUROPHYSIOLOGY Shenoy, K. V., BRADLEY, D. C., Andersen, R. A. 1999; 81 (6): 2764-2786

  Abstract

  When we move forward, the visual image on our retina expands. Humans rely on the focus, or center, of this expansion to estimate their direction of heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth rotation of the eyes adds nearly uniform visual motion to the expanding retinal image and causes a displacement of the retinal focus. In spite of this, humans accurately judge their heading during pursuit eye movements and during active, smooth head rotations even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd) because these neurons are tuned to the retinal position of the focus and they modify their tuning during pursuit to compensate partially for the focus shift. However, the question remains whether these neurons also shift focus tuning to compensate for smooth head rotations that commonly occur during gaze tracking. To investigate this question, we recorded from 80 MSTd neurons while monkeys tracked a visual target either by pursuing with their eyes or by vestibulo-ocular reflex cancellation (VORC; whole-body rotation with eyes fixed in head and head fixed on body). VORC is a passive, smooth head rotation condition that selectively activates the vestibular canals. We found that neurons shift their focus tuning in a similar way whether focus displacement is caused by pursuit or by VORC. Across the population, compensation averaged 88 and 77% during pursuit and VORC, respectively (tuning shift divided by the retinal focus to true heading difference). Moreover the degree of compensation during pursuit and VORC was correlated in individual cells (P < 0.001). Finally neurons that did not compensate appreciably tended to be gain-modulated during pursuit and VORC and may constitute an intermediate stage in the compensation process. These results indicate that many MSTd cells compensate for general gaze rotation, whether produced by eye-in-head or head-in-world rotation, and further implicate MSTd as a critical stage in the computation of heading. Interestingly vestibular cues present during VORC allow many cells to compensate even though humans do not accurately judge their heading in this condition. This suggests that MSTd may use vestibular information to create a compensated heading representation within at least a subpopulation of cells, which is accessed perceptually only when additional cues related to active head rotations are also present.

  View details for Web of Science ID 000081005800017

  View details for PubMedID 10368396

• The contributions of vestibular signals to the representations of space in the posterior parietal cortex Conference on Otolith Function in Spatial Orientation and Movement - Symposium in Memory of Volker Henn Andersen, R. A., Shenoy, K. V., SNYDER, L. H., BRADLEY, D. C., Crowell, J. A. NEW YORK ACAD SCIENCES. 1999: 282–292

  Abstract

  Vestibular signals play an important role in spatial orientation, perception of object location, and control of self-motion. Prior physiological research on vestibular information processing has focused on brainstem mechanisms; relatively little is known about the processing of vestibular information at the level of the cerebral cortex. Recent electrophysiological experiments examining the use of vestibular canal signals in two different perceptual tasks are described: computation of self motion and localization of visual stimuli in a world-centered reference frame. These two perceptual functions are mediated by different parts of the posterior parietal cortex, the former in the dorsal aspect of the medial superior temporal area (MSTd) and the latter in area 7a.

  View details for Web of Science ID 000081273000022

  View details for PubMedID 10372079

• Toward prosthetic systems controlled by parietal cortex. Soc. For Neurosci. Shenoy, K. V., Kureshi, S. A., Meeker, D., Gillikin, B. L., Dubowitz, D. J., Batista, A. P. 1999
• Influence of pursuit speed on the representation of heading in macaque MSTd. European Conf. on Visual Percep. Shenoy, K. V., Crowell, J. A., Andersen, R. A. 1999
• Prior visual motion affects self-motion judgments during eye movements. OVS/ARVO Crowell, J. A., Shenoy, K. V., Andersen, R. A. 1999; 40
• Influence of gaze rotation on the visual response of primate MSTd neurons. Journal of Neurophysiology Shenoy, K. V., Bradley, D. C., Andersen, R. A. 1999; 81: 2764-2786.
• Visual self-motion perception during head turns NATURE NEUROSCIENCE Crowell, J. A., Banks, M. S., Shenoy, K. V., Andersen, R. A. 1998; 1 (8): 732-737

  Abstract

  Extra-retinal information is critical in the interpretation of visual input during self-motion. Turning our eyes and head to track objects displaces the retinal image but does not affect our ability to navigate because we use extra-retinal information to compensate for these displacements. We showed observers animated displays depicting their forward motion through a scene. They perceived the simulated self-motion accurately while smoothly shifting the gaze by turning the head, but not when the same gaze shift was simulated in the display; this indicates that the visual system also uses extra-retinal information during head turns. Additional experiments compared self-motion judgments during active and passive head turns, passive rotations of the body and rotations of the body with head fixed in space. We found that accurate perception during active head turns is mediated by contributions from three extra-retinal cues: vestibular canal stimulation, neck proprioception and an efference copy of the motor command to turn the head.

  View details for Web of Science ID 000077323400019

  View details for PubMedID 10196591

• Selectivity of neurons to the 3D orientation of a rotating plane in area MSTd of the monkey. Soc. for Neurosci. Sugihara, H., Murakami, I., Komatsu, H., Shenoy, K. V., Andersen, R. A. 1998; 24
• Retinal and extra-retinal motion signals both affect the extent of gaze-shift compensation. IOVS/ARVO Crowell, J. A., Maxwell, M. A., Shenoy, K. V., Andersen, R. A. 1998; 39
• Neurons in area MSTd of the monkey have a selectivity to the 3D orientation of a rotating plane. Japan Soc. for Neurosci. Sugihara, H., Murakami, I., Komatsu, H., Shenoy, K. V., Andersen, R. A. 1998
• The influence of pursuit speed upon the representation of heading in Macaque cortical area MSTd. for Neurosci. Abstracts: Shenoy, K. V., Crowell, J. A., Andersen, R. A. 1998; 24
• Perception of heading is a brain in the neck. Nature Neuroscience Warren, W. H. 1998; 1: 647-649.
• Visual self-motion perception during head turns. Nature Neuroscience Crowell, J. A., Banks, M. S., Shenoy, K. V., Andersen, R. A. 1998; 1: 732-737.
• Self-motion path perception during head and body rotations. IOVS/ARVO Crowell, J. A., Banks, M. S., Shenoy, K. V., Andersen, R. A. 1997; 38
• Monolithic integration of SEEDs and VLSI GaAs circuits by epitaxy on electronics. EEE Photon. Technol. Lett. Wang, H., Luo, J., Shenoy, K. V., Fonstad, C. G., Psaltis, D. 1997; 9: 607-609.
• Perception and neural representation of heading during gaze-rotation. Soc. for Neurosci. Abstracts: Shenoy, K. V., Crowell, J. A., Bradley, D. C., Andersen, R. A. 1997; 23
• Mechanisms of heading perception in primate visual cortex SCIENCE BRADLEY, D. C., Maxwell, M., Andersen, R. A., Banks, M. S., Shenoy, K. V. 1996; 273 (5281): 1544-1547

  Abstract

  When we move forward while walking or driving, what we see appears to expand. The center or focus of this expansion tells us our direction of self-motion, or heading, as long as our eyes are still. However, if our eyes move, as when tracking a nearby object on the ground, the retinal image is disrupted and the focus is shifted away from the heading. Neurons in primate dorso-medial superior temporal area responded selectively to an expansion focus in a certain part of the visual field, and this selective region shifted during tracking eye movements in a way that compensated for the retinal focus shift. Therefore, these neurons account for the effect of eye movements on what we see as we travel forward through the world.

  View details for Web of Science ID A1996VG59700039

  View details for PubMedID 8703215

• Neural mechanisms for heading and structure-from-motion perception 61st Cold Spring Harbor Symposium on Function and Dysfunction in the Nervous System Andersen, R. A., BRADLEY, D. C., Shenoy, K. V. COLD SPRING HARBOR LAB PRESS, PUBLICATIONS DEPT. 1996: 15–25

  View details for Web of Science ID A1996XD58000004

  View details for PubMedID 9246431

• Heading computation during pursuit eye movements in cortical area MSTd. Soc. for Neurosci. Abstracts: Bradley, D. C., Maxwell, M., Andersen, R. A., Banks, M. S., Shenoy, K. V. 1996; 22
• Neural mechanisms for heading perception in primate visual cortex. Science Bradley, D. C., Maxwell, M., Andersen, R. A., Banks, M. S., Shenoy, K. V. 1996; 273: 1544-1547.
• Heading computation during head movements in macaque cortical area MSTd. Soc. for Neurosci. Abstracts: Shenoy, K. V., Bradley, D. C., Andersen, R. A. 1996; 22
• Elevated temperature stability of GaAs digital integrated circuits. IEEE Electron Device Lett. Braun, E. K., Shenoy, K. V., Fonstad, C. G., Mikkelson, J. M. 1996; 17: 37-39.
• Neuroscience: Researchers find neurons that may help us navigate. Science Barinaga, M. 1996; 273: 1489-1490.
• Monolithic optoelectronic circuit design and fabrication by epitaxial growth on commercial VLSI GaAs MESFETs. IEEE Photon. Technol. Lett. Shenoy, K. V., Fonstad, C. G., Grot, A. C., Psaltis, D. 1995; 7: 508-510.
• A technology for monolithic integration of high indium-fraction resonant tunneling diodes with commercial MESFET VLSI electronics. InP and Related Compounds. Aggarwal, R. J., Shenoy, K. V., Fonstad, C. G. 1995
• Monolithic optoelectronic VLSI design and fabrication for optical interconnects. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Ph.D. Thesis. Shenoy, K. V. 1995
• Computation by symmetry operations in a highly structured model of the brain. Phys. Rev. E McGrann, J. V., Shaw, G. L., Shenoy, K. V., Matthews, R. B. 1994; 49: 5830-5839.
• Learning and memory processes and the modularity of the brain. Neural Bases of Learning and Memory Leng, X., McGrann, J. V., Quillfeldt, J. A., Shaw, G. L., Shenoy, K. V. edited by Delacour, J. World Scientific Press.. 1994: 1
• Comparison of Si/CMOS and GaAs MESFET technologies for analog optoelectronic circuits. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1994
• Large scale integration of LEDs and GaAs circuits fabricated through MOSIS. ICO/OSA/SPIE/ LEOS International Conf. On Optical Computing. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1994
• Integration of LEDs and GaAs circuits by MBE regrowth. IEEE Photon. Technol. Lett. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1994; 6: 819-821.
• Application specific OEICs fabricated using GaAs IC foundry services. Fonstad Jr., C, G, Shenoy, K. V. 1994
• High temperature stability of refractory-metal VLSI GaAs MESFETs. IEEE Electron Device Lett. Shenoy, K. V., Fonstad, C. G., Mikkelson, J. M. 1994; 15: 106-108.
• GaAs optoelectronic winner-take-all circuit. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1994
• Lowered temperature MBE regrowth of LED structures on high density GaAs circuits fabricated through MOSIS. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1993
• Optoelectronic VLSI circuit fabrication. Shenoy, K. V., Nuytkens, P., Fonstad, C. G., Johnson, G. D., Goodhue, W. D., Donnelly, J. 1993
• MBE regrowth of LEDs on VLSI GaAs MESFETs. Shenoy, K. V., Fonstad, C. G., Grot, A. C., Psaltis, D. 1993
• Learning by selection in the Trion model of cortical organization. Cerebral Cortex Shenoy, K. V., Kaufman, J., McGrann, J. V., Shaw, G. L. 1993; 3: 239-248.
• GaAs optoelectronic neuron circuits fabricated through MOSIS. Grot, A. C., Psaltis, D., Shenoy, K. V., Fonstad, C. G. 1993
• Laser diodes and refractory-metal gate VLSI GaAs MESFETs for smart pixels. Shenoy, K. V., Fonstad, C. G., Elman, B., Crawford, F. D., Mikkelson, J. M. 1992
• Selectional learning in the Trion model of cortical organization. Shenoy, K. V., Kaufman, J., McGrann, J., Shaw, G. L. 1989
• Rotational invariance in the Trion model of cortical organization. McGrann, J. V., Shenoy, K. V., Shaw, G. 1989