Academic Appointments

All Publications

  • Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation. Journal of neuroengineering and rehabilitation Song, S., Kidzinski, L., Peng, X. B., Ong, C., Hicks, J., Levine, S., Atkeson, C. G., Delp, S. L. 2021; 18 (1): 126


    Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This "Learn to Move" competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research.

    View details for DOI 10.1186/s12984-021-00919-y

    View details for PubMedID 34399772

  • Predicting gait adaptations due to ankle plantarflexor muscle weakness and contracture using physics-based musculoskeletal simulations. PLoS computational biology Ong, C. F., Geijtenbeek, T. n., Hicks, J. L., Delp, S. L. 2019; 15 (10): e1006993


    Deficits in the ankle plantarflexor muscles, such as weakness and contracture, occur commonly in conditions such as cerebral palsy, stroke, muscular dystrophy, Charcot-Marie-Tooth disease, and sarcopenia. While these deficits likely contribute to observed gait pathologies, determining cause-effect relationships is difficult due to the often co-occurring biomechanical and neural deficits. To elucidate the effects of weakness and contracture, we systematically introduced isolated deficits into a musculoskeletal model and generated simulations of walking to predict gait adaptations due to these deficits. We trained a planar model containing 9 degrees of freedom and 18 musculotendon actuators to walk using a custom optimization framework through which we imposed simple objectives, such as minimizing cost of transport while avoiding falling and injury, and maintaining head stability. We first generated gaits at prescribed speeds between 0.50 m/s and 2.00 m/s that reproduced experimentally observed kinematic, kinetic, and metabolic trends for walking. We then generated a gait at self-selected walking speed; quantitative comparisons between our simulation and experimental data for joint angles, joint moments, and ground reaction forces showed root-mean-squared errors of less than 1.6 standard deviations and normalized cross-correlations above 0.8 except for knee joint moment trajectories. Finally, we applied mild, moderate, and severe levels of muscle weakness or contracture to either the soleus (SOL) or gastrocnemius (GAS) or both of these major plantarflexors (PF) and retrained the model to walk at a self-selected speed. The model was robust to all deficits, finding a stable gait in all cases. Severe PF weakness caused the model to adopt a slower, "heel-walking" gait. Severe contracture of only SOL or both PF yielded similar results: the model adopted a "toe-walking" gait with excessive hip and knee flexion during stance. These results highlight how plantarflexor weakness and contracture may contribute to observed gait patterns.

    View details for DOI 10.1371/journal.pcbi.1006993

    View details for PubMedID 31589597

  • OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLoS computational biology Seth, A., Hicks, J. L., Uchida, T. K., Habib, A., Dembia, C. L., Dunne, J. J., Ong, C. F., DeMers, M. S., Rajagopal, A., Millard, M., Hamner, S. R., Arnold, E. M., Yong, J. R., Lakshmikanth, S. K., Sherman, M. A., Ku, J. P., Delp, S. L. 2018; 14 (7): e1006223


    Movement is fundamental to human and animal life, emerging through interaction of complex neural, muscular, and skeletal systems. Study of movement draws from and contributes to diverse fields, including biology, neuroscience, mechanics, and robotics. OpenSim unites methods from these fields to create fast and accurate simulations of movement, enabling two fundamental tasks. First, the software can calculate variables that are difficult to measure experimentally, such as the forces generated by muscles and the stretch and recoil of tendons during movement. Second, OpenSim can predict novel movements from models of motor control, such as kinematic adaptations of human gait during loaded or inclined walking. Changes in musculoskeletal dynamics following surgery or due to human-device interaction can also be simulated; these simulations have played a vital role in several applications, including the design of implantable mechanical devices to improve human grasping in individuals with paralysis. OpenSim is an extensible and user-friendly software package built on decades of knowledge about computational modeling and simulation of biomechanical systems. OpenSim's design enables computational scientists to create new state-of-the-art software tools and empowers others to use these tools in research and clinical applications. OpenSim supports a large and growing community of biomechanics and rehabilitation researchers, facilitating exchange of models and simulations for reproducing and extending discoveries. Examples, tutorials, documentation, and an active user forum support this community. The OpenSim software is covered by the Apache License 2.0, which permits its use for any purpose including both nonprofit and commercial applications. The source code is freely and anonymously accessible on GitHub, where the community is welcomed to make contributions. Platform-specific installers of OpenSim include a GUI and are available on

    View details for PubMedID 30048444

  • OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement PLOS COMPUTATIONAL BIOLOGY Seth, A., Hicks, J. L., Uchida, T. K., Habib, A., Dembia, C. L., Dunne, J. J., Ong, C. F., DeMers, M. S., Rajagopal, A., Millard, M., Hamner, S. R., Arnold, E. M., Yong, J. R., Lakshmikanth, S. K., Sherman, M. A., Ku, J. P., Delp, S. L. 2018; 14 (7)
  • Introduction to NIPS 2017 Competition Track NIPS'17 COMPETITION: BUILDING INTELLIGENT SYSTEMS Escalera, S., Weimer, M., Burtsev, M., Malykh, V., Logacheva, V., Lowe, R., Serban, I., Bengio, Y., Rudnicky, A., Black, A. W., Prabhumoye, S., Kidzinski, L., Mohanty, S., Ong, C. F., Hicks, J. L., Levine, S., Salathe, M., Delp, S., Huerga, I., Grigorenko, A., Thorbergsson, L., Das, A., Nemitz, K., Sandker, J., King, S., Ecker, A. S., Gatys, L. A., Bethge, M., Boyd-Graber, J., Feng, S., Rodriguez, P., Iyyer, M., He, H., Daume, H., McGregor, S., Banifatemi, A., Kurakin, A., Goodfellow, I., Bengio, S., Escalera, S., Weimer, M. 2018: 1–23
  • Learning to Run Challenge: Synthesizing Physiologically Accurate Motion Using Deep Reinforcement Learning NIPS'17 COMPETITION: BUILDING INTELLIGENT SYSTEMS Kidzinski, L., Mohanty, S. P., Ong, C. F., Hicks, J. L., Carroll, S. F., Levine, S., Salathe, M., Delp, S. L., Escalera, S., Weimer, M. 2018: 101–20
  • Learning to Run Challenge Solutions: Adapting Reinforcement Learning Methods for Neuromusculoskeletal Environments NIPS'17 COMPETITION: BUILDING INTELLIGENT SYSTEMS Kidzinski, L., Mohanty, S., Ong, C. F., Huang, Z., Zhou, S., Pechenko, A., Stelmaszczyk, A., Jarosik, P., Pavlov, M., Kolesnikov, S., Plis, S., Chen, Z., Zhang, Z., Chen, J., Shi, J., Zheng, Z., Yuan, C., Lin, Z., Michalewski, H., Milos, P., Osinski, B., Melnik, A., Schilling, M., Ritter, H., Carroll, S. F., Hicks, J., Levine, S., Salathe, M., Delp, S., Escalera, S., Weimer, M. 2018: 121–53
  • Simulation-Based Design for Wearable Robotic Systems: An Optimization Framework for Enhancing a Standing Long Jump IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Ong, C. F., Hicks, J. L., Delp, S. L. 2016; 63 (5): 894-903


    Technologies that augment human performance are the focus of intensive research and development, driven by advances in wearable robotic systems. Success has been limited by the challenge of understanding human-robot interaction. To address this challenge, we developed an optimization framework to synthesize a realistic human standing long jump and used the framework to explore how simulated wearable robotic devices might enhance jump performance.A planar, five-segment, seven-degree-of-freedom model with physiological torque actuators, which have variable torque capacity depending on joint position and velocity, was used to represent human musculoskeletal dynamics. An active augmentation device was modeled as a torque actuator that could apply a single pulse of up to 100 Nm of extension torque. A passive design was modeled as rotational springs about each lower limb joint. Dynamic optimization searched for physiological and device actuation patterns to maximize jump distance.Optimization of the nominal case yielded a 2.27 m jump that captured salient kinematic and kinetic features of human jumps. When the active device was added to the ankle, knee, or hip, jump distance increased to between 2.49 and 2.52 m. Active augmentation of all three joints increased the jump distance to 3.10 m. The passive design increased jump distance to 3.32 m by adding torques of 135, 365, and 297 Nm to the ankle, knee, and hip, respectively.Dynamic optimization can be used to simulate a standing long jump and investigate human-robot interaction.Simulation can aid in the design of performance-enhancing technologies.

    View details for DOI 10.1109/TBME.2015.2463077

    View details for Web of Science ID 000375001600002

    View details for PubMedID 26258930

  • A Simple Method for Amplifying RNA Targets (SMART) JOURNAL OF MOLECULAR DIAGNOSTICS McCalla, S. E., Ong, C., Sarma, A., Opal, S. M., Artenstein, A. W., Tripathi, A. 2012; 14 (4): 328-335


    We present a novel and simple method for amplifying RNA targets (named by its acronym, SMART), and for detection, using engineered amplification probes that overcome existing limitations of current RNA-based technologies. This system amplifies and detects optimal engineered ssDNA probes that hybridize to target RNA. The amplifiable probe-target RNA complex is captured on magnetic beads using a sequence-specific capture probe and is separated from unbound probe using a novel microfluidic technique. Hybridization sequences are not constrained as they are in conventional target-amplification reactions such as nucleic acid sequence amplification (NASBA). Our engineered ssDNA probe was amplified both off-chip and in a microchip reservoir at the end of the separation microchannel using isothermal NASBA. Optimal solution conditions for ssDNA amplification were investigated. Although KCl and MgCl(2) are typically found in NASBA reactions, replacing 70 mmol/L of the 82 mmol/L total chloride ions with acetate resulted in optimal reaction conditions, particularly for low but clinically relevant probe concentrations (≤100 fmol/L). With the optimal probe design and solution conditions, we also successfully removed the initial heating step of NASBA, thus achieving a true isothermal reaction. The SMART assay using a synthetic model influenza DNA target sequence served as a fundamental demonstration of the efficacy of the capture and microfluidic separation system, thus bridging our system to a clinically relevant detection problem.

    View details for DOI 10.1016/j.jmoldx.2012.02.001

    View details for Web of Science ID 000306247900005

    View details for PubMedID 22691910

    View details for PubMedCentralID PMC3391418

  • Ligation with Nucleic Acid Sequence-Based Amplification JOURNAL OF MOLECULAR DIAGNOSTICS Ong, C., Tai, W., Sarma, A., Opal, S. M., Artenstein, A. W., Tripathi, A. 2012; 14 (3): 206-213


    This work presents a novel method for detecting nucleic acid targets using a ligation step along with an isothermal, exponential amplification step. We use an engineered ssDNA with two variable regions on the ends, allowing us to design the probe for optimal reaction kinetics and primer binding. This two-part probe is ligated by T4 DNA Ligase only when both parts bind adjacently to the target. The assay demonstrates that the expected 72-nt RNA product appears only when the synthetic target, T4 ligase, and both probe fragments are present during the ligation step. An extraneous 38-nt RNA product also appears due to linear amplification of unligated probe (P3), but its presence does not cause a false-positive result. In addition, 40 mmol/L KCl in the final amplification mix was found to be optimal. It was also found that increasing P5 in excess of P3 helped with ligation and reduced the extraneous 38-nt RNA product. The assay was also tested with a single nucleotide polymorphism target, changing one base at the ligation site. The assay was able to yield a negative signal despite only a single-base change. Finally, using P3 and P5 with longer binding sites results in increased overall sensitivity of the reaction, showing that increasing ligation efficiency can improve the assay overall. We believe that this method can be used effectively for a number of diagnostic assays.

    View details for DOI 10.1016/j.jmoldx.2012.01.004

    View details for Web of Science ID 000303616600004

    View details for PubMedID 22449695

    View details for PubMedCentralID PMC3349837