Bio


Bernard Widrow is Professor Emeritus in the Electrical Engineering Department at Stanford University. His research focuses on adaptive signal processing, adaptive control systems, adaptive neural networks, human memory, and human-like memory for computers. Applications include signal processing, prediction, noise cancelling, adaptive arrays, control systems, and pattern recognition. Before coming to Stanford in 1959, he taught at MIT where he received the Doctor of Science Degree in 1956.

Academic Appointments


Honors & Awards


  • Citation Classic for paper "Adaptive Antenna Systems,'' Proceedings of the IEEE, December 1967, Institute of Electrical and Electronics Engineers (IEEE)
  • Benjamin Franklin Medal, The Franklin Institute (2001)
  • IEEE Millenium Medal, Institute of Electrical and Electronics Engineers (IEEE) (2000)
  • Signal Processing Society Award, Institute of Electrical and Electronics Engineers (IEEE) (1999)
  • Silicon Valley Engineering Hall of Fame, Silicon Valley Engineering Council (1999)
  • Member, National Academy of Engineering (1995)
  • Neural Networks Pioneer Medal, Institute of Electrical and Electronics Engineers (IEEE) (1991)
  • Alexander Graham Bell Medal, Institute of Electrical and Electronics Engineers (IEEE) (1986)
  • Centennial Medal, Institute of Electrical and Electronics Engineers (IEEE) (1984)
  • Fellow, American Association for the Advancement of Science (1980)
  • Fellow, Institute of Electrical and Electronics Engineers (IEEE) (1976)
  • Franqui Lecture Chair, University of Louvain, Belgium, (1967)

Boards, Advisory Committees, Professional Organizations


  • Chair, Silicon Valley Engineering Council Hall of Fame Awards Committee (2006 - Present)
  • Editorial Board, Neural Networks (2014 - Present)
  • Associate Editor, Circuits, Systems and Signal Processing (2014 - Present)
  • Associate Editor, Information Sciences (2014 - Present)
  • Associate Editor, Pattern Recognition (2014 - Present)
  • President, International Neural Network Society (1989 - 1990)
  • Governing Board Member, International Neural Network Society (1988 - 1991)
  • Chairman, DARPA Neural Network Study (1987 - 1988)

Professional Education


  • Sc.D., Massachusetts Institute of Technology, Electrical Engineering (1956)
  • S.M., Massachusetts Institute of Technology, Electrical Engineering (1953)
  • S.B., Massachusetts Institute of Technology, Electrical Engineering (1951)

Patents


  • B. Widrow, J.C. Aragon, B.M. Percival. "United States Patent 7,333,963 Cognitive Memory and Auto-Associative Neural Network Based Search Engine for Computer and Network Located Images and Photographs", Feb 1, 2008
  • B. Widrow. "United States Patent 7,187,907 Simultaneous Two-Way Transmission of Information Signals in the Same Frequency Band", Mar 1, 2007
  • M.A. Lehr and B. Widrow. "United States Patent 5,793,875 Directional Hearing System", Aug 1, 1998
  • B. Widrow. "United States Patent 5,737,430 Directional Hearing Aid", Apr 1, 1998
  • J. Rector, B. Marion, B. Widrow, and I.A. Salehi. "United States Patent 5,191,557 Signal Processing to Enable Utilization of a Rig Reference Sensor with a Drill Bit Seismic Source", Mar 1, 1993
  • J. Rector, B. Marion, B. Widrow, and I.A. Salehi. "United States Patent 5,050,130 Signal Processing to Enable Utilization of a Rig Reference Sensor with a Drill Bit Seismic Source", Sep 1, 1991
  • B. Widrow. "United States Patent 4,964,087 Seismic Processing and Imaging with a Drill-Bit Source", Oct 1, 1990
  • J. Rector, B. Marion, B. Widrow, and I.A. Salehi. "United States Patent 4,926,391 Signal Processing to Enable Utilization of a Rig Reference Sensor with a Drill Bit Seismic Source", May 1, 1990
  • B. Widrow. "United States Patent 4,858,130 Estimation of Hydraulic Fracture Geometry f rom Pumping Pressure Measurements", Aug 1, 1989
  • B. Widrow. "United States Patent 4,849,945 Seismic Processing and Imaging with a Drill- Bit Source", Jul 1, 1989
  • B. Widrow and M.N. Brearley. "United States Patent 4,751,738 Directional Hearing Aid", Jun 1, 1988
  • B. Widrow. "United States Patent 4,556,962 Seismic Exploration Method and Apparatus for Cancelling Interference from Seismic Vibration Source", Dec 1, 1985
  • B. Widrow. "United States Patent 4,537,200 ECG Enhancement by Adaptive Cancellation of Electrosurgical Interference", Aug 1, 1985
  • B. Widrow. "United States Patent 4,363,112 Apparatus and Method for Determining the Posi tion of a Gas-Saturated Porus Rock in the Vicinty of a Deep Borehole in the Earth", Dec 1, 1982
  • B. Widrow. "United States Patent 4,365,322 Apparatus and Method for Determining the Position of a Gas-Saturated Porus Rock in the Vicinty of a Deep Borehole in the Earth", Dec 1, 1982
  • J.R. Zeidler, J.M. McCool, and B. Widrow. "United States Patent 4,355,368 Adaptive Correlator", Oct 1, 1982
  • J.M. McCool, B. Widrow, J.R. Zeidler, R.H. Hearn, D.M. Cha bries, and R.H. Moore. "United States Patent 4,243,935 Adap- tive Detector", Jan 1, 1981
  • J.M. McCool, B. Widrow, J.R. Zeidler, R.H. Hearn, and D.M. Chabries. "United States Patent 4,238,746 daptive Line Enhancer", Dec 1, 1980
  • B. Widrow, M.E. Hoff, Jr.. "United States Patent 3,454,753 Analog Multiplier and Modulati ng Circuits Employing Electrolytic Elements", Jul 1, 1969
  • B. Widrow, G. Frick, R.H. Gordon. "United States Patent 3,395,402 Adaptive Memory Element", Jul 1, 1968
  • B. Widrow and M.E. Hoff, Jr. "United Statesogic Circuit and Electrolyt ic Memory Element", Dec 1, 1965

Current Research and Scholarly Interests


Prof. Widrow's research focuses on adaptive signal processing, adaptive control systems, adaptive neural networks, human memory, and human-like memory for computers. Applications include signal processing, prediction, noise cancelling, adaptive arrays, control systems, and pattern recognition.

Projects


  • Hearing Aid Device, Stanford University

    A directional acoustic receiving system is constructed in the form of a necklace, including an array of two or more microphones mounted on a housing supported on the chest of the user by a conducting loop encircling the user's neck. This method enables the design of highly-directive-hearing instruments which are comfortable, inconspicuous, and convenient to use. The array provides the user with a dramatic improvement in speech perception over existing hearing aid designs, particularly in the presence of background noise, reverberation, and feedback.

    B. Widrow, ``A Microphone Array for Hearing Aids,'' IEEE Circuits and Systems Magazine, 1(2):26-32, 2001.

    Location

    Stanford, California

  • Quantization Noise, Stanford University

    Prof. Widrow's most recent book, Quantization Noise, co-authored with Istvan Kollar, is available for purchase at the Cambridge University Press website

    Location

    Stanford, California

Journal Articles


  • The No-Prop algorithm: a new learning algorithm for multilayer neural networks. Neural networks Widrow, B., Greenblatt, A., Kim, Y., Park, D. 2013; 37: 182-188

    Abstract

    A new learning algorithm for multilayer neural networks that we have named No-Propagation (No-Prop) is hereby introduced. With this algorithm, the weights of the hidden-layer neurons are set and fixed with random values. Only the weights of the output-layer neurons are trained, using steepest descent to minimize mean square error, with the LMS algorithm of Widrow and Hoff. The purpose of introducing nonlinearity with the hidden layers is examined from the point of view of Least Mean Square Error Capacity (LMS Capacity), which is defined as the maximum number of distinct patterns that can be trained into the network with zero error. This is shown to be equal to the number of weights of each of the output-layer neurons. The No-Prop algorithm and the Back-Prop algorithm are compared. Our experience with No-Prop is limited, but from the several examples presented here, it seems that the performance regarding training and generalization of both algorithms is essentially the same when the number of training patterns is less than or equal to LMS Capacity. When the number of training patterns exceeds Capacity, Back-Prop is generally the better performer. But equivalent performance can be obtained with No-Prop by increasing the network Capacity by increasing the number of neurons in the hidden layer that drives the output layer. The No-Prop algorithm is much simpler and easier to implement than Back-Prop. Also, it converges much faster. It is too early to definitively say where to use one or the other of these algorithms. This is still a work in progress.

    View details for DOI 10.1016/j.neunet.2012.09.020

    View details for PubMedID 23140797

  • A unified framework for 3D radiation therapy and IMRT planning: plan optimization in the beamlet domain by constraining or regularizing the fluence map variations PHYSICS IN MEDICINE AND BIOLOGY Meng, B., Zhu, L., Widrow, B., Boyd, S., Xing, L. 2010; 55 (22): N521-N531

    Abstract

    The purpose of this work is to demonstrate that physical constraints on fluence gradients in 3D radiation therapy (RT) planning can be incorporated into beamlet optimization explicitly by direct constraint on the spatial variation of the fluence maps or implicitly by using total-variation regularization (TVR). The former method forces the fluence to vary in accordance with the known form of a wedged field and latter encourages the fluence to take the known form of the wedged field by requiring the derivatives of the fluence maps to be piece-wise constant. The performances of the proposed methods are evaluated by using a brain cancer case and a head and neck case. It is found that both approaches are capable of providing clinically sensible 3D RT solutions with monotonically varying fluence maps. For currently available 3D RT delivery schemes based on the use of customized physical or dynamic wedges, constrained optimization seems to be more useful because the optimized fields are directly deliverable. Working in the beamlet domain provides a natural way to model the spatial variation of the beam fluence. The proposed methods take advantage of the fact that 3D RT is a special form of intensity-modulated radiation therapy (IMRT) and finds the optimal plan by searching for fields with a certain type of spatial variation. The approach provides a unified framework for 3D CRT and IMRT plan optimization.

    View details for DOI 10.1088/0031-9155/55/22/N01

    View details for Web of Science ID 000283789700001

    View details for PubMedID 21030744

  • Adaptive Cancellation of Floor Vibrations in Standing Ballistocardiogram Measurements Using a Seismic Sensor as a Noise Reference IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Inan, O. T., Etemadi, M., Widrow, B., Kovacs, G. T. 2010; 57 (3): 722-727

    Abstract

    An adaptive noise canceller was used to reduce the effect of floor vibrations on ballistocardiogram (BCG) measurements from a modified electronic bathroom scale. A seismic sensor was placed next to the scale on the floor and used as the noise reference input to the noise canceller. BCG recordings were acquired from a healthy subject while another person stomped around the scale, thus causing increased floor vibrations. The noise canceller substantially eliminated the artifacts in the BCG signal due to these vibrations without distorting the morphology of the measured BCG. Additionally, recordings were obtained from another subject standing inside a parked bus while the engine was running. The artifacts due to the vibrations of the engine, and the other vehicles moving on the road next to the bus, were also effectively eliminated by the noise canceller. The system with automatic floor vibration cancellation could be used to increase BCG measurement robustness in home monitoring applications. Additionally, the noise cancellation approach may enable BCG recording in ambulances-or other transport vehicles-where noninvasive hemodynamic monitoring may otherwise not be feasible.

    View details for DOI 10.1109/TBME.2009.2018831

    View details for Web of Science ID 000274990800025

    View details for PubMedID 19362900

  • Adaptive Cancellation of Floor Vibrations in Standing Ballistocardiogram Measurements Using a Seismic Sensor as a Noise Reference IEEE Transactions on Biomedical Engineering Widrow, B., Inan, O., Etemadi, M., Kovacs, G. 2010; 57 (3): 722-727
  • Predicting respiratory tumor motion with multi-dimensional adaptive filters and support vector regression PHYSICS IN MEDICINE AND BIOLOGY Riaz, N., Shanker, P., Wiersma, R., Gudmundsson, O., Mao, W., Widrow, B., Xing, L. 2009; 54 (19): 5735-5748

    Abstract

    Intra-fraction tumor tracking methods can improve radiation delivery during radiotherapy sessions. Image acquisition for tumor tracking and subsequent adjustment of the treatment beam with gating or beam tracking introduces time latency and necessitates predicting the future position of the tumor. This study evaluates the use of multi-dimensional linear adaptive filters and support vector regression to predict the motion of lung tumors tracked at 30 Hz. We expand on the prior work of other groups who have looked at adaptive filters by using a general framework of a multiple-input single-output (MISO) adaptive system that uses multiple correlated signals to predict the motion of a tumor. We compare the performance of these two novel methods to conventional methods like linear regression and single-input, single-output adaptive filters. At 400 ms latency the average root-mean-square-errors (RMSEs) for the 14 treatment sessions studied using no prediction, linear regression, single-output adaptive filter, MISO and support vector regression are 2.58, 1.60, 1.58, 1.71 and 1.26 mm, respectively. At 1 s, the RMSEs are 4.40, 2.61, 3.34, 2.66 and 1.93 mm, respectively. We find that support vector regression most accurately predicts the future tumor position of the methods studied and can provide a RMSE of less than 2 mm at 1 s latency. Also, a multi-dimensional adaptive filter framework provides improved performance over single-dimension adaptive filters. Work is underway to combine these two frameworks to improve performance.

    View details for DOI 10.1088/0031-9155/54/19/005

    View details for Web of Science ID 000270051600006

    View details for PubMedID 19729711

  • Statistical efficiency of adaptive algorithms NEURAL NETWORKS Widrow, B., Kamenetsky, M. 2003; 16 (5-6): 735-744

    Abstract

    The statistical efficiency of a learning algorithm applied to the adaptation of a given set of variable weights is defined as the ratio of the quality of the converged solution to the amount of data used in training the weights. Statistical efficiency is computed by averaging over an ensemble of learning experiences. A high quality solution is very close to optimal, while a low quality solution corresponds to noisy weights and less than optimal performance. In this work, two gradient descent adaptive algorithms are compared, the LMS algorithm and the LMS/Newton algorithm. LMS is simple and practical, and is used in many applications worldwide. LMS/Newton is based on Newton's method and the LMS algorithm. LMS/Newton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training data. Many least squares adaptive algorithms have been devised over the years, but no other least squares algorithm can give better performance, on average, than LMS/Newton. LMS is easily implemented, but LMS/Newton, although of great mathematical interest, cannot be implemented in most practical applications. Because of its optimality, LMS/Newton serves as a benchmark for all least squares adaptive algorithms. The performances of LMS and LMS/Newton are compared, and it is found that under many circumstances, both algorithms provide equal performance. For example, when both algorithms are tested with statistically nonstationary input signals, their average performances are equal. When adapting with stationary input signals and with random initial conditions, their respective learning times are on average equal. However, under worst-case initial conditions, the learning time of LMS can be much greater than that of LMS/Newton, and this is the principal disadvantage of the LMS algorithm. But the strong points of LMS are ease of implementation and optimal performance under important practical conditions. For these reasons, the LMS algorithm has enjoyed very widespread application. It is used in almost every modem for channel equalization and echo cancelling. Furthermore, it is related to the famous backpropagation algorithm used for training neural networks.

    View details for DOI 10.1016/S0893-6080(03)00126-6

    View details for Web of Science ID 000184011900028

    View details for PubMedID 12850029

  • Statistical Efficiency of Adaptive Algorithms Neural Networks Widrow, B., Kamenetsky, M. 2003: 735-744
  • Neurointerfaces IEEE Transactions on Control Systems Technology Widrow, B. 2002: 221-228
  • Neural dynamic optimization for control systems - Part II: Theory IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 490-501

    Abstract

    The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the theory of NDO, while the two other companion papers of this topic explain the background for the development of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.

    View details for Web of Science ID 000170320400002

    View details for PubMedID 18244816

  • Neural dynamic optimization for control systems - Part III: Applications IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 502-513

    Abstract

    For pt.II. see ibid., p. 490-501. The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper demonstrates NDO with several applications including control of autonomous vehicles and of a robot-arm, while the two other companion papers of this topic describes the background for the development of NDO and present the theory of the method, respectively.

    View details for Web of Science ID 000170320400003

    View details for PubMedID 18244817

  • Neural dynamic optimization for control systems - Part I: Background IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 482-489

    Abstract

    The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the background and motivations for the development of NDO, while the two other subsequent papers of this topic present the theory of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.

    View details for Web of Science ID 000170320400001

    View details for PubMedID 18244815

  • A Microphone Array for Hearing Aids IEEE Circuits and Systems Magazine Widrow, B. 2001: 26-32
  • Adaptive inverse control based on linear and nonlinear adaptive filtering INTERNATIONAL WORKSHOP ON NEURAL NETWORKS FOR IDENTIFICATION, CONTROL, ROBOTICS, AND SIGNAL/IMAGE PROCESSING - PROCEEDINGS Widrow, B., Plett, G. L. 1996: 30-38
  • Sensitivity of feedforward neural networks to weight errors. IEEE transactions on neural networks Stevenson, M., Winter, R., Widrow, B. 1990; 1 (1): 71-80

    Abstract

    An analysis is made of the sensitivity of feedforward layered networks of Adaline elements (threshold logic units) to weight errors. An approximation is derived which expresses the probability of error for an output neuron of a large network (a network with many neurons per layer) as a function of the percentage change in the weights. As would be expected, the probability of error increases with the number of layers in the network and with the percentage change in the weights. The probability of error is essentially independent of the number of weights per neuron and of the number of neurons per layer, as long as these numbers are large (on the order of 100 or more).

    View details for PubMedID 18282824

  • Fundamental Relations Between the LMS Algorithm and the DFT IEEE Transactions on Circuits and Systems Widrow, B., Baudrenghien, P., Vetterli, M., Titchener, P. 1987: 814-820
  • On the Statistical Efficiency of the LMS Algorithm with Nonstationary Inputs IEEE Transactions on Information Theory Widrow, B., Walach E. 1984: 211-221
  • Statistical Analysis of Amplitude-Quantized Sampled-Data Systems AIEE Transactions on Applications and Industry Widrow, B. 1961: 1-14
  • A Study of Rough Amplitude Quantization by Means of Nyquist Sampling Theory IRE Transactions on Circuit Theory Widrow, B. 1956; CT-3(4): 266-276

Books and Book Chapters


  • Quantization Noise: Round Off Error in Digital Computation, Signal Processing, Control and Communications Widrow, B., Kollar, I. Cambridge University Press. 2008
  • Least-Mean-Square Adaptive Filters Widrow, B., Haykin, S. Wiley-Interscience. 2003
  • Nonlinear Control with Neural Networks Backpropagation: Theory, Architectures, and Applications Chauvin, Y., D. Rumelhart, D. Erlbaum Associates. 1995
  • Noise Canceling and Channel Equalization Handbook of Brain Theory and Neural Networks Widrow, B., Lehr, M. MIT Press. 1995
  • 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Neural Networks: Theoretical Foundations and Analysis Lau, C. IEEE Press. 1992
  • 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Artificial Neural Networks: Paradigms, Applications, and Hardware Implemenation Sanchez-Sinencio, E., Lau, C. IEEE Press. 1992: 82-108
  • Adaptive Signal Processing Widrow, B., Stearns, S. Prentice Hall. 1985
  • Adaptive Filters Aspects of Network and System Theory Kalman, R., DeClaris, N. Holt, Rinehart and Winston. 1971

Conference Proceedings


  • 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Widrow, B., Lehr, M. 1990: 1415-1442
  • Adaptive Antenna Systems [a citation classic] Widrow, B., Mantey, P. 1967: 2143-2159
  • Adaptive Switching Circuits RE WESCON Convention Record Widrow, B. 1960: 96-104