Dr Gustaf Hendeby’s research is centered around statistical sensor fusion - primarily model based methods - with one foot in theory and one foot in more applied research. Sensor fusion provides methods that are used to combine information from many sources in order to produce a coherent view of phenomena observed in the world not obtainable from the separate sources on their own. With recent advances in sensor technology, resulting in smaller and more affordable sensors, a multitude of sensors are available almost everywhere; and much in life that we have learned to take for granted would not be possible without advanced sensor fusion. This knowledge drives his interest in the field.
Below he present his research divided into three main categories: core algorithm understanding, on-body sensor networks, and security and surveillance applications.
From the beginning, Dr Hendeby have been interested in the inner working of the algorithms that make up sensor fusion. By studying the Cramér-Rao lower bound (CRLB), he derived a method using intrinsic accuracy to indicate the potential performance gain from using nonlinear estimation and detection methods. Intrinsic accuracy is a noise property, which is closely related to Fisher information, that can be interpreted as measure of how informative a noise distribution is. In practice it acts like an effective variance term. To analyze the second order statistical properties of algorithms this way is very common, but does also in many situations not capture all aspects of an estimate that are important for the end result. By utilizing the above mentioned results, it is possible to provide insights into how to approach problems, and to beforehand answer the important question: “Is it reasonable to expect the results I need to solve this specific task?” This without having to resort to extensive Monte Carlo simulations.
In the same way, understanding how popular methods relate to each other makes it possible to make better design decisions early in the design process. Therefore, Dr Hendeby has analyzed the popular unscented Kalman filter (UKF) and managed to find clear connections to the classic extended Kalman filter (EKF), which in part explains its behavior. He currently pursue similar questions with Lic Eng Michael Roth. He has also analyzed the Rao-Blackewellized particle filter (RBPF), and was able to make connections between it and the important class of methods that are based on filter banks. Interpreting the RBPF this slightly different way can for instance be utilized to make efficient generalizable implementations.
Dr Hendeby’s interest for the particle filters (PFs) also resulted in the first complete PF implementation on a graphics card (GPU), which is a first step to make the PF realizable on low-cost readily available parallel hardware. A current fucus is to investigate the ensemble Kalman filter (EnKF), another approximate stochastic method that is heavily used in the geo-sciences, but so far not by the signal processing society. It could have the potential to solve extremely high-dimensional problems that would otherwise not be solvable at all.
At DFKI Dr Hendeby’s work was centered around on-body sensor networks consisting of inertial measurement units (IMUs) and in some cases other sensors such as cameras. Here he had the opportunity to work with and contribute to the whole signal processing chain from low-level sensor interaction to high-level fusion, and successfully derived methods to estimate the pose of the wearer of the sensors. In doing so we faced many practical problems, such as how to handle sensor synchronization, calibration of a heterogeneous sensor network, etc.
The sensor network was used in two European projects for: monitoring of physical exercise to ensure proper and safe execution (EU AAL project PAMAP (Physical Activity Monitoring for Aging People); and as input to workflow recognition and monitoring to aid in assembly tasks (EU FP7 COGNITO (Cognitive Workflow Capturing and Rendering with On-Body Sensor Networks). Dr Hendeby was technical coordinator of COGNITO comprising 7 partners from 4 countries with specialties such as biomechanics, computer vision, workflow recovery, computer graphics, human-machine interaction, and hardware manufacturing.
Stripped down wearable sensor network were also used in the PAMAP project to derive the wearer’s everyday daily activities. This is a problem that can be solved with good results using classifiers based on ensemble learners (Boosting).
At the Swedish Defence Research Agency (FOI), Dr Hendeby work with applied research (algorithm development, analysis, and implementation) in the area of security and surveillance. The focus is target tracking, where he overseea the tracking efforts in the group. He coordinates the design and development of an in-house tracking software. In this position he has spent considerable time with questions regarding decentralized and distributed fusion, and tracking systems in production code.
Recently, the focus of his research has been multi-sensor multi-target tracking to provide situational awareness, which is important aspects of both internal as well as external projects, e.g., the EU FP7 projects ADABTS (Automatic detection of abnormal behavior and threats in crowded spaces) and P5 (Privacy preserving perimeter protection project), he has been involved in. For this purpose he has developed a multi-hypothesis tracker (MHT), which is now used as an off-the-shelf toolbox to solve tracking tasks at FOI. In many cases detections in images are used as input, but the developed framework is not limited to this. Another application is tracking in sonar data, for which a Gaussian-Mixture Probability Hypothesis Distribution (PHD) filter solution was developed.
A topic of recent work, jointly with FOI and Linköping University, is passive tracking in acoustic networks and how the Doppler shifts in received signals can be used to track targets and also direction of arrival estimation. Another area of cooperation is SLAM.
Dr Hendeby is also in the Wild Life Security initiative and its Vinnova financed project “Smarta Savanner” intended to develop solutions to help protect and document endangered spices. In this project he coordinates and has technical lead for many of the different efforts to develop cheap, portable, and efficient methods and technical solutions to protect endangered wildlife. This allows him to work with both rather low-level sensor data as well as with higher level of abstraction in algorithm and demonstrator development.
Multi-Sensor Image-Based Navigation
Aerial navigation in GNSS contested environments is a challenge for both manned and unmanned aircraft. This research project aims to study and develop technologies and algorithms to address the problem of aerial navigation without relying on GNSS, focusing in image-based navigation with multi-sensor fusion in order to get a precise localization of the aircraft. The proposed application to be used in this research project is related to an aerial transport of goods in a remote area (challenging environment and low risk area), using real time image processing and another embedded sensor processing to improve the aircraft localization, without using GNSS data. This application has a dual use (both military and civil).
Funding from Vinnova for one PostDoc, Jeong Min Kang.
Classical methods for filtering are married with machine learning methodology to provide better tracking and navigation solutions. Focus is on addressing the time update bottle neck in filtering, due to poor signal- to-noise (SNR), which has appeared as a result of the high requirements of situational awareness posed by autonomous systems. The goal is to be able to provide high quality filtering solutions, for tracking and navigation, as a result of being able to learn more accurate motion models.
Funding from WASP for one PhD student, Anton Kullberg.
Applied Research Platform for Sensor Fusion (CENIIT Project 17.2)
The aim of this project is to create a research group that using probabilistic learning methods addresses the needs introduced by the surge of less expensive sensors, and in the process to complement the existing group with competence in computer science and more knowledge of hardware. The goal of the proposal is to perform agile development of theory for emerging applications with early demonstrators and active involvement of students.
Funding from ZENITH.
Distributed Learning of Augmented State-Space Models
Methods for distributed learning of augmented state-space models will be researched. This class of models enables domain knowledge to be incorporated in the learning process to guarantee a minimum of perfor- mance and enable an efficient learning process; still they are flexible enough to permit learning of partially unknown model dynamics and inputs. Thus, the targeted methods are foreseen to play an important role to realize large-scale sensing systems. Focus of the research will be on learning of state-space models augmented with sparse Gaussian process models. To enable distributed and efficient learning of the mod- els, techniques for distributed Gaussian process regression, such as Bayesian commit machines and inner product quantization, will be integrated with distributed filter algorithms, such as covariance intersection, consensus, and diffusion Kalman filters. The use of information-gain based active learning strategies to streamline the learning process, both on local and global scale, will be investigated.
Funding from WASP for one PhD student.
Lic. Kristin Nielsen (Epiroc). Robust LIDAR-Based Localization in Underground Mines, May 2021.
Lic. Robin Forsling (Saab). Decentralized Estimation Using Conservative Information Extraction, Jan 2021. (Co-supervisor)
Lic. Anton Kullberg. On Joint State Estimation and Model Learning using Gaussian Process Approximations, Dec 2021.
Daniel Bossér. (Co-supervisor)
Peng Liu. (Co-supervisor)
Gustav Zetterqvist (Co-supervisor)
Chuan Huang (Co-supervisor)
Jakob Åslund (Co-supervisor)
Lic. Jonas Nordlöv (FOI). On Landmark Densities in Minimum-Uncertainty Motion Planning, March 2022. (Co-supervisor)
Dr. Per Boström-Rost (SAAB Aeronotics). Sensor Management for Target Tracking Applications, May 2021.
Dr. Kamiar Radnosrati. On Time of Flight Estimation for Radio Network Positioning, March 2020. (Co-supervisor)
Lic. Andreas Bergström (Ericsson). Timing-Based Localization using Multipath Information, Feb 2020. (Co-supervisor)
Dr. Parinaz Kasebzadeh. Learning Human Gait, Sept 2019. (Co-supervisor)
Lic. Du Ho. Some results on closed-loop identification of quadcopters, Nov 2018. (Co-supervisor)
Lic. Martin Lindfors. Frequency Tracking for Speed Estimation, Aug 2018. (Co-supervisor)
Dr. Jonatan Olofsson. On Multi-UAS Sea Ice Monitoring, Jan 2019. (Co-supervisor, NTNU Norway)
Dr. Clas Veibäck. Tracking the Wanders of Nature, Dec 2018.
Dr. Michael Roth. Advanced Kalman Filtering Approaches to Bayesian State Estimation, April 2017. (Co-supervisor)
Lic. Hanna Nyqvist. On Pose Estimation in Room-Scaled Environments, Dec 2016.
Lic. George Mathai. Direction of Arrival Estimation of Wideband Acoustic Wavefields in a Passive Sensing Environment, Sept, 2015. (Co-supervisor)
Dr. Attila Reiss. Personalized Mobile Physical Activity Monitoring for Everyday Lif, Jan 2014. (Co-supervisor)
Lic. Marek Syldatk. On Calibration of Ground Sensor Networks, Sept 2013. (Co-supervisor)
For more information:
Performance and Implementation Aspects of Nonlinear Filtering
Nonlinear filtering is an important standard tool for information and sensor fusion applications, e.g., localization, navigation, and tracking. It is an essential component in surveillance systems and of increasing importance for standard consumer products, such as cellular phones with localization, car navigation systems, and augmented reality. This thesis addresses several issues related to nonlinear filtering, including performance analysis of filtering and detection, algorithm analysis, and various implementation details.
The most commonly used measure of filtering performance is the root mean square error (RMSE), which is bounded from below by the Cramér-Rao lower bound (CRLB). This thesis presents a methodology to determine the effect different noise distributions have on the CRLB. This leads up to an analysis of the intrinsic accuracy (IA), the informativeness of a noise distribution. For linear systems the resulting expressions are direct and can be used to determine whether a problem is feasible or not, and to indicate the efficacy of nonlinear methods such as the particle filter (PF). A similar analysis is used for change detection performance analysis, which once again shows the importance of IA.
A problem with the RMSE evaluation is that it captures only one aspect of the resulting estimate and the distribution of the estimates can differ substantially. To solve this problem, the Kullback divergence has been evaluated demonstrating the shortcomings of pure RMSE evaluation.
Two estimation algorithms have been analyzed in more detail; the Rao-Blackwellized particle filter (RBPF), by some authors referred to as the marginalized particle filter (MPF), and the unscented Kalman filter (UKF). The RBPF analysis leads to a new way of presenting the algorithm, thereby making it easier to implement. In addition the presentation can possibly give new intuition for the RBPF as being a stochastic Kalman filter bank. In the analysis of the UKF the focus is on the unscented transform (UT). The results include several simulation studies and a comparison with the Gauss approximation of the first and second order in the limit case.
This thesis presents an implementation of a parallelized PF and outlines an object-oriented framework for filtering. The PF has been implemented on a graphics processing unit (GPU), i.e., a graphics card. The GPU is a inexpensive parallel computational resource available with most modern computers and is rarely used to its full potential. Being able to implement the PF in parallel makes new applications, where speed and good performance are important, possible. The object-oriented filtering framework provides the flexibility and performance needed for large scale Monte Carlo simulations using modern software design methodology. It can also be used to help to efficiently turn a prototype into a finished product.
Fundamental Estimation and Detection Limits in Linear Non-Gaussian Systems
Many methods used for estimation and detection consider only the mean and variance of the involved noise instead of the full noise descriptions. One reason for this is that the mathematics is often considerably simplified this way. However, the implications of the simplifications are seldom studied, and this thesis shows that if no approximations are made performance is gained. Furthermore, the gain is quantified in terms of the useful information in the noise distributions involved. The useful information is given by the intrinsic accuracy, and a method to compute the intrinsic accuracy for a given distribution, using Monte Carlo methods, is outlined.
A lower bound for the covariance of the estimation error for any unbiased estimator is given by the Cramér-Rao lower bound (CRLB). At the same time, the Kalman filter is the best linear unbiased estimator (BLUE) for linear systems. It is in this thesis shown that the CRLB and the BLUE performance are given by the same expression, which is parameterized in the intrinsic accuracy of the noise. How the performance depends on the noise is then used to indicate when nonlinear filters, e.g., a particle filter, should be used instead of a Kalman filter. The CRLB results are shown, in simulations, to be a useful indication of when to use more powerful estimation methods. The simulations also show that other techniques should be used as a complement to the CRLB analysis to get conclusive performance results.
For fault detection, the statistics of the asymptotic generalized likelihood ratio (GLR) test provides an upper bound on the obtainable detection performance. The performance is in this thesis shown to depend on the intrinsic accuracy of the involved noise. The asymptotic GLR performance can then be calculated for a test using the actual noise and for a test using the approximative Gaussian noise. Based on the difference in performance, it is possible to draw conclusions about the quality of the Gaussian approximation. Simulations show that when the difference in performance is large, an exact noise representation improves the detection. Simulations also show that it is difficult to predict the exact influence on the detection performance caused by substituting the system noise with Gaussian noise approximations.