Adding Intelligence to Sensors and Sensor Networks
The field of wireless vital sign monitoring has a relatively long history, almost as long as radar itself. Doppler radar has been the primary sensor technology for detecting blood flow or fetal heart rate. Over the past decade, there has been an increasing interest and need for creating wireless long-range vital sign monitoring systems. In addition, there has been a strong preference for using non-contact technologies for detection applications, especially for search-and-rescue operations that require the ability to see through debris, walls, and snow. There are also several emerging applications using short-range radar systems for automotive safety systems, remote health monitoring, and heart-based biometric authentication.
The HeartSignature™ sensor scans the individual continuously for real-time authentication, serving a number of high-security access applications.
Named after the Austrian physicist Christian Doppler, the Doppler Effect is a fundamental frequency shift phenomenon that occurs whenever a wave source and an observer are moving with respect to one another. When a vehicle sounding a siren or horn approaches, passes, and recedes, for example, the by-standing observer will hear the sound higher during the approach, identical at the instant of passing by, and lower during the recession. The frequency increase has well established applications in astrophotonics, biological diagnostics, weather and aircraft radar systems, velocimetry, and vibrometry. For instance, ultrasonic pulse probes utilize this Doppler Effect to detect the relative motion of blood flow in the human body.
Microwave Doppler radar has the capability to detect vital signs, such as heart and breathing function. Doppler radar achieves the findings by sensing mechanical displacements of the chest cavity in the order of millimeters, resulting from shock waves created by heart and respiration motion. This is known as Radar Seismocardiogram (R-SCG). Using the technology, other cardiac dynamic parameters/features that are unique to each person can be extracted.
In order to enable commercial applications for R-SCG devices, several key problems need to be overcome, including device cost and motion artifacts that distort the signal of interest. The cost and performance have dramatically improved with the advent of RF integrated circuits, contributing to the commercialization of small low-power radar units for many different purposes. It has been shown that high transmit power is not necessary to achieve good results with Doppler radar and some other detection methods, such as Ultra-Wide Band sensing that requires only -41dBm of output power and uses approximately 3 million times lower power than a typical smartphone.
More recently, the increasing computing power of embedded computers and small microcontrollers now enable the implementation of significant computational algorithms to analyze, filter, and clean the data from such small radar systems. With the advent of advanced pattern recognition and machine learning technologies, the screening of unusual heart activity, as well as the monitoring of physiological activity in certain stressful situations, is now feasible.
Despite the advances, however, significant problems remain, including how to find better ways to reduce background motion noise in such systems while maintaining very low cost. To be practical and useful, the devices must acquire, interpret, and provide information in real-world operating environments at competitive costs. There is a tremendous need to demonstrate the efficacy of this system in real-world, naturalistic operating environments.
Enter NASA’s FINDER Technology
Figure 1. The Finding Individuals for Disaster and Emergency Response prototype technology was demonstrated in Lorton, Virginia, on May 7, 2015. (Image Credit: NASA/JPL-Caltech/DHS)
The Finding Individuals for Disaster and Emergency Response (FINDER) prototype technology developed at NASA’s Jet Propulsion Laboratory has demonstrated the ability to locate individuals buried under rubble in disaster scenarios. FINDER uses radar technology to sense the heartbeats and breathing of humans hidden behind piles of debris. With the NASA FINDER, four men were rescued in April of 2015 from the wreckage of a collapsed textile factory in the Nepalese village of Chautara. The technology detected the men’s presence even as they were buried under about 10 feet of brick, mud, and wood (see Figure 1).
The FINDER technology was developed in collaboration between NASA's Jet Propulsion Laboratory and the Department of Homeland Security’s Science and Technology Directorate.
The radar tool beams microwave signals into piles of debris, and the patterns of the reflected signals are analyzed for changes. The FINDER device senses the tiny motions caused by victims’ breathing and heartbeats, and displays the vital signs along with a reliability score.
The Doppler sensor technology includes a local oscillator which is split into two signals (I and Q) that are offset by 90 degrees -- each going to two different detectors along with the receive signal. I/Q data shows the changes in magnitude (or amplitude) and phase of the signal. If amplitude and phase changes occur in an orderly, predetermined fashion, they can be used to encode information upon the signal, a process known as modulation. I/Q data is highly prevalent in RF communications systems, and more generally in signal modulation, because it is a convenient way to modulate signals.
Doppler signal processing separates reflected signals into a number of frequency filters, with a separate set of filters for each ambiguous range. The I and Q samples begin the filtering process, which is an adaptive method that adjusts automatically to background noise and environmental influences.
Adaptive filtering and additional post-signal processing enable the system to determine patterns that identify the presence of a stationary human. Using the technology, other cardiac dynamic parameters and features unique to humans can be extracted. Such detection is normally impossible to do with traditional Doppler radar-based sensors, which require large motion of objects for the sensor to function. The advanced recognition technology prevents a wider range of false positives caused by the RF clutter from movement of background objects in the environment. Human Presence Detection is enabled as an important and new sensing system that combines microwave radar along with advanced detection algorithms.
The Role of Doppler Radar in Biometrics Today
Microwave Doppler radar has shown significant promise in vital sign detection. Research efforts in the past decades have been focused on overcoming challenges of removing unwanted artifacts in the signal received due to its various environments detected. New signal processing techniques and novel receiver architectures have been deployed to minimize these effects. Additionally, the ability to biometrically identify and/or monitor human subjects remotely using physiological data, without intervention, is growing in importance for a wide variety of applications.
Figure 2. U.S. Army Pfc. Andrew Bock uses a handheld identification detection device to gather information on an Afghan during Operation Helmand Spider in Afghanistan, Feb. 15, 2010. The device is part of a multimodal biometric system that collects and compares fingerprints and iris and facial photos against an internally downloaded biometric watch list. (Image Credit: U.S. Air Force, Tech. Sgt. Efren Lopez)
Current biometric modalities, such as fingerprint, retina scans, and facial recognition, limit the effectiveness of remote or unattended applications. In Afghanistan, retina scans are commonly used by armed forces to track possible insurgents. Images collected, however, are frequently of low quality, containing random specular reflections in and around the pupil and iris that impact the performance of the enrollment process, due to difficulties in aligning the sensor to the human subject’s eye in the field (see Figure 2).1 To complicate matters, it is impossible to use facial recognition to identify individuals who wear burkas. The biometric sensors also do not have the ability to provide real-time subject health status.
The human heart bears a signature that is unique to the individual. Such a “heart signature” can be captured using a variety of techniques, such as an electrocardiogram (ECG) record of electrical stimuli, echocardiogram, or ultrasound-based motion detection. Laser Doppler vibrometry and Doppler radar are able to wirelessly sense blood vessel and heart motion, respectively. Wireless cardiac biometric identification is advantageous compared to other biometric methods in that physically invasive measures or active cooperation by the subject are not required.
A biometric system for authentication is essentially a pattern recognition problem, and thus the methodology can be broken down into four parts: (i) signal acquisition, (ii) pre processing, (iii) feature extraction, and (iv) classification.
All biometric modalities have advantages and disadvantages. Face recognition, for example, is also a non-contact and easy-to-use biometric modality. It is difficult, however, to present the face consistently and, as a result, accuracy suffers. While face-recognition cameras are inexpensive, they are also sensitive to ambient lighting. Facial recognition is generally considered acceptable in our society because photographs are common. In cultures where faces are covered, however, facial recognition becomes untenable. Additionally, many systems can be defeated with images of faces. Commonly used biometric modalities are not universal, as pertinent operational requirements must determine the particular application.
Figure 3. The HeartSignature™ device can be placed in a user's pocket. (Image Credit: Olea Sensor Networks™)
HeartSignature™, a biometric modality developed by Olea Sensor Networks™, has the potential to offer the same distinctiveness, repeatability, and accuracy as other systems. With the device, the user does not actively engage with the sensor. The sensor can be deployed as a wearable (such as a pendant), embedded in car seats to monitor drivers and passengers, placed in a shirt pocket, or eventually be part of one’s active wear (see Figure 3).
The non-contact HeartSignature™ biometric sensor is insensitive to environmental conditions and able to operate in darkness. The technology scans the individual continuously for real-time authentication, serving a number of high-security access applications.
Adding Intelligence to Sensors and Sensor Networks
Advancements in artificial intelligence and machine learning algorithms enable the system to extract and analyze relevant identity features, including biometric characteristics of cardio signals. The same set of algorithms allows the system to classify and identify stable characteristics within heart and respiration data signatures. A specifically designed neural network can then match the signatures needed for comparison analysis of the data sets. The cardio signal workbench contains independent bands to filter for peak analysis, cycle isolation, and normalization. The software reveals stable characteristics unique to the individual.
Facial recognition has had to overcome the challenge of the human face being dynamic. As the camera focuses and tries to acquire an image for identification purposes, it captures many variables, such as a subject’s smile, an angle of the head, or poor lighting. In order to remove dynamic characteristics of the image, algorithms have been developed to only focus on invariant features of the face, including the eyes, ears, and nose. Olea Sensor Networks has developed a similar technique to remove the dynamic characteristics of the heart signal and only extract the stable, invariant features. The sensor ignores changes in the heart signal patterns related to stress, activity, age, or rest.
While these sensors offer access to data about us and the physical world, forecasts of more than 26 billion sensor connections by 2020 confirm that sensory data is growing so fast that our ability to understand it is highly constrained.(*2)
An “intelligent partitioning” architecture locally generates, fuses, and analyzes multi-sensor data. The partitioning aggregates and further analyzes multiple sensor data streams in the cloud, streamlining the data processing flow. Performing primary sensor data analytics at the local sensor network level can reduce data traffic by filtering, combining, and analyzing unstructured sensor data prior to streaming only the more meaningful, structured information to the cloud for secondary analysis and ultimate action. The arrangement leverages the power of multi-sensor networks for real-time applications by using far fewer cloud processing resources.
An intelligence-based sensor network collects and processes actionable information from a cluster of sensors, enabling information-sharing on a large scale and collaboration of applications among cloud users. Several networks are integrated, allowing applications to be crossdisciplinary and span over multiple organizations.
Future sensor network platforms will be possible in many industries, including security for identity and access management, as well as connected technologies for cars, healthcare, and fitness. Unique platforms will be created for industrial, home, or mobile settings. The integrated technologies have the potential to solve two significant Internet of Things (IoT) challenges: data overload/analytics and identity management. Through the use of advanced sensor analytics, signal processing, and micro-Doppler technology, the new detection approach provides a way to create sensors that will revolutionize the way we monitor ourselves and the world around us, and offer new applications in man-to-machine interface.
1 Source: Carnegie Mellon University, “Iris and Face Recognition,” https://www.cylab.cmu.edu/partners/success-stories/iris-recognition.html
2 Source: Stamford, Conn., December 12, 2013, “Gartner Says the Internet of Things Installed Base Will Grow to 26 Billion Units By 2020,” http://www.gartner.com/newsroom/id/2636073