Automated detection of infant crying and fussing for clinical applications

Excessive crying and fussing of unknown cause in infants is common and of relevant parental concern. Conventionally, the diagnosis of its related medical conditions is made based on the subjective parental reports.

To reliably and objectively identify and distinguish crying from fussing and to quantify both, machine-based learning algorithms were developed using the LENA (Language ENvironment Analysis) system. The overall accuracy of the machine based learning approach is more than 90%.


Crying: a challenge

Excessive crying and fussing in the early months is a prevalent symptom and is often extremely distressing for the infant and parents. The careful assessment of healthcare professional is necessary to understand and adequately manage the problem and offer support to exhausted parents.

Parents are often asked to judge and report the baby’s symptoms, such as crying, but this is prone to inaccuracies as parental perceptions can vary. Until recently, no method for the automated detection and quantification of crying and fussing existed to provide an objective metric for diagnostic and clinical study purposes.

Collaboration: addressing the challenge

In collaboration with LENA Foundation, Danone Nutricia Research utilised the LENA (Language ENvironment Analysis) system to automatically identify, quantify, and distinguish periods of crying versus fussing in a pilot sample of infants.

The LENA system comprises a small digital audio recording device fitted in specialised clothing worn by the infant plus processing software. The device enables the continuous capture of up to 24 hours of each infant’s natural sound environment, and the machine-learning-based software automatically processes the recorded audio data to generate metadata on distinct sound categories, including infant cries.

As a first step, a new algorithm was generated by machine learning, and then validated in a human-rated sample of 124 cry periods from 41 infants. An observational pilot study of 12 term infants is currently ongoing where parents record crying and fussing with diaries in parallel to LENA recordings in order to enable a direct comparison of both.

Presented at 2018 ESPGHAN Congress

Model performance comparing simultaneously generated parental report diaries, is presented at 2018 ESPGHAN (European Society for Paediatric Gastroenterology Hepatology and Nutrition) annual congress. Clinical applications of the technology are also discussed in the congress, which is the largest conference of its kind worldwide.

The new algorithm distinguished periods of crying from fussing with 90% sensitivity, 92% specificity, and 91% overall accuracy in the validation sample.

Relevant differences between parental diary reports and automatically detected absolute crying and fussing duration, time resolution, and distribution have been observed in the ongoing pilot study. Results from this ongoing research were also reported to validate the utility of the automated method in the current and other potential clinical applications, such as identifying and characterizing infantile colic.

“We are proud that machine-based learning algorithms were developed to reliably and objectively identify and distinguish crying from fussing and to quantify both in recordings obtained using the LENA system. As compared to parental diaries, this innovation is easy to use, provide objective data, and with high temporal resolution,”

said Thomas Ludwig, Principal Scientist Paediatric Gastroenterology at Danone Nutricia Research.

“This enhancement to LENA’s capabilities has so far surprised even us in its performance. We’re thrilled to collaborate with the Danone Nutricia Research team to uncover the ultimate potential for this expanded capability in clinical applications and beyond,”

said Stephen Hannon, President of LENA.