Master Thesis »deep Learning-based Audio Analysis - Ilmenau, Deutschland - Digital Media Technology

Beschreibung

The Fraunhofer Institute for Digital Media Technology IDMT is part of the Fraunhofer-Gesellschaft.

Headquartered in Ilmenau, Germany, the institute is internationally recognized for its expertise in applied electroacoustics and audio engineering, AI-based signal analysis and machine learning, and data privacy and security.

At the headquarters, on the campus of "Technische Universität Ilmenau" researchers work on technologies for robust, trustworthy AI-based analysis and classification of audio and video data.

These are used, among other things, to monitor industrial production processes, but also in traffic monitoring or in the media context, for example when it comes to automatic metadata extraction and audio manipulation detection.

Another focus is the development of algorithms for the areas of virtual product development, intelligent actuator-sensor systems and audio for the automotive sector.

There are currently around 70 employees working at Fraunhofer IDMT in Ilmenau.

In the group Semantic Music Technologies at the Fraunhofer IDMT, one of the main research focuses is on extracting meaningful information, identifying patterns, and making sense of complex acoustic recording.

For this purpose, methods from audio signal processing and machine learning are often combined.


What you will do
This thesis aims to explore and develop audio analysis methods specifically designed for counting bird populations in outdoor environments.

The study will primarily focus on developing deep learning-based audio analysis methods to count distinct bird calls in complex and noisy outdoor soundscapes.

These calls may originate from individuals of the same species or from individuals of different species.

The ability to detect and count bird calls within a short audio segment enables the derivation of long-term statistics about population density at specific acoustic sensor locations.

Existing bioacoustics research has mainly focused on the task of bird species classification.

The most popular model, BirdNet [1], uses a convolutional neural network (CNN) architecture to classify up to 3,000 bird species.

Some research exists on animal counting, which employs either computer vision-based methods [2, 3] or audio-based algorithms [4,5]. In previous research at Fraunhofer IDMT, various approaches for polyphony estimation were developed to characterize audio signals.

The term "polyphony" can refer to the number of simultaneous pitches [6], the ensemble size in a music recording [7], or the number of audible sound sources in a short environmental audio recording [8].

Objectives
(1) As part of the first objective, the student should investigate whether there exist suitable bird song datasets with polyphony annotations.

Alternatively, the student should examine the following approach to dataset creation:

After defining a taxonomy of approximately 10-20 common bird species (with assistance from domain experts from an ongoing EU research project), corresponding audio recordings should be collected from the Xeno-Canto platform [9], which houses numerous audio recordings of individual bird calls.

These recordings can then be randomly mixed to generate a larger dataset of mixtures, covering different degrees of polyphony and allowing for model training and evaluation.

(2) For the second objective, two deep learning-based approaches should be identified from the literature and re-implemented. Utilizing the previously compiled dataset, models should be trained and evaluated for the task of bird counting.

The student should compare two strategies:

an explicit counting approach, where the number of birds is predicted directly, and implicit counting approach, where the number of unique species is counted that are classified before by the BirdNet model.

References

What you bring to the table

The prerequisites for this master's thesis topic are excellent skills in audio signal processing and deep learning, practical experience using Python and deep learning libraries such as TensorFlow or PyTorch, as well as a general interest in bioacoustic research topics.

What you can expect
- exciting market-related topics with complex issues to be solved - you can be actively involved in shaping the future
- challenges at a high level - on top we offer you excellent opportunities for professional and technical trainings
- space to also implement your own ideas, such as in our quarterly open-topic idea contest
- an excellent technical infrastructure
- renowned partners and customers who work closely with you to develop the technologies of tomorrow
- a very good work-life balance thanks to flexible working hours, a co-child office, the option of digital childcare in case of daycare shortages, and the possibility of mobile working, because family comes first - we know that
- an open-minded and interested team, a tolerant and familiar atmosphere as well as regular team events
- good transport connections and proximity to th