Title:Families In Wild Multimedia (FIW-MM): A Multi-Modal Database for Recognizing Kinship

Abstract:Kinship is a soft biometric that researchers found detectable in media, and although it is difficult, it comes with an abundance of practical applications. There is continual interest to solve the problem shown by the consistent performance improvements kinship recognition problems based on the large-scale still-image Families In the Wild (FIW) database-- systems are at levels unforeseeable a decade ago: approaching ever closer to being acceptable for real-world use. Biometric tasks have shown to benefit from multi-modal data, as knowledge from the additional modalities complements that of still images. To narrow the gap between research-and-reality, and to enhance the power of kinship recognition systems, we extend FIW with multimedia data (i.e., video, audio, and contextual transcripts). Specifically, we introduce the first large-scale dataset for recognizing kinship in multimedia, the FIW-MM database. We leverage automated machinery to collect, annotate, and prepare the data with minimal human input and no financial cost. This large-scale, multimedia corpus allows problem formulations to follow more realistic template-based protocols. We show significant improvements in benchmarks for multiple kin-based tasks by using the added modalities. We provide insights by highlighting edge cases to inspire future research and areas of improvement. In addition, the richness, depth, and multi-modal data of FIW-MM can support a large number of multimedia tasks (e.g., recognition, generative modeling, speech understanding, and genetics/nature-based studies). FIW-MM provides the data required to increase the potential of systems built to automatically detect kinship in multimedia. Furthermore, it allows experts from diverse fields to collaborate in ways not possible prior.