2018 | 14 | Audiovisual Data in Digital Humanities

This issue of VIEW provides a critical survey of new Digital Humanities methods and tools directed toward audiovisual media. While DH as a field seems dominated by a focus on textual studies, the mandate to improve the capacities to search, discover, and study AV is clear and unquestioned. New and emergent tools are reasonably expected to change this methodological landscape within the digitally accelerated near-future.
Co-edited by Andreas Fickers, Pelle Snickars and Mark J. Williams

Now showing 1 - 11 of 11

Collective Collecting: The Syrian Archive and the New Challenges of Historiography
Dang, Sarah-Mai; Strohmaier, Alena (2018-12-31)
Massive digitization makes histories appear as well as disappear. While digital archives facilitate the access to documents, recordings, films, and other s urces there is the risk that offlin sources get lost. Thus, the question about how digital collections are generated is essential for today’s film and media historians. Which artefacts are getting digitiz d – and which are not? In addition, for what reasons? Who is responsible for preserving historical material? Moreover, how can we access it? How can we make sense of the abundance of audio-visual sources, which are at the same time ephemeral? In this article, we analyse tools and methods useful for coping with digital archives and databases. Presenting a case study on the Syrian Archive, we discuss how concepts of authenticity and provenance relate to current media practices. We argue that besides posing productive research questions, conducting critical online search becomes more and more important in the humanities. Therefore, we examine not only what but also how the use of audio-visual material affects us. Furthermore, we argue that regarding the abundance of material the practice of curating – of selecting, structuring, and providing access – becomes a key activity in digital media practices.
Computer Vision and the Digital Humanities: Adapting Image Processing Algorithms and Ground Truth through Active Learning
Musik, Christoph; Zeppelzauer, Matthias (2018-12-31)
Automated computer vision methods and tools offer new ways of analysing audio-visual material in the realm of the Digital Humanities (DH). While there are some promising results where these tools can be applied, there are basic challenges, such as algorithmic bias and the lack of sufficient transparency, one needs to carefully use these tools in a productive and responsible way. When it comes to the socio-technical understanding of computer vision tools and methods, a major unit of sociological analysis, attentiveness, and access for configuration (for both computer vision scientists and DH scholars) is what computer science calls “ground truth”. What is specified in the ground truth is the template or rule to follow, e.g. what an object looks like. This article aims at providing scholars in the DH with knowledge about how automated tools for image analysis work and how they are constructed. Based on these insights, the paper introduces an approach called “active learning” that can help to configure these tools in ways that fit the specific requirements and research questions of the DH in a more adaptive and user-centered way. We argue that both objectives need to be addressed, as this is, by all means, necessary for a successful implementation of computer vision tools in the DH and related fields.
Describing Gender Equality in French Audiovisual Streams with a Deep Learning Approach
Doukhan, David; Poels, Géraldine; Rezgui, Zohra; Carrive, Jean (2018-12-31)
A large-scale description of men and women speaking-time in media is presented, based on the analysis of about 700.000 hours of French audiovisual documents, broadcasted from 2001 to 2018 on 22 TV channels and 21 radio stations. Speaking-time is described using Women Speaking Time Percentage (WSTP), which is estimated using automatic speaker gender detection algorithms, based on acoustic machine learning models. WSTP variations are presented across channels, years, hours, and regions. Results show that men speak twice as much as women on TV and on radio in 2018, and that they used to speak three times longer than women in 2004. We also show only one radio station out of the 43 channels considered is associated to a WSTP larger than 50%. Lastly, we show that WSTP is lower during high-audience time-slots on private channels. This work constitutes a massive gender equality study based on the automatic analysis of audiovisual material and offers concrete perspectives for monitoring gender equality in media.The software used for the analysis has been released in open-source, and the detailed results obtained have been released in open-data.
Editorial Special Issue Audiovisual Data in Digital Humanities
Fickers, Andreas; Snickars, Pelle; Williams, Mark J. (2018-10-31)
This issue of VIEW provides a critical survey of new digital humanities (DH) methods and tools directed toward audiovisual (AV) media. DH as a field is still dominated by a focus on textual studies (studies of word culture) that are largely “deaf and blind” in their capacity to search, discover, and study AV materials. The mandate to improve these capacities is clear and unquestioned, though the pathways are fecund and numerous. New and emergent tools related to deep learning algorithms are reasonably expected to change this methodological landscape within the digitally accelerated near-future.
Fingal's Cave: The Integration of Real-Time Auralisation and 3D Models
Noble, Shona Kirsty (2018-12-31)
Fingal’s Cave: an Audiovisual Experience is an immersive virtual reality application that combines 3D models, a narrative soundscape and interactive auralisation in a recreation of a visit to Fingal’s Cave. This research explores the importance of audio in heritage visualisations and its practical implementation. Fingal’s Cave is a sea cave on the Isle of Staffa off the west coast of Scotland revered for its extraordinary acoustics. Audio is extremely important in the history and culture of Fingal’s Cave and it has long been romanticised, inspiring countless folklore, art, poetry and music. The visualisation is designed to encourage viewers to become a part of the cultural narrative and explore the cave for themselves, move around and speak to hear their voice auralised as it would be inside the cave. This is the first time the acoustic characteristics of a heritage site have been included in a visualisation in this interactive manner. This paper reviews whether auralisation is effective and meaningful and supports a creative response to heritage sites. The impact of the visualisation in terms of engaging with communities of interest and in the field of audio in heritage visualisation is discussed. The research suggests it is necessary that audio be included in heritage visualisations to give a full and complete understanding of how people experience it.
Maps, Distant Reading and the Internet Movie Database: New Approaches for the Analysis of Large-Scale Datasets in Television Studies
Taurino, Giulia; Boni, Marta (2018-12-31)
The presence of large-scale data sets, made available thanks to information technology, fostered in the past few years a new scholarly interest for the use of computational methods to extract, visualize and observe data in the Humanities. Scholars from various disciplines work on new models of analysis to detect and understand major patterns in cultural production, circulation and reception, following the lead, among others, of Lev Manovich’s cultural analytics. The aim is to use existing raw information in order to develop new questions and offer more answers about today’s digital landscape. Starting from these premises, and witnessing the current digitisation of television production, distribution, and reception, in this paper we ask what digital approaches based on big data can bring to the study of television series and their movements in the global mediascape.
Narratological Approaches to Multimodal Cross-Cultural Comparisons of Global TV Formats
Larkey, Edward (2018-12-31)
This article cross-culturally compares different versions of the Quebec sitcom/sketch comedy television series Un Gars, Une Fille (1997-2002) by examining the various gender roles and family conflict management strategies in a scene in which the heterosexual couple visits the male character’s mother-in-law. The article summarizes similarities and differences in the narrative structure, sequencing and content of several format adaptations by compiling computer-generated quantitative and qualitative data on the length of segments. To accomplish this, I have used the annotation function of Adobe Premiere, and visualized the findings using Microsoft Excel bar graphs and tables. This study applies a multimodal methodology to reveal the textual organization of scenes, shots and sequences which guide viewers toward culturally proxemic interpretations. This article discusses the benefits of applying the notion of discursive proximity suggested by Uribe-Jongbloed and Espinosa-Medina (2014) to gain a more comprehensive and complex understanding of the multimodal nature of cross-cultural comparison of global television format adaptations.
Newsreels versus Newspapers versus Metadata: A Comparative Study of Metadata Modelling the 1930s in Estonia
Ibrus, Indrek; Ojamaa, Maarja (2018-12-31)
This article offers a comparative take on the ways audiovisual versus verbal digital archives model our understanding of the past. We focus on content metadata schemas and their role in modeling histories and framing the uses of audiovisual databases. Our empirical corpus includes verbal and audiovisual objects from the five-year period just before the World War II (1935-1939) as presented in two digital databases – the Analytic Bibliography of Estonian Journalism and the Estonian Film Database. The article compares how the different metadata schemas for newspaper articles and newsreels model their objects. As a consequence, metadata schemas shape contemporary perceptions of historical realities in different ways.
Speech Analytics in Research Based on Qualitative Interviews: Experiences from KA3
Leh, Almut; Köhler, Joachim; Gref, Michael; Himmelmann, Nikolaus P. (2018-12-31)
The paper presents aims and results of the project KA³ (Kölner Zentrum Analyse und Archivierung von audio-visual-Daten), in which advanced speech technologies are developed and provided to enhance the process of indexing and analysing speech recordings from the oral history domain and the language sciences. Close cooperation between speech technology scientists and digital humanities researchers is an important aspect of the project making sure that the development of the technologies answers the needs of research based on qualitative audio-visual interviews. For practical research reasons, the project focuses on the audio aspect, although visual aspects are of course equally important for the analysis of audio-visual data. The Cologne Centre for Analysis and Archiving of audio-visual data will provide the technologies as a central service.
Tales of a Tool Encounter: Exploring Video Annotation for Doing Media History
Aasman, Susan; Melgar Estrada, Liliana; Slootweg, Tom; Wegter, Rob (2018-12-31)
This article explores the affordances and functionalities of the Dutch CLARIAH research infrastructure – and the integrated video annotation tool – for doing media historical research with digitised audiovisual sources from television archives. The growing importance of digital research infrastructures, archives and tools, has enticed media historians to rethink their research practices more and more in terms of methodological transparency, tool criticism and reflection. Moreover, also questions related to the heuristics and hermeneutics of our scholarly work need to be reconsidered. The article hence sketches the role of digital research infrastructures for the humanities (in the Netherlands), and the use of video annotation in media studies and other research domains. By doing so, the authors reflect on their own specific engagements with the CLARIAH infrastructure and its tools, both as media historians and co-developers. This dual position greatly determines the possibilities and constraints for the various modes of digital scholarship relevant to media history. To exemplify this, two short case studies – based on a pilot project ‘Me and Myself. Tracing First Person in Documentary History in AV-Collections’ (M&M) – show how the authors deployed video annotation to segment interpretative units of interest, rather than opting for units of analysis common in statistical analysis. The deliberate choice to abandon formal modes of moving image annotation and analysis ensued from a delicate interplay between the desired interpretative research goals, and the integration of tool criticism and reflection in the research design. The authors found that due to the formal and stylistic complexity of documentaries, also alternative, hermeneutic research strategies ought to be supported by digital infrastructures and its tools.
The Researcher as Storyteller: Using Digital Tools for Search and Storytelling with Audio-Visual Materials
Hagedoorn, Berber; Sauer, Sabrina (2018-12-31)
This article offers a first exploratory critique of digital tools' socio-technical affordances in terms of support for narrative creation by media researchers. More specifically, we reflect on narrative creation processes of research, writing and story composition by Media Studies and Humanities scholars, as well as media professionals, working with crossmedia and audio-visual sources, and the pivotal ways in which digital tools inform these processes of search and storytelling. Our study proposes to add to the existing body of user-centred Digital Humanities research by presenting the insights of a cross-disciplinary user study. This involves, broadly speaking, researchers studying audio-visual materials in a co-creative design process, set to fine-tune and further develop a digital tool (technically based on linked open data) that supports audio-visual research through exploratory search. This article focuses on how 89 researchers – in both academic and professional research settings – use digital search technologies in their daily work practices to discover and explore (crossmedia, digital) audio-visual archival sources, especially when studying mediated and historical events. We focus on three user types, (1) Media Studies researchers; (2) Humanities researchers that use digitized audio-visual materials as a source for research, and (3) media professionals who need to retrieve materials for audio-visual text productions, including journalists, television/image researchers, documentalists, documentary filmmakers, digital storytellers, and media innovation experts. Our study primarily provides insights into the search, retrieval and narrative creation practices of these user groups. A user study such as this which combines different qualitative methods (focus groups with co-creative design sessions, research diaries, questionnaires), first, affords fine-grained insights. Second, it demonstrates the relevance of closely considering practices and mechanisms conditioning narrative creation, including self-reflexive approaches. Third and finally, it informs conclusions about the role of digital tools in meaning-creation processes when working with audio-visual sources, and where interaction is pivotal.

Recent Submissions