The Vanderbilt Television News Archive's Director, Jim Duran, and Data Engineer, Vaibhav Ravinutala, are rebuilding metadata pipelines for newly digitized or recorded television news broadcasts. Thanks to the Vanderbilt University's Heard Libraries' investment in a new web application and database service, the TVNA is adopting the latest innovations in technology as well as best practices in library metadata standards. Duran and Ravinutala reviewed the entire TVNA workflow, from initial capture of video to the search experience for users, and developed automated and semi-automated steps to create new records. This flow will be applied to news content captured after May 1st, 2025 and will power searches on tvnews.vanderbilt.edu.
Legacy of Innovation
The Television News Archive created its method for describing content a few years after the archive first began recording nightly evening news broadcasts in 1968. The video tapes were re-watched by indexing and abstract writing staff, including Skip Pfeiffer, Carol Bomar, and John Lynch. The archive staff identified each news segment following a style guide for writing headlines and summary abstracts. They found key names, places, events and commercial products and aggregated them for a monthly publication as well as an annual index. This process of watching the news, describing the content, and publishing a catalog continued from 1971 to 1995, when an electronic database replaced the print catalog. However, writing abstracts continued well into the 21st century. The archive published its last handwritten abstract in 2024, when abstract writer Skip Pfeiffer retired after 50 years with the department. He alone wrote hundreds of thousands of news segment summaries. By the time Pfeiffer retired, over one-third of the TVNA database entries were authored by him.
Automation increases capacity and availability
Any library or archive charged with preserving contemporary cultural heritage has been challenged by the rise of digital media. Traditional methods of archival processing are strained under the sheer volume of culturally significant digital artifacts. The TV News Archive, with its mission to create a record of popular audiovisual news, began facing this challenge with the launch of cable television in the 1980s. The inception of digital streaming channels in the 2010s further compounded the problem. As Americans' media choices expanded, so did the collection scope of the TVNA. Each collection expansion increased the need for staff time to summarize news segments and create a catalog for discovery. Unfortunately, the TVNA experienced a common archive problem: a backlog of uncataloged content.
In 2019, Duran introduced voice-to-text transcripts as a replacement for handwritten summaries. Users of the TV News Archive website may notice backlogged content from Fox News Channel beginning in 2008, and backlogged content from CBS and NBC beginning in 2014. All content after 2024 includes some form of transcript-based description. Transcripts have advantages and disadvantages in comparison to abstracts. Transcripts include all entities mentioned as speech such as names, locations, and organizations. However, automated speech recognition is not 100% accurate, so spelling mistakes will impact searches. The TVNA will continue to improve its transcript process to increase accuracy, but users should also take this change into consideration when developing a search strategy.
Duran and Ravinutala are integrating the latest advances in artificial intelligence to improve catalog records even further. This includes using AI to generate headlines, summaries, and identifying entities (names, locations, organizations, topics, etc.). This work, combined with improved database design and website search, will increase the accuracy of searches and ultimately save users time. These automations also free staff time, enabling the TVNA to expand its collection and process backlogs that are currently not available to researchers. If you would like to learn more about the TVNA metadata pipeline, you can watch their presentation "Metadata Matters: Modernizing the Vanderbilt Television News Archive Database" for the Coalition for Networked Information's Winter 2026 Pre-Record Edition Live.