Customer Stories
Automated Labels for Better Broadcasting Insights with Gen AI
Xebia helps AVROTROS automate video labeling to get more insights into viewing behavior
AVROTROS is a Dutch radio and television public broadcaster that produces a wide range of content. It aims to make culture accessible to all Dutch people, from young to old.
Some notable content includes:
- • News and consumer affairs programs such as EenVandaag and Radar
- • Entertainment shows like Beste Zangers (Best Singers) and Op zoek naar... (Searching for...)
- • Comedy shows such as Dit Was Het Nieuws (This Was the News)
- • Music events like Muziekfeest op het Plein (Music Festival on the Square)
- • Coverage of the Eurovision Song Contest and Junior Eurovision Song Contest
AVROTROS focuses on connecting creative talent with audiences and bringing people together through shared cultural experiences.
AVROTROS reaches 2 out of 3 people in the Netherlands per monthu200b. They wanted more insight into their viewing numbers to determine what's driving people in or away.
Why
To get better insights into viewers' interests and preferences, AVROTROS wanted to build a solution to get granular insights into their videos stored in GCP.
What
Together with AVROTROS, Xebia built a GenAI-powered solution that automatically analyzes raw video files from TV programs to extract granular insights. The solution identifies changes in topic and people in the recordings.
How
Before the Project
In a fast-paced media industry, AVROTROS is responsible for broadcasting and maintaining quality TV content. Before the project with Xebia, AVROTROS already collected viewing numbers with minute-level granularity, allowing them to analyze their produced content. They had also created a data analytics platform to democratize access to said viewing data.
However, they also wanted to segment those viewing numbers based on the topics covered on the show and the people present in the recording. This would allow them to ensure their content aligns with the audience's needs. Since TV is a fast-changing medium, the topics covered might not be known beforehand and can, therefore, only be extracted from the recordings after broadcasting. This requires manually timestamping each episode of the show, identifying topics and people, and storing the results, which is a time-consuming process currently performed only on a selected number of shows and episodes.
Automated Labeling Solution
AVROTROS asked Xebia to help implement an automated labeling solution that would identify and timestamp the topics covered during the show as well as the guest speakers, and prepare that data for ingestion into the broadcaster's Google Cloud Platform (GCP). The implemented solution relies on Google Speech API to detect language and transcribe videos, Gemini to extract topics, and Google Video Intelligence API for face detection and clustering to identify people. An extra computer vision model, in combination with a language model, attempts to identify the name and role of the person on screen. Once the labels are extracted and timestamped, this information is stored in a vector database to allow for efficient and fast search results.
Additionally, to build trust in the implemented solution and to inspect or correct automatically-created labels if necessary, an intuitive web app was built for end users.
The Impact
The implemented solution can correctly identify and timestamp about 90% of the topics covered in a specific episode, as well as close to 90% of the people. In addition, an intuitive app was deployed, allowing users to correct labels detected by the algorithm if necessary.
With the detected timestamps for topic changes and relevant people in TV programs, AVROTROS is now able to associate viewer numbers directly with the topics discussed and the people in their TV content. This allows them to steer the content to align better with the audience's needs in subsequent broadcasts.
Contact