By Marc Górriz Blanch, ESR at BBC Research and Development
The perception of colour has been essential in modern image and video processing systems, playing an important role in many applications and domains. Colour technologies have long been the heart of the broadcasting industry. Back in the sixties, the invention of the colour television represented a total revolution, and one of the most complex and transformative technological innovations in the history of the world. Since then, broadcasting technologies have been in constant evolution leading towards the digital transformation. The requirement of colour reproduction has been a priority all along the process, such that the design of new colour imaging systems relied on maximising human perceptual plausibility. Digitisation has also driven the establishment of new computer vision disciplines, such as video enhancement and restoration, which have had a significant impact on the broadcasting industry in recovering and colourising degraded content from legacy archives. Archived broadcast content such as previously broadcasted shows, and historical event coverage is of relevance to journalism to support incoming news stories, and provide context to current events.
Colour imaging technologies have become essential in managing the reproduction of colour in a consistent and systematic way, such that the visual appearance remains perceptually constant. Another important element in the broadcasting chain are technologies for distribution. Video compression has become an essential asset for tackling the increasing demand for higher quality video content. Colour prediction plays an increasingly important role in video coding, proven to be effective on achieving better compression rates by means of exploiting cross-component information. Finally, Artificial Intelligence (AI) is increasingly creating disruption and innovation. Broadcasting applications driven by machine learning have rapidly shifted from research environments to deployed scenarios, enabling the automatisation of many production workflows. As recently issued by the International Telecommunication Union (ITU), such technologies are providing improvements in production efficiency and correlated cost reduction, as well as optimised content delivery at lower bandwidth.This research investigates extending existing AI approaches for application to colour processing in video broadcast systems. Video technologies are essential in the digital broadcasting and production workflow, especially for content production and distribution. In the context of video enhancement, this research studies the use of Generative Adversarial Network (GAN) models towards assisting automatic colour adjustment and colourisation with practical use cases in restoration and processing of archived content. The research targets practical deployable solutions, developing a cost-effective pipeline which integrates the activity of the user/producer into the processing workflow, together with automatic methodologies based on style transfer or smart retrieval. Distribution video technologies are essential to effectively deliver the processed content to the broad audience. Colour prediction extends directly to such research disciplines such as video compression. This research investigates the use of neural networks for improving existent chroma prediction methods, aiming an efficient deployment within the latest video coding standards. Finally, interpretability of neural networks is considered to verify the predictions of complex end-to-end models as well as identify potential simplifications towards more efficient implementations.