The SMARTLENS project leverages Mo-Sys' leadership in studio-quality virtual production and opens up broad new markets to affordable high-quality video production.
Video is ubiquitous, comprising 82% of all Internet traffic and displacing other forms of media in advertising, in education, news and beyond; attracting professionals from across the creative industries (£268 Bn) to explore and participate in new forms of content production.
'Virtual Production' technology allows filmmakers to combine images captured on camera with computer-generated elements. The SMARTLENS project creates a new automated method of essential lens calibration, allowing significant cost-saving (and carbon-saving), as production teams no longer need to travel to remote locations to capture difficult shots.
This is a collaborative project building on long-time collaboration with Professor Simon Julier (UCL/TMI), innovating novel AI-based automated lens calibration. All lenses distort the path of light in some way: changing the focal length changes the field of view of the lens, and straight lines in the real world do not necessarily look like straight lines in the image plane. Failure to account for this distortion means that the virtual graphics do not line up with the real world, and any notion of the graphics being anchored in the real world is lost. Conventional lens calibration techniques are slow and cumbersome, requiring the operator to manually measure targets and camera movements repeatedly.
SMARTLENS develops a set of algorithms and techniques which will automate the calibration approach. First, a suitable model of lens parameters will be chosen. Conventional computer vision models do not describe effects such as depth-dependent radial distortion for out of focus images. Therefore, we will chose a suitable model from the photogrammetry literature. Second, we will develop the back end optimisation techniques which will fit the parameters to suitable data. Third, we will investigate the use of various kinds of calibration targets, including points and lines to see which ones are most robust over the range of operating conditions. Finally, we will investigate suitable practices which give the best performance.
Mo-Sys is the largest virtual production technology provider today, with a substantial IP portfolio and a sophisticated high-end customer base (Sony, Warner Bros, Disney, Netflix, BBC, NHK, Fox, ESPN, Sky, CNN). Credits range from BBC's Match of the Day to films such as Gravity and Life of Pi. Through this project, this collaboration now continues to innovate and extend lens calibration and tracking systems,
Project DELAMBDA aims to develop innovative technology to extend the capabilities of online face verification systems. Building on iProov's world-leading, patented technology it introduces new variables and new analysis techniques to the task of detecting replica and replay attacks on individuals, using pioneering ways of using open standard technology that have not previously been used in a broad application context. The outcome will be greater security for citizens accessing online services, without compromising usability.
FOCAL International (FOCAL) is the leading global trade association facilitating use of commercial footage and other content in all forms of media production, with over 300 international members comprising content libraries, archives, production researchers and service providers. The company has offered a popular "Footage Finder" service for many years, generating sales leads and routing these to members. The QUEMAT project creates novel deep learning technology (via technology partners Dithen and The Media Institute) to innovate a high-impact new "Footage Finder" service, leveraging the established footprint of FOCAL International, and in turn generating an abundance of learning and training data through adoption to continuously improve the project's deep learning performance. The QUEMAT sustainable ecosystem delivers 'network effects' inside and outside the project, yielding high-precision and commercially relevant results to visual search queries within Footage Finder and across an abundance of applications.
The project is ideally timed as inflection points have been reached concurrently in three disparate arenas: a) film and television production is burgeoning with new routes to market (e.g. Netflix now invest more in original production than CBS); b) deep neural network (DNN) technology has reached a level of maturity allowing investments in machine learning to deliver unprecedented returns; and c) technology to perform 'video signature extraction' (i.e., allowing for rapid search of video without the need to process pixels) has been validated by new MPEG standards activity: 'Compact Descriptors for Video Analysis' (CDVA), creating a foundation for widespread adoption.
The project will innovate the novel DNN (Deep Neural Network) -based media asset processing and discovery capabilities with the following unique features: i) it will allow for the continuous production of compact signatures per query type while remaining standards-compliant; ii) it will be specifically trained to be repurposed for various query types with an automated query-driven customization stage that can be performed offline; and iii) the operational complexity will be tunable and the entire software pipeline will be portable to any public cloud provider for easy scaling and adoption across media enterprises.
Overall, QUEMAT technology will deliver increased revenues to content owners and increase operational efficiencies at a time when content producers and footage libraries are struggling to reconfigure traditional value chains to benefit from new monetization opportunities. The project's three partners: FOCAL, The Media Institute and Dithen -- bringing together media industry presence and leading technology expertise in deep learning and media processing, and a record of successful collaboration.
Creative content is the undisputed driving force and focal point of the majority of online consumer activity today. However, a lot of the current and past commercial value is lost by not allowing for image and video content to be uniquely and robustly linked to copyright and ownership details. Further, in the professional media industry, supply chains encounter complex rights issues which cause 80% of all content not to be 'rights ready', or commercially viable. While at a first glance this seems like a problem that can be addressed with off-the-shelf components, deliberate obfuscations or accidental variations in content and rights (collectively called "uncertainty"), do not allow for conventional online search tools to work well for this problem. This leads to the current situation where rights discovery for online content is a manual, error prone and cumbersome process, with substantial effort required to remain within the law, and virtually no effort (and minimum risk) for those that wish to violate copyright law. The LUCID Rights project brings together an interdisciplinary team of internationally-leading experts in machine learning, knowledge discovery, high-performance image & video engineering, copyright law, and business & content licensing models in order to address this important challenge. The key objective is to create a unique solution for rights discovery and indelible signature creation for the content properties and copyright information that will be robust to noise or uncertainty in the rights description. The confluence of standards for machine-readable contracts and compact signature generation allows for this to also be trialed within standard-compliant mechanisms, thereby enabling openness, and commercial traction within short timeframes. This will allow for the first time to apply advanced machine learning to disrupt the domain of digital content rights, thereby unlocking the potential for new markets and services within the UK and internationally.
Project DELIGHTA aims to develop innovative technology to extend the capabilities of online face verification systems. Building on iProov's world-leading, patented technology it introduces new variables and new analysis techniques to the task of detecting replica and replay attacks on individuals, using pioneering techniques that have not previously been used in a broad application context. The outcome will be greater security for citizens accessing online services, without compromising usability.
The DELVE-VIDEO project creates a robust and performant ecosystem of software tools and infrastructure components to uniquely identify and describe video attributes within networks and file systems. This establishes a foundation for content owners and service providers to protect their video assets from piracy, measure viewer traffic, and enrich asset management, rights management and recommendation services, all with substantially advanced simplicity and automation in comparison to existing methods. The project builds on novel video signature extraction technology developed by TMI/UCL (in part via Innovate UK –supported project Video Clarity) and Dithen. Deep learning methods will now be added and the signature extraction and content classification will be carried out predominantly using compressed domain information, coupled with very limited decoding. The principal aim is to enable unique semantic identification of media regardless of platform (film, television, web, OTT, mobile) or encoding characteristics. Compelling new commercial services and technology licensing opportunities will be launched by the project partners, enabling new levels of analysis and reliability for copyright protection and information services.
Project COMVIDIA creates an innovative new identity assurance technology, capable of being used as a distinctive factor in a highly secure multi-factor authentication solution, or to offer users a simple substitute for CAPTCHA-type mechanisms, which restrict system access for human-only entry. The technology combines extreme simplicity for the user with scalable security for the service provider, enabling a wide range of applications. The project builds on insights gained by lead partner iProov in the course of innovating world-class ‘ID as a Service’ technology, and from new research and experiments in image processing using novel ‘Eulerian video’, a method of magnifying human features in order to detect characteristics of human liveness. iProov will investigate image science and other aspects of pre-industrial research, and The Media Institute (a wholly-owned entity of University College London) will collaborate on validation of the approach, providing advanced usability testing.
Project VASOCISE addresses a core challenge for SMEs today - how to create a secure environment for staff who frequently are working from home, part-time, on site or travelling, whilst not creating an intolerable burden of cost, complexity and unenforceable rules. The project builds on iProov's highly useable zero-effort authentication technology to create a zero effort access solution for SMEs, tightly controlling access to their systems and their data in a highly useable way. VASOCISE also addresses the complexity of implementation and management, with some highly innovative solutions to make these so simple for SMEs that the threshold of adoption can be crossed, and the data of staff, management and customers can be protected.
Video content is a primary business asset of the thriving UK and global creative industries, and represents a substantial proportion of today’s “big data” deluge. Beyond the creative industries, video comprises a major communications tool, occupying 60% of today’s Internet traffic. However, with millions of users creating, processing and (not always legally) uploading exabytes of video content each week, video remains the least-manageable element of the big data ecosystem. This is because all current methods for high-level semantic description in video require either manual annotating or compute-intensive video decoding & processing. Delivering cost-effective and meaningful video search has therefore proved to be an insurmountable problem. This project takes a novel insight: that a hidden source of coding-related metadata already exists within modern compressed video file containers, and it is sufficient for automated tagging and visualization. Tapping into this “hint” metadata enables video streams to be analyzed with 3~6 orders increase in speed and decrease in cost, enabling exabyte-scale video datasets to be newly-discovered and analysed over commodity hardware.
Project ZIVVI (Zero-effort Identity Verification using Video Illumination) takes a novel approach to user identification and access management. Today's devices combine high quality embedded cameras and Internet connectivity; using the unique signature of a flash of illumination, project technology is able to generate a video sequence which enables users to simply view their device in order to be authenticated to service providers such as banks, ecommerce, social media and paid content sites. The service will be offered as an OAUTH service, and incorporates unique technologies from image processing and biometric feature detection.