Kingston University – Video Image People Tracking

While some say that CCTV “watches” us, in truth it’s sometimes just staring at a point.

If someone or something happens to move into that frame, it can be recorded and viewed. But what if you want something a little more intelligent and reactive?

In 2012, a prototype created by Ipsotek, a tech company in South West London, fought off Samsung and Panasonic to win CCTV System of the Year at the annual International Fire and Security Exhibition and Conference (IFSEC).

Its innovation? A system known as “Tag and Track”. Tag and Track allowed a user to identify an individual and follow them between multiple cameras by simply clicking on their image. The system could then search through its records to map out where that person had gone, or even make a prediction as to the route they might be taking. It was commercialised in conjunction with BAE Systems, one of the world’s leading defence organisations.

This is just one of the applications that has come out of Kingston’s Digital Imaging Research Centre (DIRC), a group that investigates how computer systems can be trained to track.

Did you make it to Led Zeppelin’s much-anticipated reunion show in 2007? The technology was deployed to help London’s O2 Arena record the size of the crowd filing into the venue. It also gave TV audiences watching the 2012 Paralympics a flavour of the speed and intensity of competition, measuring the speed at which wheelchair basketball and rugby players collided.

“The applications are in various fields, such as security, sports and commerce”, says lead computer scientist Dr James Orwell. “For example, you could be tracking a missing person, or modelling where people are going in a shop to help with layout design.

“This is a very fast moving area of research enabled by developments in modern computer technology. In the early ‘90s, it was very difficult to load even one whole image onto a computer. But in the last 20 years, we’ve seen an exponential surge in the power of computing devices.”

The Tag and Track prototype is one of a number of collaborations between DIRC and Ipsotek, whose customers include Transport for London, BP, BAA and the Australian government.

When the Digital Imaging Research Centre, at Kingston first formed in 1997, it was focused on how to discern interesting information from jumbled crowds captured on CCTV, especially across multiple cameras. In bustling spaces, it can be hard for humans to monitor a range of cameras on their own, so there is a need for systems to step in that can identify cues and threats with some level of intelligence.

Dr Orwell says: “Prof. Sergio Velastin did a lot of work on technology to alert transport providers to potentially-dangerous scenarios in their transport networks. If there’s a holdup in the flow of people and a couple of key points become blocked, it can quickly get to the stage where you need intervention.

“The research looked into systems that could identify anomalies, whether that was too much traffic or too little. For example, if there’s no one in a certain area, it might mean that a route leading to that area is blocked, or it can draw attention to an overcrowded part of the system.”

Kingston’s work on automatic video analytics was trailed in the field in underground stations in Paris, London and Rome. By teaching the system to identify flows of people and interpret what that might mean, operators were able to get useful information without needing to rely on the tracking of individual people.

These systems formed the basis of the commercial products created by Ipsotek, which was co-founded by Sergio Velastin, a professor of Applied Computer Vision at the university.

While many systems searched for a single “trigger”, Ipsotek developed Scenario-Based Detection that would note when multiple events happened at the same time. So rather than flag up a vehicle every time it sat at an airport terminal for more than three minutes, it would do so when it confirmed that the vehicle was stationary, that it was in a particular area, and that it had been abandoned by its occupant.

In the meantime, work on individual detection was taking place. Working via a Knowledge Transfer Partnership with Ipsotek, researchers such as Professor Tim Ellis and Dr Dimitrios Makris shared their findings to help boost the capabilities of their technology in people tracking and counting. The project also benefitted from the hard work of Sateesh Pedagadi, a DIRC PhD student who also works at Ipsotek.

To track an individual, the system must be able to match a single object across multiple cameras, and differentiate them from passing people and background objects.

Dr Orwell says: “The difficult bit is determining what is caused by people moving around as opposed to shadows and windswept trees. This segmentation of ‘foreground’ from background is a core technology for many applications.

“Our approach uses optical tracking as opposed to RFID tracking, which would require tags. The advantage with optical is that it’s completely passive and non-invasive. But there are some real challenges, such as when crowds come together or it starts snowing.

“Our background models have now become much more sophisticated. Different climates can be learned, so a system can recognise a background is the same whether it’s sun or shade. It’s also using observations from different cameras to develop a clearer understanding of who’s where.”

This technology has been used in situations where crowds are particularly daunting. In 2010, Ipsotek deployed an “anti-backtrack” system which monitored the flow of passengers through airports including Edinburgh, Birmingham and Bristol. And it helped to count capacity crowds into the O2 Arena during the Led Zeppelin performance and the 2011 Prince concert. This footfall data was used to maintain safety, as well as calculate the amounts charged to promoters and advertisers for future events.

The intrusion detection technology developed for the London Eye demonstrated the system’s ability to monitor approaches on water as well as land.

Dr Orwell says: “This system was working in areas that included open water, as obviously the Eye is on the banks of the Thames. This showed that it was improving, and could cope with more complex moving environments as well as static land.”

Sport has also been an intriguing application of the technology. Initially, DIRC researchers set up cameras at football stadia to see if they could effectively track the positions of players. DIRC went on to work with Premier League football clubs to gather valuable information on performance, tactics and fitness.

In 2012, DIRC’s statistics were being beamed into living rooms around Britain during the Paralympic Games in London. It was commissioned by global sports business company deltatre to generate graphics that were seen during the coverage of 42 wheelchair rugby and wheelchair basketball matches on Channel 4.

Dr Orwell says: “We developed a system that was tracking individuals and collisions, and marking the speed at which that collision occurred. The production system allowed a graphic to be produced that could be overlaid       on-screen by a director in a couple of seconds.“

DIRC has continued to develop its interest in sport, particularly in football. A project designed to accurately track all players on a football pitch began trials earlier this year.

As technology continues to improve, Kingston’s researchers are carrying on their work to develop systems that can accurately and promptly identify important events as they happen.

Dr Orwell says:

We’re talking about improving the effectiveness of algorithms and, ultimately, getting near or surpassing human performance.

“If you’re trying to identify people across multiple cameras where you’ve got a constrained area and you’re walking from A to B, that works nicely. But if you have no idea where people are and when they’re coming through a door, it’s a much tougher nut to track. Looking at the future, the emergence of even higher resolution cameras can make detection even easier.”

Links to Additional Information

Digital Imaging Research Centre