First technology to recognize flow of people from low-resolu
Source: Fujitsu

Figure 1: Recognition and utilization of patterns in the flow of people

Fujitsu today announced that it has developed the industry's first technology that provides highly accurate detection of people's movements and that can recognize the flow of people from low-resolution imagery incapable of distinguishing faces. In existing systems that analyze video taken from surveillance cameras and recognize flows of people, the problem has been that the resolution may be high enough to identify individuals, rendering the footage unusable.

Now Fujitsu Laboratories has developed technology that, even when the overall image resolution has been degraded from high to low resolution so that people's faces cannot be recognized, extracts multiple defining features of an individual, thereby making it possible to detect the person's movement with a high degree of accuracy. This makes it possible to detect people in video with resolution low enough to mitigate anxieties about privacy, and thereby enable recognition of people's movement. Details of this technology are being presented at the Information Processing Society of Japan's technical committee conference, "Computer Vision and Image Media", opening March 6 at Tohoku University.

In recent years, data on the movement paths of people inside a facility or on the streets of a city have started to be used by stores, for example, to optimize the selection of a store location or the arrangement of merchandize or deployment of staff inside a store, in order to improve customer service. There are rising expectations about the use of data on pedestrian flows in cities to relieve congestion when there are events, to devise operating plans for public transportation systems, and to guide evacuations during disaster emergencies.

When video images from surveillance cameras installed on city streets or in commercial facilities are used to recognize flows of pedestrian traffic, the privacy of individuals whose movements are being recorded need to be taken into consideration. As the resolution of these cameras has improved, it has become possible to clearly recognize a person's face, which, although unproblematic from the perspective of crime prevention, raises privacy issues when third parties have information on a person's movements, such as what the person was doing at a particular time and place. Typically, the following technologies are used to extract information on flows of people.

Person detection: extract a person-like shape from an image.

Person matching: using multiple cameras, determine whether a person detected by one camera is the same person detected by other cameras.

Because these technologies have used information detected from a person's face, build, gender, and other personal characteristics to determine if it is the same person, the problem has been that the person may be identified, causing the video to be unusable.
First technology to recognize flow of people from low-resolution imaging
Figure 2: Human features extraction from low-resolution imagery (Left: input image; Middle: First-round detection; Right: Detection results)

Using features that people have in common, Fujitsu Laboratories has devised person-recognition technology that extracts body-like shapes such as heads and torsos in low resolution images, and determines if the person is the same based on clothing colors. Key features of the technology are as follows:

1. Person detection using low-resolution images (Figure 2)

Fujitsu Laboratories focused on the fact that, even in low-resolution images, the head persists as a distinctive human characteristic that is unaffected by a person's posture or the angle from which the person is seen. Regardless of whether viewed from the front, back, left, or right side, the established characteristics of the head being a roughly elliptical shape and at the top of a person's body have the advantage of persisting even in low-resolution images. The actual extraction process is as follows:

Focusing on head-shaped characteristics in the input image (Figure 2-left), extract head-shaped candidates (Figure 2-middle).

Because many similar shapes were detected, treating the lower ones as torso shapes makes it possible to accurately detect people even from low-resolution images (Figure 2-right).
First technology to recognize flow of people from low-resolution imaging
Figure 3: Person matching using low-resolution images

Based on the people detected in Step 2, detect their relative proximity to the camera, beginning with the closest person, and update the image of more distant persons to compensate for partially obscured areas of a body. This makes it possible to accurately detect even multiple, overlapping people.

This technology can independently recognize the heads and torsos of multiple people.

2. Person matching from low-resolution images using clothing colors (Figure 3)

Matching a single detected person between multiple cameras makes it possible to recognize a person's movements between cameras. The same color can be extracted regardless of resolution, but typically in low-resolution images, the colors of smaller areas are absorbed and mixed into adjacent areas, making good detection impossible. By selectively extracting only the distinctive colors of a person's clothing, Fujitsu Laboratories has been able to perform reliable person matching using low-resolution images.

Until now, it has been difficult to perform person detection and person matching using low-resolution images in which a person's face cannot be identified, but using the technology it developed, Fujitsu Laboratories performed an indoor trial, in which it was able to detect, in the flow of people, the paths of approximately 80% of the people. As a result, when arranging merchandize or deploying staff inside a store, for example, it makes it possible to use surveillance camera images that respect the privacy of the people whose images are being recorded, and enables the use of images that do not identify individuals to improve customer service.