My research interests are in the areas of real-time computer vision and image processing, with applications in human activity monitoring and recognition. The long term goal in this field of research is to move farther up the semantic ladder by learning to understand complex activities in terms of abstract events, much like humans do. For now, we are working to add levels of abstraction in order to be able to understand a sequence of simple, image level changes as a specific event or activity. I am also interested in problems related to 3D inference, where the goal is to infer 3D information about the scene, an object, or a motion given cues in an image or video.

Human Activity Monitoring for Homeland Security

I am a member of a team that is developing a large-scale distributed video surveillance system for the Department of Homeland Security. The system is designed to be used in busy public transportation areas, and can automatically detect several different kinds of activities that are deemed suspicious. It operates in real-time and combines this activity recognition functionality with standard surveillance capabilities, such as video recording, playback and camera control. The system is currently in use at some small test sites, and is in the process of being deployed at a large site with more than 100 cameras. The activities that are recognized are a set of well-defined, complex behaviors. For example, the system currently detects abandoned objects, tracks the trajectories of thrown objects, and detects sudden irregular crowd movements, to name just a few. A couple of the activity recognition modules that I have worked on are described below.

For more information about this and other projects, refer to the following links:
Artificial Intelligence, Robotics, and Vision Laboratory
The Monitoring Human Activity Project

Publication: P. Kilambi, E. Ribnick, A. Joshi, O. Masoud, and N. Papanikolopoulos. "Estimating Pedestrian Counts in Groups", Computer Vision and Image Understanding (CVIU), 2007 (to appear).

Camera Tampering Detection

The purpose of this module is to automatically detect camera tampering in real-time. Examples of camera tampering are a person holding a hand or other object in front of the camera, a light shining directly into the camera lens, or a person reaching up and turning the camera so that it points in a different direction. Such events should be brought to the attention of security personnel, since they may be indicative of more profound suspicious activity.

Camera Tampering is detected based on the idea that, when it occurs, the incoming frames of video will be very different than older frames of video from before the event began. As such, several different measures of image dissimilarity are used to robustly determine when such a large change has occurred. This involves comparing the old and new frames of video in a way that is insensitive to short, transient events that are not of interest. One of the challenging aspects of this problem is that false detections should not be caused by large changes in the scene that are not Camera Tampering. This method is robust in that it is insensitive to non-Tampering events such as large objects passing through the scene (i.e., trains, busses), changes in illumination, and large crowds of people entering the image, while still maintaining a high detection rate for real Camera Tampering events.

Publication: E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Real-Time Detection of Camera Tampering", Proceedings of the IEEE Int'l Conf. on Advanced Video and Signal Based Surveillance (AVSS 2006), Nov. 2006. (pdf)

Video: Tampering Demo - DivX (4.6MB) Note: you will need the DivX Codec to play this video.

Detection and 3D Localization of Projectiles

Automatic detection of projectiles in video has applications in both security/surveillance and sports video indexing. In this work, the goal is to detect thrown objects in video sequences in real-time, and identify their trajectories. Thrown objects are detected based on the assumption that they are small, fast-moving, and follow parabolic trajectories in the world coordinate system. This is done by first identifying areas of high motion in the scene via a sophisticated inter-frame differencing implementation, and then applying Expectation-Maximization to identify parabolic trajectories. Challenges are presented by other fast-moving objects in the scene, such as the hands and feet of running people. However, often times thrown objects can be distinguished from these other, uninteresting events based on some of the properties of their trajectories.

Given the apparent trajectory of a projectile in the image, it is also possible to infer its 3D position and velocity in the world.

Publication: E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Detection of Thrown Objects in Indoor and Outdoor Scenes", IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS 2007), Oct. 2007.

Video: Thrown Object Demo - DivX (14.9MB) Note: you will need the DivX Codec to play this video.