Region-based surveillance video retrieval with effective object representation

Retrieving the most relevant video frames that contain the object specified in a given query (query-by-region) remains a challenging task. Two common challenges of region-based retrieval approaches are to accurately extract or segment object(s) and select a proper matching strategy. This paper addresses these problems by proposing a retrieval approach that uses a new region-based matching technique equipped with an effective object representation method. In the first stage, the proposed approach selects the most informative instances of each object that appeared in the video by utilizing an adapted clustering algorithm over the extracted features. In the retrieval stage, the new matching technique returns the most relevant sequences of video by mapping a given region with those identified representative instances of objects based on their similarity scores. The proposed approach is evaluated on standard datasets and the results demonstrate a 31% improvement in the retrieval performance compared to other state-of-the-art methods.