Task C – Activity Detection in Extended Videos Sequestered Data Leaderboard (ActEV SDL)

The ActEV SDL (Activities in Extended Video Sequestered Data Leaderboard) is an ongoing ranking of software systems that process lengthy videos and detect activities of interest. The ActEV SDL guest task under the CVPR'20 ActivityNet workshop is based on sequestered data from the MEVA Test 3 dataset, which contains ~140 hours of videos, including indoor and outdoor scenes, night and day, crowds and individuals, and videos are from both EO/IR sensors. The top two teams on the ActEV SDL leaderboard by the submission deadline (May 10, 2020) will be invited to give an oral presentation at the CVPR'20 ActivityNet workshop

ActEV SDL website: https://actev.nist.gov/sdl

If you have any question about the SDL evaluation, please email to: actev-nist[at]nist.gov

Schedule

March 01, 2020

ActEV SDL opens with MEVA Test3

May 10, 2020 at 11 AM EST

Deadline for CLI submissions to be included in ActEV guest task under CVPR'20 ActivityNet workshop.

June 01, 2020

We will invite the top two teams on the SDL leaderboard to give ActEV guest task oral presentations at the CVPR'20 ActivityNet workshop based on the CLI submission deadline.

June 14, 2020

CVPR'20 ActivityNet workshop ActEV SDL guest task presentations

Dataset

The data is from the Multiview Extended Video with Activities (MEVA) dataset [mevadata.org] and the videos are from both EO (Electro-Optical) and IR (Infrared) sensors. The data used for ActEV SDL evaluation from March 01, 2020 is the MEVA test3 dataset (~140 hours of video) instead of MEVA Test2 dataset (72 hours of video). The public MEVA dataset includes hundreds of hours of data from the same cameras at the same facility, which can be used for training. If you register for ActEV you can download the MEVA dataset for free; info on how to download the data is on the data tab. We also provide 20 hours of MEVA annotations, and instructions on how to make and share activity annotations are at mevadata.org.

Evaluation

Detect if and when an activity occurs. Given a target activity type and a set of videos, submitted systems must automatically detect all instances of the activity in the videos. While different activities instances have different durations, a submitted system will be considered to have detected an activity if it correctly identifies at least 1 second of the activity. Submitted systems are scored for Pmiss and TFA at multiple thresholds, creating a detect error tradeoff (DET) curve. The leaderboard ranking of a system is based on a summary of its DET curve: the average value of 1-Pmiss across the TFA range between 0% to 20%. You can find more details of the ActEV SDL evaluation plan and the ActEV scoring software.