Download and Visualize using FiftyOne

We have collaborated with the team at Voxel51 to make loading, visualizing, and evaluating ActivityNet a breeze using their open-source tool FiftyOne.

As with any other dataset in the FiftyOne Dataset Zoo, loading ActivityNet is as easy as calling:

dataset = fiftyone.zoo.load_zoo_dataset("activitynet-200", split="validation")

The function allows you to:

• Choose which split to download.
• Choose which classes of activities to download (e.g. Horseback riding).
• Limit the number of samples, to do a first exploration of the data.
• Specify the maximum length of videos to download.

These properties give you the ability to quickly download subsets of the dataset that are relevant to you.

dataset = fiftyone.zoo.load_zoo_dataset(
        "activitynet-200",
        split="validation",
        classes=["Horseback riding"],
        max_duration=30,
        max_samples=3,
    )

FiftyOne also provides native support for ActivityNet-style evaluation to compute mAP, plot PR curves, interact with confusion matrices, and explore individual label-level results.

results = dataset.evaluate_detections("predictions", gt_field="detections", method="activitynet")

For additional details, see the FiftyOne and ActivityNet integration documentation

Missing Videos

To accommodate missing data requests, we have made the full dataset available on Google and Baidu drives. Please fill in this request form to have a 7-day-access to download the videos from the drive folders.

How to read the annotation files?

The annotation files are stored in JSON format. Each annotation file contains three different key fields: "database", "taxonomy", and "version". The key field "database" contains information about the videos in the dataset and all the available annotations. In the key field "taxonomy", we include a parent-child relationship for every activity in the dataset. Finally, we add the key field "version" to keep track over different releases of ActivityNet. Below, we present a detailed explanation of each key field.

database

Below, we show an example entry from the key field "database":

The example shows the information and annotations related to this video. As noted, we assign the unique identifier "5n7NCViB5TU" to that video. From the entry, we can retrieve the duration, resolution, url, and the subset where the video belongs. Additionaly, the key field "annotations" includes a list of all annotated activity instances in the video (segments). Each item in the list contains the key fields: "label" and "segment". The key field "label" refers to the activity class, and the key field "segment" contains the starting and ending time of the activity instance (in seconds). For example, the example above contains two different instances of the activity Discus throw: Instance 1, Instance 2.

taxonomy

The key field "taxonomy" contains information about all the nodes in the ActivityNet hierarchy. Below, we show an example that contains the leaf-to-root path of the activity Discus throw:

Each entry contains information about a node in the ActivitNet hierarchy. As noted, the leaf-to-root path can be recovered by walking the hierarchy (using the "parentId" field).

version

The key field "version" identifies which version of ActivityNet is contained in the annotation file. It is intended to keep track over different releases.