Joint Contest on Multimedia Challenges Beyond Visual Analysis @ICPR2016
Workshop and Challenge on Multimedia Challenges
December, 2016 (exact day TBA)
This page provides additional information about the data and evaluation metrics for the four tracks of the Joint Contest on Multimedia Challenges Beyond Visual Analysis @ ICPR16
Track 1: First impressions
As part of the speeds interviews project, we are organizing a challenge on “first impressions”, in which participants will develop solutions for recognizing personality traits of users in short video sequences. We are making available a large newly collected data set sponsored by Microsoft of at least 10,000 15-second videos collected from YouTube, annotated with personality traits by AMT workers. A first round of the first impressions challenge was run as part of the ECCV16 workshop program. In this second stage of the challenge participants can further improve their solutions and collaborate with other participants, through the coopetition scheme: in which participants are motivated to collaborate for succeeding in the competition.
Table 1. Sample videos that will be used for the first impression challenge.
The traits to be recognized will correspond to the “big five” personality traits used in psychology and well known of hiring managers using standardized personality profiling: Extroversion, Agreeableness, Conscientiousness, Neuroticism, and Openness to experience.
Evaluation of track 1. Participants
will be evaluated by a measure associated to the number of correct
traits being recognized in the test set. More specifically, the
evaluation consists of computing the mean accuracy among the video
tested for the 5 traits between the predicted continuous values and the
continuous ground truth values.
Track 2: Isolated gesture recognition track
are organizing a track on isolated gesture recognition from RGB-D data,
where the goal is developing methods for recognizing the category of
human gestures from segmented RGB-D video. A new data set called ChaLearn
LAP RGB-D Isolated Gesture Dataset (IsoGD) is considered for this track
(sample images taken from this data set are shown in Figure 1). This
database includes 47,933 RGB-D gesture videos (about 9G). Each RGB-D
video depicts a single gesture and there are 249 gesture categories
performed by 21 different individuals. Performance of methods will be
judged by its recognition performance.
As previous challenges organized by Chalearn, all the four tracks of the contest will be based on the CodaLab platform, information about registration and participation can be found in the CodaLab's tracks' sites:
In broad terms the challenge will proceed as follows (track 1 will adhere to slightly different procedure):
Important dates (quantitative challenge):