Activity Recognition of Nurse Training Activity using Skeleton and Video Dataset with Generative AI
Brought to you by Sozolab in conjunction with ABC
Registration is closed! Thank you for your participation.
Follow @abc-research
Free entry + 100,000 jpy championship prize !!
Overview
The 6th ABC Challenge is the Activity Recognition of Nurse Training Activity using Skeleton and Video Dataset with Generative AI. Activity types are each task of Endotracheal suctioning.
- The dataset will provide the skeleton data for training / testing and the video data only during the training.
- The skeleton data include many samples which recognized only partial parts of the body due to camera location limitations.
- Participants are required to use a Generative AI or a Large Language Model (LLMs) (hereafter, Generative AI) in a creative way.
Endotracheal suctioning (ES) is a necessary practice carried out in intensive care units. It involves the removal of pulmonary secretions from a patient with an artificial airway in place. The procedure is associated with complications and risks including bleeding, and infection. Therefore, there is a need to develop an activity recognition system that can ensure the safety of patients as well as reflect to improve their skills while they conduct this complicated procedure. Activity recognition can be used to aid nurses in better managing and increasing the quality of their work, as well as evaluate their performance when they conduct ES. The activity recognition is the initial stage to determine the order of actions and assess the nurse’s skills.
Participants are required to recognize activities based on skeletal data. Since the data collection was a practical experiment, camera locations were limited by not showing the face, the size of the room, etc. Therefore, it is not possible to recognize all of the body parts, and many skeleton data had missing body parts. Additionally, Generative AI has been a hot topic in recent years and its momentum will continue to increase. In order to explore the potential for use in the field of activity recognition, participants are required to utilize the Generative AIs in a creative way.
Challenge Goal and Task
The goal of this challenge is to recognize 9 activities in Endotracheal suctioning (ES) by using skeleton data for training / testing and video only for training. In this challenge, participants are required to use the Generative AIs in a creative way. For evaluation, we will consider the F1 score and the paper contents. We will take an average F1 score for all the subjects.
The data we are providing is a part of the dataset used in our previous work, entitled “Toward Recognizing Nursing Activity in Endotracheal Suctioning Using Video-based Pose Estimation” [1]. The authors of this work proposed an algorithm to define and track the main subject. Also, missing keypoints problems due to the performance of the pose estimation algorithm are improved by smoothing keypoints.
[1] Hoang Anh Vy Ngo, Quynh N Phuong Vu, Noriyo Colley, Shinji Ninomiya, Satoshi Kanai, Shunsuke Komizunai, Atsushi Konno, Misuzu Nakamura, Sozo Inoue: “Toward Recognizing Nursing Activity in Endotracheal Suctioning Using Video-based Pose Estimation”, The 5th International Conference on Activity and Behavior Computing, 2023, (Germany).
Tutorial Webinar [CONCLUDED]
As part of this year's challenge, we organized tutorial webinars on January 17th, 2024 both in Japanese and English. Tutorial resources can be found in the following links:
Challenge Registration and Dataset Distribution [CLOSED]
Registration is now closed! Thank you all for your participation. Please note that the dataset is published only to those who have registered for the Challenge before the deadline.
Result submission
- closed
Paper submission
* When you submit your peper from CMT, please chose "Create new submission" with "Challenge" paper. * The challenge paper is limited to 4 - 8 pages.
Prizes
- The winning team will be awarded 100,000 jpy.
- The registration fee for the 1st and 2nd runner-up teams will be waived.
- Each of the participating teams will be awarded with participation certificate.
Important dates
This Challenge will be held as part of the ABC Conference 2024
Tutorial: Jan. 17, 2024Challenge opens: Jan. 17, 2024Registration closes: Mar. 6, 2024Submission of results: Mar. 23, 2024 (AoE) (Extended)Submission of paper: Apr. 12, 2024 (AoE) (Extended)Review sent to participants: Apr. 23, 2024(sorry for the delay)- Camera-ready papers: Apr 30, 2024(Extended)
- Conference: May. 29 - 31, 2024