repository
Action Segmentation using Bezier Curvature as Spatio-Temporal Feature by Triplet Learning
- Author(s)
- 장승민
- Advisor
- 손경아
- Department
- 일반대학원 인공지능학과
- Publisher
- The Graduate School, Ajou University
- Publication Year
- 2022-02
- Language
- eng
- Keyword
- Action Segmentation
- Alternative Abstract
- With the development of recording technologies, the demand for video-based techniques is increasing. Despite the success in action segmentation which classifies short trimmed video, it remains a challenge to use long untrimmed videos. Action segmentation is the field of detecting and temporally locating segments in a video. Although previous approaches have shown an outstanding architectural development, the feature extractor remains. Recent approaches require additional temporal information such as action boundary information, which is difficult to obtain in real-world assumptions. This is because temporal features are not as well developed as spatial features. In this thesis, we propose a new feature synthesis framework, called a Temporal Curvature Feature (TCF). This framework consists of two stages: (a) framewise embedding and (b) curvature synthesis. In framewise embedding stage, we use a triplet network to map a video into T points. which are based on each action label corresponding to the frame. In curvature synthesis stage, we approximate a curve with these embedding points and synthesize the curvatures from the curve. These curvatures are used to enhance the temporal information of data through a framewise residual operation. The outputs have the same shape as the old shape and are used as the new input to bring out the potential from various models. To validate the effectiveness of our approach, curvatures are plugged into three action segmentation datasets, i.e., GTEA, 50Salads, and Breakfast, and we use the new input to train the previous state-of-the-art models: MS-TCN, MS-TCN2, ASRF, and ASFormer. The result tables show the overall increases in the performances. In particular, the F1 scores show the effectiveness of the approach in solving segmentation problem. Finally, the figures demonstrate that the curvature helps the model to better understand the temporal information.
- Fulltext
- Appears in Collections:
- Graduate School of Ajou University > Department of Artificial Intelligence > 3. Theses(Master)
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.
-
- License
- STATISTICS
- Total Visit :4,041,756
- Total Download :1,962
- Today View :671
