Detailed Information

Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

Binary dense sift flow based two stream CNN for human action recognition

Authors
Park, Sang KyooChung, Jun HoKang, Tae KooLim, Myo Taeg
Issue Date
11월-2021
Publisher
SPRINGER
Keywords
Action recognition; Binary dense SIFT flow; Binary descriptor; Two-Stream CNN
Citation
MULTIMEDIA TOOLS AND APPLICATIONS, v.80, no.28-29, pp.35697 - 35720
Indexed
SCIE
SCOPUS
Journal Title
MULTIMEDIA TOOLS AND APPLICATIONS
Volume
80
Number
28-29
Start Page
35697
End Page
35720
URI
https://scholar.korea.ac.kr/handle/2021.sw.korea/135831
DOI
10.1007/s11042-021-10795-2
ISSN
1380-7501
Abstract
Two-stream CNN is a widely-used network for human action recognition. Two-stream CNN consists of a spatial stream and a temporal stream. The spatial stream, through which the RGB image passes, extracts the shape features of human motion. The temporal stream, through which the optical flow images pass, extracts the sequence features of the listed motions. However, because of the constraints of the optical flow, such as brightness, constancy, and piecewise smoothness, there are limitations to the performance of two-stream CNN. One of the efficient methods to solve this problem is to expand the network model to a three-stream network, fuse it with LSTM, and add a modified pooling layer. This method improves the performance of the model but it increases the computational cost. Besides, the limitations of the optical flow are still present. In this paper, without extending the network model, a binary dense SIFT flow-based two-stream CNN is used instead of the optical flow. Unlike the optical flow, binary dense SIFT flow, which is a feature-based matching flow field is robust in brightness, constancy and piecewise smoothness. To evaluate the binary dense SIFT flow-based two-stream CNN, the UCF-101 dataset was selected for human action recognition. Furthermore, to evaluate the robustness of its brightness constancy and piecewise smoothness, a custom dataset was made up of classes that were extracted from UCF-101. Finally, the proposed method was compared with the state-of-the-art, which uses an optical flow-based two-stream CNN.
Files in This Item
There are no files associated with this item.
Appears in
Collections
College of Engineering > School of Electrical Engineering > 1. Journal Articles

qrcode

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Researcher Lim, Myo taeg photo

Lim, Myo taeg
공과대학 (전기전자공학부)
Read more

Altmetrics

Total Views & Downloads

BROWSE