Detailed Information

Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

Binary Dense SIFT Flow Based Position-Information Added Two-Stream CNN for Pedestrian Action Recognitionopen access

Authors
Park, Sang KyooChung, Jun HoPae, Dong SungLim, Myo Taeg
Issue Date
10월-2022
Publisher
MDPI
Keywords
pedestrian-action recognition; two-stream convolutional neural network (CNN); binary descriptor dense scale-invariant feature transform (SIFT) flow; position-information feature
Citation
APPLIED SCIENCES-BASEL, v.12, no.20
Indexed
SCIE
SCOPUS
Journal Title
APPLIED SCIENCES-BASEL
Volume
12
Number
20
URI
https://scholar.korea.ac.kr/handle/2021.sw.korea/145498
DOI
10.3390/app122010445
ISSN
2076-3417
Abstract
Pedestrian behavior recognition in the driving environment is an important technology to prevent pedestrian accidents by predicting the next movement. It is necessary to recognize current pedestrian behavior to predict future pedestrian behavior. However, many studies have recognized human visible characteristics such as face, body parts or clothes, but few have recognized pedestrian behavior. It is challenging to recognize pedestrian behavior in the driving environment due to the changes in the camera field of view due to the illumination conditions in outdoor environments and vehicle movement. In this paper, to predict pedestrian behavior, we introduce a position-information added two-stream convolutional neural network (CNN) with multi task learning that is robust to the limited conditions of the outdoor driving environment. The conventional two-stream CNN is the most widely used model for human-action recognition. However, the conventional two-stream CNN based on optical flow has limitations regarding pedestrian behavior recognition in a moving vehicle because of the assumptions of brightness constancy and piecewise smoothness. To solve this problem for a moving vehicle, the binary descriptor dense scale-invariant feature transform (SIFT) flow, a feature-based matching algorithm, is robust in moving-pedestrian behavior recognition, such as walking and standing, in a moving vehicle. However, recognizing cross attributes, such as crossing or not crossing the street, is challenging using the binary descriptor dense SIFT flow because people who cross the road or not act the same walking action, but their location on the image is different. Therefore, pedestrian position information should be added to the conventional binary descriptor dense SIFT flow two-stream CNN. Thus, learning biased toward action attributes is evenly learned across action and cross attributes. In addition, YOLO detection and the Siamese tracker are used instead of the ground-truth boundary box to prove the robustness in the action- and cross-attribute recognition from a moving vehicle. The JAAD and PIE datasets were used for training, and only the JAAD dataset was used as a testing dataset for comparison with other state-of-the-art research on multitask and single-task learning.
Files in This Item
There are no files associated with this item.
Appears in
Collections
College of Engineering > School of Electrical Engineering > 1. Journal Articles

qrcode

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Researcher Lim, Myo taeg photo

Lim, Myo taeg
공과대학 (전기전자공학부)
Read more

Altmetrics

Total Views & Downloads

BROWSE