Detailed Information

Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

CATs++: Boosting Cost Aggregation with Convolutions and Transformers

Authors
Cho, S.Hong, S.Kim, S.
Issue Date
2022
Publisher
IEEE Computer Society
Keywords
Computer architecture; Correlation; cost aggregation; Costs; efficient transformer; Feature extraction; Semantic visual correspondence; Semantics; Task analysis; Transformers
Citation
IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.1 - 20
Indexed
SCIE
SCOPUS
Journal Title
IEEE Transactions on Pattern Analysis and Machine Intelligence
Start Page
1
End Page
20
URI
https://scholar.korea.ac.kr/handle/2021.sw.korea/147045
DOI
10.1109/TPAMI.2022.3218727
ISSN
0162-8828
Abstract
Cost aggregation is a process in image matching tasks that aims to disambiguate the noisy matching scores. Existing methods generally tackle this by hand-crafted or CNN-based methods, which either lack robustness to severe deformations or inherit the limitation of CNNs that fail to discriminate incorrect matches due to limited receptive fields and inadaptability. In this paper, we introduce Cost Aggregation with Transformers (CATs) to tackle this by exploring global consensus among initial correlation map with the help of some architectural designs that allow us to benefit from global receptive fields of self-attention mechanism. To this end, we include appearance affinity modeling, which helps to disambiguate the noisy initial correlation maps. Furthermore, we introduce some techniques, including multi-level aggregation to exploit rich semantics prevalent at different feature levels and swapping self-attention to obtain reciprocal matching scores to act as a regularization. Although CATs can attain competitive performance, it may face some limitations, <italic>i.e.</italic>, high computational costs, which may restrict its applicability only at limited resolution and hurt performance. To overcome this, we propose CATs++, an extension of CATs. Concretely, we introduce early convolutions prior to cost aggregation with a transformer to control the number of tokens and inject some convolutional inductive bias, then propose a novel transformer architecture for both efficient and effective cost aggregation, which results in apparent performance boost and cost reduction. With the reduced costs, we are able to compose our network with a hierarchical structure to process higher-resolution inputs. We show that the proposed method with these integrated outperforms the previous state-of-the-art methods by large margins. Codes and pretrained weights are available at: <uri>https://ku-cvlab.github.io/CATs-PlusPlus-Project-Page/</uri> IEEE
Files in This Item
There are no files associated with this item.
Appears in
Collections
Graduate School > Department of Computer Science and Engineering > 1. Journal Articles

qrcode

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Altmetrics

Total Views & Downloads

BROWSE