Rank over class: The untapped potential of ranking in natural language processing

Amir, Atapour-Abarghouei, A; Bonner, S; McGough, AS

doi:10.1109/BigData52589.2021.9671386

Rank over class: The untapped potential of ranking in natural language processing

Lookup NU author(s): Stephen Bonner ORCiD, Dr Stephen McGough

Downloads

Accepted version [.pdf]

Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

Abstract

Text classification has long been a staple withinNatural Language Processing (NLP) with applications spanningacross diverse areas such as sentiment analysis, recommendersystems and spam detection. With such a powerful solution,it is often tempting to use it as the go-to tool for all NLPproblems since when you are holding a hammer, everythinglooks like a nail. However, we argue here that many taskswhich are currently addressed using classification are in factbeing shoehorned into a classification mould and that if weinstead address them as a ranking problem, we not only improvethe model, but we achieve better performance. We propose anovel end-to-end ranking approach consisting of a Transformernetwork responsible for producing representations for a pairof text sequences, which are in turn passed into a contextaggregating network outputting ranking scores used to determinean ordering to the sequences based on some notion of relevance.We perform numerous experiments on publicly-available datasetsand investigate the applications of ranking in problems oftensolved using classification. In an experiment on a heavily-skewed sentiment analysis dataset, converting ranking results toclassification labels yields an approximately 22% improvementover state-of-the-art text classification, demonstrating the efficacyof text ranking over text classification in certain scenarios.