Search Paper
  • Home
  • Login
  • Categories
  • Post URL
  • Academic Resources
  • Contact Us

 

Transformer Models for Text Summarization: A Comparative Study of BART, BERT and RoBERTa

google+
Views: 24                 

Author :  Daisy Aptovska and Vinayak Elangovan

Affiliation :  Penn State University Abington

Country :  USA

Category :  Artificial Intelligence

Volume, Issue, Month, Year :  17, 3, May, 2026

Abstract :


Text summarization refers to the task of condensing a document into a shorter version while preserving itskey information. Automatic text summarization (ATS), driven by advancements in natural language processing (NLP), has developed rapidly in recent years. ATS methods are commonly categorized by input type (such as single-document or multi-document summarization) and by output type (extractive, abstractive, and hybrid). This article presents a focused review of modern summarization techniques with an emphasis on transformer-based models and large language models (LLMs), specifically BERT, RoBERTa and BART. It examines their architectures, pretraining strategies, and their suitability for extractive and abstractive summarization tasks. The paper also discusses key challenges, including computational requirements, data limitations, and issues such as factual inconsistency in generated summaries, and highlights the strengths and limitations of encoder-only and encoder–decoder models.

Keyword :  Abstractive summarization, extractive summarization, Large Language Models, transformer models, BART, BERT, RoBERTa.

Journal/ Proceedings Name :  International Journal of Artificial Intelligence & Applications (IJAIA)

URL :  https://aircconline.com/ijaia/V17N3/17326ijaia02.pdf

User Name : alex
Posted 24-06-2026 on 21:26:10 AEDT



Related Research Work

  • S-ai-iot: A Sparse Artificial Intelligence Architecture With Hormonal Orchestration, Parsimonious Agent Activation, And Symbolic Memory For Adaptive, Secure, And Explainable Internet Of Things Systems
  • Enhancing Yolov8 For Infrared Object Detection Via Learnable Multi-scale Context And Attention
  • Procedural Generation In 2d Metroidvania Game With Answer Set Programming And Perlin Noise
  • Attention-driven Deep Image Prior For Radar Restoration With Known Psf

About Us | Post Cfp | Share URL Main | Share URL category | Post URL
All Rights Reserved @ Call for Papers - Conference & Journals