Abstract

Text summarization plays a crucial role in natural language processing by condensing large volumes of textual information into concise and meaningful summaries. With the rapid growth of digital content, existing summarization approaches often struggle to balance contextual understanding and semantic relevance. This paper presents a PMI-driven BERT-based text summarization framework that integrates Pointwise Mutual Information (PMI) as a statistical pre-processing mechanism with a fine-tuned Conventional BERT model to enhance summary quality. PMI is employed to identify and rank semantically significant terms based on co-occurrence patterns, enabling effective keyword and phrase prioritization before summarization. The ranked textual representation is then processed using a summarization-specific decoder layer added on top of the BERT encoder to generate coherent and context-aware summaries. The proposed framework is evaluated on the CNN/Daily Mail dataset comprising over 300,000 news articles, using ROUGE-1, ROUGE-2, and ROUGE-L metrics for performance assessment. Experimental results demonstrate that the proposed method achieves ROUGE-1, ROUGE-2, and ROUGE-L scores of 46.9, 27.61, and 45.68 respectively, outperforming baseline models such as Seq2Seq, Seq2Sick, and Prefix-Tuning by an average margin of 2–3%. The experiments were conducted using Python with the PyTorch deep learning framework on a CPU-based environment. The results indicate that PMI-based pre-processing significantly improves contextual relevance and semantic consistency in generated summaries. This framework demonstrates robustness and scalability, making it suitable for large-scale document summarization tasks.

References

Citation

Copy

Copyright

Article metrics