How to Build an AI-Powered Web Article Summarizer with Python 🐍

Hey DEV Community! 👋

Today, I want to share a super useful Python script I built that uses AI to automatically fetch and summarize any web article. It’s perfect for saving time and automating your daily reading list!

🛠️ What This Script Does:

1. Takes any article URL.
2. Extracts the main text content cleanly.
3. Uses an AI model to generate a bullet-point summary.

💻 The Code

python
import requests
from bs4 import BeautifulSoup
from transformers import pipeline

def summarize_article(url):
    print("⏳ Fetching article content...")
    # 1. Fetch the webpage
    response = requests.get(url)
    if response.status_code != 200:
        return "Failed to retrieve the article."

    # 2. Parse HTML and extract text
    soup = BeautifulSoup(response.text, 'html.parser')
    paragraphs = soup.find_all('p')
    article_text = ' '.join([p.get_text() for p in paragraphs])

    # 3. Initialize the AI summarization pipeline
    print("🤖 Generating summary using AI...")
    summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

    # Limit text length for the model
    input_text = article_text[:2000] 

    summary = summarizer(input_text, max_length=130, min_length=30, do_sample=False)
    return summary[0]['summary_text']

# Example Usage
if __name__ == "__main__":
    article_url = "https://dev.to/t/python" 
    result = summarize_article(article_url)
    print("\n📝 AI Summary:")
    print(result)
---
💼 **Need a custom Python automation tool or web scraper?** You can hire me directly on Fiverr: [Hire Me on Fiverr](https://www.fiverr.com/kabunji_uxui)

Enter fullscreen mode Exit fullscreen mode