Skip to content


MIT 500 6 13,294
202.4 thousand (month) Dec 28 2012 0.2.8(5 years ago)
1,518 7 99 GNU GPL 3
2020.1.16(3 years ago) Dec 14 2008 1.5 million (month)

newspaper is a Python package that allows developers to easily extract text, images, and videos from articles on the web.

It is designed to be fast, easy to use, and compatible with a wide variety of websites. It uses advanced algorithms to extract relevant information and metadata from articles, and it also supports several languages.

newspaper includes a http client or can ingest pre-scraped HTML documents.

html2text is a Python library that allows developers to convert HTML code into plain text. It is designed to be easy to use, and it provides several options to customize the output.

The package uses the python's built-in html.parser to parse the HTML and then convert it to plain text.

html2text also comes with a CLI tool that can convert HTML files to text:

Usage: html2text [filename [encoding]]

Option  Description
--version   Show program's version number and exit
-h, --help  Show this help message and exit
--ignore-links  Don't include any formatting for links
--escape-all    Escape all special characters. Output is less readable, but avoids corner case formatting issues.
--reference-links   Use reference links instead of links to create markdown
--mark-code Mark preformatted and code blocks with [code]...[/code]

Example Use

from newspaper import Article

# Create a new article object
article = Article('')

# Download the article

# Parse the article

# Print the article text

# Print the article title

# Print the article authors

# Print the article publication date
import html2text

h = html2text.HTML2Text()

# Ignore converting links from HTML
h.ignore_links = True
print h.handle("<p>Hello, <a href=''>world</a>!")
"Hello, world!"

print(h.handle("<p>Hello, <a href=''>world</a>!"))

"Hello, world!"

# Don't Ignore links anymore, I like links
h.ignore_links = False
print(h.handle("<p>Hello, <a href=''>world</a>!"))
"Hello, [world](!"

Alternatives / Similar