embedvsparsel

MIT 68 6 2,072

4.2 thousand (month) Oct 26 2013 v4.4.11(a month ago)

1,101 8 41 BSD-3-Clause

Jul 26 2019 1.3 million (month) 1.9.1(3 months ago)

PHP library to get information from any web page (using oembed, opengraph, twitter-cards, scrapping the html, etc). It's compatible with any web service (youtube, vimeo, flickr, instagram, etc) and has adapters to some sites like (archive.org, github, facebook, etc).

parsel is a library for parsing HTML and XML using selectors, similar to beautifulsoup. It is built on top of the lxml library and allows for easy extraction of data from HTML and XML files using selectors, similar to how you would use CSS selectors in web development. It is a light-weight library which is specifically designed for web scraping and parsing, so it is more efficient and faster than beautifulsoup in some use cases.

Some of the key features of parsel include:

CSS selector & XPath selector support:
Two most common html parsing path languages are both supported in parsel. This allows selecting attributes, tags, text and complex matching rules that use regular expressions or XPath functions.
Modifying data:
parsel allows you to modify the contents of an element, remove elements or add new elements to a document.
Support for both HTML and XML:
parsel supports both HTML and XML documents and you can use the same selectors for both formats.

It is easy to use and less verbose than beautifulsoup, so it's quite popular among the developers who are working with Web scraping projects and parse data from large volume of web pages.

Highlights

css-selectorsxpath-selectors

Example Use

use Embed\Embed;

$embed = new Embed();

//Load any url:
$info = $embed->get('https://www.youtube.com/watch?v=PP1xn5wHtxE');

//Get content info

$info->title; //The page title
$info->description; //The page description
$info->url; //The canonical url
$info->keywords; //The page keywords

$info->image; //The thumbnail or main image

$info->code->html; //The code to embed the image, video, etc
$info->code->width; //The exact width of the embed code (if exists)
$info->code->height; //The exact height of the embed code (if exists)
$info->code->ratio; //The aspect ratio (width/height)

$info->authorName; //The resource author
$info->authorUrl; //The author url

$info->cms; //The cms used
$info->language; //The language of the page
$info->languages; //The alternative languages

$info->providerName; //The provider name of the page (Youtube, Twitter, Instagram, etc)
$info->providerUrl; //The provider url
$info->icon; //The big icon of the site
$info->favicon; //The favicon of the site (an .ico file or a png with up to 32x32px)

$info->publishedTime; //The published time of the resource
$info->license; //The license url of the resource
$info->feeds; //The RSS/Atom feeds

from parsel import Selector

# this is our HTML page:
html = """
<head>
  <title>Hello World!</title>
</head>
<body>
  <div id="product">
    <h1>Product Title</h1>
    <p>paragraph 1</p>
    <p>paragraph2</p>
    <span class="price">$10</span>
  </div>
</body>
"""

selector = Selector(html)

# we can use CSS selectors:
selector.css("#product .price::text").get()
"$10"

# or XPath:
selector.xpath('//span[@class="price"]').get()
"$10"

# or get all matching elements:
print(selector.css("#product p::text").getall())
["paragraph 1", "paragraph2"]

# parsel also comes with utility methods like regular expression parsing:
selector.xpath('//span[@class="price"]').re("\d+")
["10"]