mxjvsjsonpath-ng
mxj is a Go library for working with JSON and XML data. It allows you to convert between JSON and XML, merge JSON and XML documents, and extract values from JSON and XML using a simple and intuitive API.
One of the main features of mxj is its ability to work with JSON and XML data in a struct-like manner, allowing you to access values using dot notation.
jsonpath-ng is a Python library for parsing and querying JSON data.
It is a powerful tool for extracting and manipulating data from JSON structures,
The library uses a syntax similar to XPath, which is a well-known language for querying and manipulating XML data, to query and extract data from JSON structures. This makes it familiar and intuitive for many developers who have worked with XML in the past. JSONPath implementation in Python.
JSONPath is a JSON query path language inspired by XPath (path language for querying XML/HTML). For more see the initial syntax proposal.
jsonpath-ng is implemented in pure Python and can be easily extended with additional python functions if needed.
Most commonly used JSONPath feature in web scraping is the recursive key lookup ($..key)
which is a convenient way to find specific data fields in large datasets.
Example Use
package main
import (
"fmt"
"github.com/clbanning/mxj"
)
func main() {
// Parse the JSON string
jsonData := []byte(`
{
"name": "John Doe",
"age": 30,
"address": {
"street": "Main St",
"city": "Anytown",
"state": "CA",
"zip": "12345"
},
"phones": [
"555-555-5555",
"555-555-5556"
]
}
`)
mv, err := mxj.NewMapJson(jsonData)
if err != nil {
fmt.Println("Error:", err)
return
}
// Extract the name
name, _ := mv.ValueForPath("name")
fmt.Println("name:", name) // "John Doe"
// Extract the city
city, _ := mv.ValueForPath("address.city")
fmt.Println("city:", city) // "Anytown"
// Extract all phone numbers
phones, _ := mv.ValuesForPath("phones")
for _, phone := range phones {
fmt.Println("phone:", phone)
}
// "555-555-5555"
// "555-555-5556"
}
from jsonpath_ng import jsonpath, parse
# A robust parser, not just a regex. (Makes powerful extensions possible; see below)
jsonpath_expr = parse('foo[*].baz')
# Extracting values is easy
[match.value for match in jsonpath_expr.find({'foo': [{'baz': 1}, {'baz': 2}]})]
[1, 2]
# Matches remember where they came from
[str(match.full_path) for match in jsonpath_expr.find({'foo': [{'baz': 1}, {'baz': 2}]})]
['foo.[0].baz', 'foo.[1].baz']
# And this can be useful for automatically providing ids for bits of data that do not have them (currently a global switch)
jsonpath.auto_id_field = 'id'
[match.value for match in parse('foo[*].id').find({'foo': [{'id': 'bizzle'}, {'baz': 3}]})]
['foo.bizzle', 'foo.[1]']
# A handy extension: named operators like `parent`
[match.value for match in parse('a.*.b.`parent`.c').find({'a': {'x': {'b': 1, 'c': 'number one'}, 'y': {'b': 2, 'c': 'number two'}}})]
['number two', 'number one']
# You can also build expressions directly quite easily
from jsonpath_ng.jsonpath import Fields
from jsonpath_ng.jsonpath import Slice
jsonpath_expr_direct = Fields('foo').child(Slice('*')).child(Fields('baz')) # This is equivalent