HTTP debugging tools (aka REST API clients, API testing tools etc.) are GUI programs that allow to test sending of HTTP request.
These tools are useful for testing requests used in web scraping as provides a playground for reverse engineering website behavior and it's hidden APIs.
Here are some of the most common tools used in web scraping:
|postman||Very popular, code export|
|insomnia||similar to postman|
|httpie web||web client|
Step beyond HTTP debuggers there are tools that can intercept HTTP traffic. This can be used to reverse engineer phone app traffic through MITM traffic capture or as an alternative to browser devtools for inspecting web traffic.
|mitmproxy||web+terminal UI, easy to extend with Python, free|
|http-toolkit||easy setup, great documentation, freemium + 7.5usd/mo|
|proxyman||powerful interception features, only windows/macos, 69usd/license|
|fiddler everywhere||very popular, 12usd/mo|