Categories

Web Tools web scraping

API Provider

APISumo
APISumo

Developer links

Scraper API
Scraper API Bestseller

Scrape any website bypassing all rate limitations. Ability to simulate originating IP from any country. Fast and simple.

What is the Scraper API?

Web scraping involves complex routines like simulating a browser's behavior and dealing with rate limits of the destination site. Our Scraper API allows many features of simulating a desktop browser's behavior by giving you the ability of:

  • Setting the referrer header,
  • Selecting the originating country where we fire the HTTP request,
  • Adding cookies to the request header,
  • Setting HTTP Auth username and password.

Prompt API Scraper API is one of the versatile and affordable ways to collect any data from any web site.

How can we do it?

We utilize more than 1 million IP addresses from more than 100 countries from all over the world. We use anonymizer proxy servers, as well as our own infrastructure to change the IP addresses frequently and randomly change the HTTP request header information, so that any fingerprinting and IP based rate limiting functionality on the remote server is useless.

Privacy

Accept for obvious legal purposes, we do not store any information on our servers. We'll just work as a proxy and we will never inject any code in response nor interfare with your data in anyway. We'll just take the scraping request, fetch data from the remote server and pass it back to you in JSON format. That's it!

How can you use it?

Scraper API is useful for:

  • Web data extraction
  • Brand protection
  • Collecting stock market data
  • E-commerce
  • Competitor tracking

and many more

All that is done with a few lines of code on your side. Just check below for a cURL sample. Many more on the Live Demo section.

Scraper API Code

How secure is this service?

Prompt API is a legal entity in European Union and regulated by its laws. We never store any sensitive information on our servers and fully respect your privacy.

Is it free?

Scraper API is a paid service but we have a free plan that will be more than enough for you to test its functionalities and see if it works for your needs.

How does Prompt API's pricing work?

Our subscriptions are tiered. We offer a monthly subscription scheme and you decide which tier works for you the best. No strings attached.

Can I cancel/upgrade/downgrade at any time?

Sure you can. You can cancel your subscription or upgrade/downgrade yourself, anytime without any restrictions.

Ability to fetch any website
Ability to bypass rate limits
Supports both GET and POST requests
Ability to choose originating country
Ability to set Cookies
Supports HTTP Authentication

Ready to try it out?

We offer a free plan. No credit cards required!


or see documentation

Scraper API Reference

This API is organized around REST. Our API has predictable resource-oriented URLs, accepts form-encoded request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs.

Just Getting Started?

Check out our development quickstart guide.

Authentication

Scraper API uses API keys to authenticate requests. You can view and manage your API keys in the Accounts page.

Your API keys carry many privileges, so be sure to keep them secure! Do not share your secret API keys in publicly accessible areas such as GitHub, client-side code, and so forth.

All requests made to the API must hold a custom HTTP header named "apikey". Implementation differs with each programming language. Below are some samples.

All API requests must be made over HTTPS. Calls made over plain HTTP will fail. API requests without authentication will also fail.

Endpoints

Scraper Http Post

Parameters

url (required)

URL to scrape

Location: Query, Data Type: string

auth_password (optional)

Optional: HTTP Realm auth password

Location: Query, Data Type: string

auth_username (optional)

Optional: HTTP Realm auth username

Location: Query, Data Type: string

body (optional)

Optional: HTTP Body as a Dictionary Ex: {"name": "test", "email": "[email protected]"}

Location: Body, Data Type: string

cookie (optional)

Optional: URL Encoded cookie header.

Location: Query, Data Type: string

country (optional)

Optional: 2 character country code. If you wish to scrape from an IP address of a specific country, set it here.

Location: Query, Data Type: string

referer (optional)

Optional: HTTP referer header

Location: Query, Data Type: string

** A word enclosed with curly brackets "{ }" in the code means that it is a parameter and it should be replaced with your own values when executing. (also overwriting the curly brackets).
Returns

Below is a sample response from the endpoint


If you wish to play around interactively with real values and run code, see...

Scraper Http Get

Parameters

url (required)

URL to scrape

Location: Query, Data Type: string

auth_password (optional)

Optional: HTTP Realm auth password

Location: Query, Data Type: string

auth_username (optional)

Optional: HTTP Realm auth username

Location: Query, Data Type: string

cookie (optional)

Optional: URL Encoded cookie header.

Location: Query, Data Type: string

country (optional)

Optional: 2 character country code. If you wish to scrape from an IP address of a specific country, set it here.

Location: Query, Data Type: string

referer (optional)

Optional: HTTP referer header

Location: Query, Data Type: string

** A word enclosed with curly brackets "{ }" in the code means that it is a parameter and it should be replaced with your own values when executing. (also overwriting the curly brackets).
Returns

Below is a sample response from the endpoint


If you wish to play around interactively with real values and run code, see...

Rate Limiting

Each subscription has its own rate limit. When you become a member, you start by choosing a rate limit that suits your usage needs. Do not worry; You can upgrade or downgrade your plan at any time. For this reason, instead of starting with a larger plan that you do not need, we can offer you to upgrade your plan after you start with "free" or "gold plan" options and start using the API.

When you reach a rate limit (both daily and monthly), the service will stop responding and returning the HTTP 429 response status code (Too many requests) for each request with the following JSON string body text.

{
  "message": "API rate limit exceeded"
}
A reminder email will be sent to you when your API usage reaches both 80% and 90%, so that you can take immediate actions such as upgrading your plan in order to prevent your application using the API from being interrupted.

You can also programmatically check your rate limit yourself. As a result of each request made to the Prompt API, the following 4 fields provide you with all the necessary information within the HTTP Headers.

x-ratelimit-limit-month: Request limit per month
x-ratelimit-remaining-month: Request limit remaining this month
x-ratelimit-limit-day: Request limit per day
x-ratelimit-remaining-day: Request limit remaining today

You can contact our support unit if you need any assistance with your application regarding to handle the returned result by looking at the header information.

Error Codes

Prompt API uses standard HTTP response codes to indicate the success or failure of an API request. In general: Codes in the 2xx range indicate success. Codes in the 4xx range indicate a clientside error, which means that failed given the information provided (e.g., a missing parameter, unauthorized access etc.). Codes in the 5xx range indicate an error with Prompt API's servers (normally this should'nt happen at all).

If the response code is not 200, it means the operation failed somehow and you may need to take an action accordingly. You can check the response (which will be in JSON format) for a field called 'message' that briefly explains the error reported.

Status Code Explanation
400 - Bad Request The request was unacceptable, often due to missing a required parameter.
401 - Unauthorized No valid API key provided.
404 - Not Found The requested resource doesn't exist.
429 - Too many requests API request limit exceeded. See section Rate Limiting for more info.
5xx - Server Error We have failed to process your request. (You can contact us anytime)

You can always contact for support and ask for more assistance. We'll be glad to assist you with building your product.

See Also

View All