Rewritely is a revolutionary platform that helps you to turns the World Weird Web into useful and consumable information with minimum efforts.

The current web is so fragmented and we can't just wait for the Semantic Web to become a reality. Instead of betting on so-called APIs like this, this, and this, Rewritely help you to tackle the problems today, not tomorrow, using your familiar technologies such as JavaScript, jQuery and REST API.

How does it work?

An example is perhaps the best way to show how does it work.

1. Create a task

The first thing you always need to do is to create a task. A task is the atomic unit of work executed by our server which is either all success, or fail. For example, the following task returns the number of images embedded in a given web page.

$ curl -H "Content-Type: application/javascript" -X POST https://api.rewritely.com/tasks  -d \
  '
    REWRITELY.output.num_of_images = function() {
        return $("img").size();
    };    
  '    
  
HTTP/1.1 201 Created
Location: https://api.rewritely.com/tasks/84bac155-6dca-11e1-8432-080027d25c16

You can issue GET. UPDATE or DELETE requests to the task like a normal resource.

2. Create a job

Now you have the task created. A task is rather meaningless without being run, so you need to create a job and execute it.

$ curl -H "Content-Type: application/json" -X POST https://api.rewritely.com/jobs -d \
  '
    {
      "task": "https://api.rewritely.com/tasks/84bac155-6dca-11e1-8432-080027d25c16",
      "url": "http://www.apple.com",
      "callback": false,
      "retry": 0,
      "parallel": 2,
      "depth": -1,
      "crawlDelay": 1,
      "logLevel": "INFO"
    }
  '   
 
HTTP/1.1 201 Created
Location: https://api.rewritely.com/jobs/0b984b48-6e07-11e1-8432-080027d25c16
Content-Type: application/json

[
  {"http://www.apple.com": {"num_of_images": 24}}
]

You can issue GET requests using the job url returned to replay the results in the future. (We will cache the remote response in our system according to their Cache-Control response headers specified)

3. Chain the tasks to make something interesting

The strength of Rewritely is you can chain as many tasks (limited by the parameter depth) as you wanted to create something really interesting and useful. For example, the following task extract the urls from the Google's search results and pass them the task we have just created above.

$ curl -H "Content-Type: application/javascript" -X POST https://api.rewritely.com/tasks  -d \
  '
    REWRITELY.chain.searchResults = function() {
      var results = [];
      
      $("h3.r a").each(function() {
        var url = $(this).attr("href");
      
        results.push({
          "url" : url,
          "task" : "https://api.rewritely.com/tasks/84bac155-6dca-11e1-8432-080027d25c16",
        });
      });
      
      return results;
    };
  '
       
HTTP/1.1 201 Created
Location: https://api.rewritely.com/tasks/85d06c0a-6e0a-11e1-8432-080027d25c16

We then execute the task against a real Google's search result (We use it for demo only, read their terms carefully if you really want to use it), and now the job API is equivalent to: Find the number of images embedded in each url of the Google's search result.

$ curl -H "Content-Type: application/javascript" -X POST https://api.rewritely.com/jobs -d \
  '
    {
      "task": "https://api.rewritely.com/tasks/85d06c0a-6e0a-11e1-8432-080027d25c16",
      "url": "http://www.google.com/search?q=apple",
      "callback": false,
      "retry": 0,
      "parallel": 4,
      "depth": -1,
      "crawlDelay": 1,
      "logLevel": "INFO"
    }
  '  
    
HTTP/1.1 201 Created
Location: https://api.rewritely.com/jobs/ed51556d-6e0a-11e1-8432-080027d25c16
Content-Type: application/json

[
  {"http://www.apple.com": {"num_of_images": 24}},
  {"http://www.apple.com/hk/en/": {"num_of_images": 22}},
  {"http://www.apple.com/ipad/": {"num_of_images": 17}},
  {"http://www.apple.com/iphone/": {"num_of_images": 28}},
...

More examples are coming, I'll leave the rest up to your imagination at the moment.

Our platform is currently in private use. To know more or interested in collaboration? Let us know!