Link Details

Link 969565 thumbnail
User 448255 avatar

By dotCore
via blog.scrapinghub.com
Submitted: May 16 2013 / 02:05

MongoDB was used early on at Scrapinghub to store scraped data because it’s convenient. Scraped data is represented as (possibly nested) records which can be serialized to JSON. The schema is not known ahead of time and may change from one job to the next. We need to support browsing, querying and downloading the stored data. This was very easy to implement using MongoDB (easier than the alternatives available a few years ago) and it worked well for some time.
  • 3
  • 0
  • 115
  • 61

Add your comment


Html tags not supported. Reply is editable for 5 minutes. Use [code lang="java|ruby|sql|css|xml"][/code] to post code snippets.

Voters For This Link (3)



Voters Against This Link (0)



    Debugging JavaScript
    Written by: Ashutosh Sharma
    Featured Refcardz: Top Refcardz:
    1. Design Patterns
    2. OO JS
    3. Cont. Delivery
    4. Java EE7
    5. HTML5 Mobile
    1. Java EE7
    2. Spring Annotations
    3. Git
    4. Java
    5. REST