Link Details

Link 969565 thumbnail
User 448255 avatar

By dotCore
via blog.scrapinghub.com
Submitted: May 16 2013 / 02:05

MongoDB was used early on at Scrapinghub to store scraped data because it’s convenient. Scraped data is represented as (possibly nested) records which can be serialized to JSON. The schema is not known ahead of time and may change from one job to the next. We need to support browsing, querying and downloading the stored data. This was very easy to implement using MongoDB (easier than the alternatives available a few years ago) and it worked well for some time.
  • 3
  • 0
  • 119
  • 61

Add your comment


Html tags not supported. Reply is editable for 5 minutes. Use [code lang="java|ruby|sql|css|xml"][/code] to post code snippets.

Voters For This Link (3)



Voters Against This Link (0)



    Reactive Programming with Akka
    Written by: Ryan Knight
    Featured Refcardz: Top Refcardz:
    1. Design Patterns
    2. OO JS
    3. Cont. Delivery
    4. Java Performance
    5. HTML5 Mobile
    1. Java Performance
    2. Node.js
    3. Debugging JavaScript
    4. Java
    5. Java Concurrency