Link Details

In the past I’ve often used in-memory data structures (vs on disk) in situations where allocating say 5-10MB of data in the local VM is much better than continually hitting a database. This has only worked because I’m not using very large data structures. My language classification codes uses about 8MB of memory stored in a TreeMap for it’s trained corpus based on Wikipedia. Recently, I’ve been playing with a much larger corpus for a new feature for Spinn3r.