Link Details

Our sentiment analysis relies on various models to do feature extraction for the purposes of determining sentiment. These models are stored in the form of Python dictionaries and they’re big – about 40MB each, and there are 14 models in total. Doing the math, that’s 560MB of data that must be available to a process. Now, not all of the data is used for each request; only some of the keys in these dictionaries are used for a particular piece of text that needs to be analyzed.

Posted by dotCore  |   Nov 10 2012 / 10:28

Add your comment


Html tags not supported. Reply is editable for 5 minutes. Use [code lang="java|ruby|sql|css|xml"][/code] to post code snippets.

Recommended Links

Scala
Written by: Ryan Knight
Featured Refcardz: Top Refcardz:
  1. Apache Hadoop
  2. Play
  3. Akka
  4. Debugging JavaScript
  5. Design Patterns
  1. Apache Hadoop
  2. REST
  3. Java
  4. Git
  5. Java Performance
Connect with DZone