In my continued playing around with meetup data I wanted to plot the number of members who join the Neo4j group over time. I wanted to plot the actual count alongside a rolling average for which I created the following data frame:
Hadoop isn't the only big data tool out there. Check out this list of big data tools available.
The marketing department of software vendors have done a good job making Big Data go mainstream, whatever that means. The promise of we can achieve anything if we make use of Big Data; business insight and beating our competitions to submission. Yet, there is no well-publicised Big Data successful implementation. The question is: why not?
With MapReduce, companies no longer need to delete old logs that are ripe with insights—or dump them onto unmanageable tape storage—before they’ve had a chance to analyze them. Today, the Apache Hadoop project is the most widely used implementation of MapReduce.
I realized something odd about accessing protected properties the other day. It's possible in PHP to access protected properties from other objects, as long as they are from the same class, as illustrated here:
Due to the obstacles presented by large scale data management, the goal for developers and data scientists is two-fold: first, systems must be created to handle large scale data, and two, business intelligence and insights should be acquired from analysis of the data.
This week we're talking to Sander Mak, Senior Software Engineer at Luminis Technologies, JavaOne Rockstar, and featured author in DZone's 2014 Guide to Big Data.
Monday was the official launch of DZone's 2014 Guide to Big Data and to kick off this event, we gathered a panel of six Big Data specialists to participate in DZone's first Twitter Q&A, which appeared under the #DZBigData hashtag.
Two years ago, Phil Shelley, the former Chief Technology Officer of Sears Holdings and CEO of Metascale, predicted the death of ETL. Now, two years later. Let’s take a look at what has happened to ETL.
A brief review that summarizes the few changes in the Magic Quadrant for Data Integration Tools between 2013 and 2014.
This weekend I discovered that the New York Times has a pretty deep developer API. I thought I'd try to build a little experiment. What if we could use the API to map the number of times a keyword appeared in headlines over time?
DZone's 2014 Guide to Big Data was produced to help you discover emerging information about the Big Data landscape and learn about how the shifting needs of data scientists and developers are influencing new tools and technologies.
Data collection should be synthesized into meaningful events. Getting users addicted to a platform by the quality and frequency of decisions versus encouraging them to spin the wheel to see what happens and becoming a 5th limb.
How do you measure the "sortedness" of a list? Here, I propose another measure for sortedness. The procedure is to sum the difference between the position of each element in the sorted list, x, and where it ends up in the unsorted list, f(x). We divide by the square of the length of the list and multiply by two, because this gives us a nice number between 0 and 1. Subtracting from 1 makes it range from 0, for completely unsorted, to 1, for completely sorted.
Do not take for granted that if your application works with Java version X it will automatically and flawlessly work with any Java version Y > X.
I’ve been playing around with some time series data in R and since there’s a bit of variation between consecutive points I wanted to smooth the data out by calculating the moving average.
I’ve been learning how to do moving averages in R and having done that calculation I wanted to plot these variables on a line chart using ggplot.
This week we're talking to Chanwit Kaewkasi, Assistant Professor at the Suranaree University of Technology’s School of Computer Engineering in Thailand, co-developer of a series of low-cost Big Data clusters, and featured author in DZone's upcoming 2014 Guide to Big Data.
By the time you have developed something and fixed any issues with it, your version is simply not going to be as tested as a ready built component that is used by thousands of people.
In anticipation of our 2014 Guide to Big Data, we have arranged for a panel of experts - Kirk Borne and Jonathan Ellis, to answer your Big Data questions on Twitter on Monday, September 22, 2014. To participate, simply ask a question using the hashtag #DZBigData.
The DB2 CONCAT function will combine two separate expressions to form a single string expression. It can leverage database fields, or explicitly defined strings as one or both expression when concatenating the values together.
A “quantile forecast” is a quantile of the forecast distribution. Still assuming normality, we could generate the forecast quantiles from 1% to 99% in R using...
Computer Security breaches can end up costing even the average small business up to $200,000
This article represents key aspects of starting up a Big Data practice in your organization. Currently, I have started working in the same area and this blog is the result of my research. Hope you find it useful.
The article discusses what stream processing is, how it fits into a big data architecture with Hadoop and a data warehouse (DWH), when stream processing makes sense, and what technologies and products you can choose from. Comparison of open source and proprietary stream processing / streaming analytics alternatives: Apache Storm, Spark, IBM InfoSphere Streams, TIBCO StreamBase, Software AG's Apama, etc.