Big Data has become one of the most overused and misunderstood terms in the IT lexicon. Many a marketer has used big data to represent anything from social media content to Internet artifacts and anywhere in between. It’s this wide range of use that makes the term almost meaningless. From my perspective the simplest definition of “big data” is a massive collection of data that is so varied in type and format that it is impossible to analyze with conventional tools. Still not clear? Let’s look at a non-IT related example that you might be familiar with.
Most all of us are at some point in our day stuck in traffic. If you are like me when you find yourself stuck in traffic you want to know the cause. Frustratingly though most of the time you have no idea why the traffic jam started or what caused it to clear. As the individual driver you have limited knowledge of the circumstances. Now let’s say that instead of being an individual driver you are in a helicopter hovering over the crowded highway. From this vantage point you have ultimate visibility over the entire traffic jam. You see that a mile down the road a stalled car is in the center lane is causing the slowdown. With this information you can alert emergency services to the location of the stalled car and get the incident cleared. In this example all of us driving down the highway are single data points that have little value individually, however when we are in the helicopter we can look at all the vehicles as a collection of information and identify issues and trends. This is the aim of big data analysis, reviewing all those disparate data points as a whole to identify “outliers”. The stalled car is the outlier in our example. Make sense? Now back to the IT world.
All of our networks are generating mountains of information every day, every hour, every minute. Buried within this information are indicators of compromises, compliance violations, and vulnerabilities that need to be addressed. Sadly though, many organizations are not equipped with the resources required to consistently perform this big data analysis. For those that can, the value of their analysis is limited by to information about their own environment. Alert Logic’s takes big data analysis to the next level; check out this short video to learn more.