10 Powerful Facts About Big Data
Source: Jeff Bertolucci
More than a buzzword
Big data, however you define it, has been praised and vilified. It's many things to many people: a boon to scientists and retailers, but also an enabling technology for a host of privacy and security threats.
Whether savior or scam -- or maybe even a mixture of the two -- big data remains a popular topic among pundits, prognosticators, marketers, and security buffs. Its unofficial definition is evolving as well. So what is it? Wikipedia's description is a good start: "any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications."
But the challenges of managing massive volumes of varied data sets arriving at high velocities -- the classic 3V's definition -- are changing as the number of data-sharing devices grows exponentially. This hardware, collectively known as the Internet of Things (IoT), includes machine sensors and consumer-oriented devices such as connected thermostats, light bulbs, refrigerators, and wearable health monitors. IDC predicts the IoT market will soar in the coming years -- from 9.1 billion installed units at the end of 2013 to 28.1 billion by 2020.
Organizations see a potential boon in actionable insights derived from big data, not only to sell more widgets and services, but also to better manage healthcare, stop the flow of counterfeit drugs, track terrorists, and maybe even track your phone calls. Hence it's a given that big data isn't inherently good or evil. It's how you use it that counts.
The irony of big data is that despite its potential to enhance the human experience, it's often difficult to collect, filter, analyze, and interpret to gain those cherished insights. This slideshow examines the challenges and capabilities of big data. The facts and figures may surprise you. What to expect? Well, the future appears bright for Hadoop, the leading big data platform. And data scientists and related big data gurus should be gainfully (and lucratively) employed for years to come.
Industry insiders have predicted the buzz term "big data" will fade away. "It is all just data, after all. Big data and all the predictions for this space will collapse into 'data management' by the analysts and all those following, including a lot of the 'big' vendors," wrote Hortonworks president Herb Cunitz in a December 2012 blog.
Cunitz may have prematurely predicted the demise of "big data," but he's spot on: It's all just data. Only the tools needed to manage it will change. Now dig into our slideshow and get a look at some revealing statistics and research.
How much data is ignored?
Most companies estimate they're analyzing a mere 12% of the data they have, according to a recent study by Forrester Research. Is this good or bad? Well, these firms might be missing out on data-driven insights hidden inside the 88% of data they're ignoring. Or perhaps they're wisely avoiding a resource-gobbling, boil-the-ocean strategy. A lack of analytics tools and "repressive" data silos are two reasons companies ignore a vast majority of their own data, says Forrester, as well as the simple fact that often it's hard to know which information is valuable and which is best left ignored.
Big data job growth
The big data craze is a boon for tech workers with a particular set of skills. According to Dice, a career site for tech and engineering professionals, demand is soaring for data mavens. Job postings for NoSQL experts were up 54% year over year, and those for "big data talent" rose 46%, the site reported in April. Similarly, postings for Hadoop and Python pros were up 43% and 16%, respectively. Impressive stats, certainly, but small potatoes compared with job postings for cyber-security specialists, which soared 162% year-over-year.
How big will big data get?
The digital universe will grow from 3.2 zettabytes today to 40 zettabytes in only six years. (One zettabyte is roughly a billion terabytes.) "When we look at the data volumes coming at us, it's mind-blowing," said Hortonworks CEO Rob Bearden in his keynote address at Hadoop Summit 2014 in San Jose, Calif. "The data volume in the enterprise is going to grow 50x year-over-year between now and 2020. I think the most important thing to recognize is that 85% of that data is coming from net-new data sources." And these sources, including mobile, social media, and web- and machine-generated data, present both a challenge and an opportunity for enterprises globally, Bearden noted.
Big data = big bucks
Big data jobs pay quite well. According to Salaries of Data Scientists, an April 2014 study from Burtch Works, the 2014 mean base salary for a staff data scientist is $120,000, and $160,000 for a manager. The estimates are based on interviews with more than 170 data scientists from a Burtch Works employment database. The pay scale is almost as good for the broader category of big data professionals, meaning those who "apply sophisticated quantitative skills to data-describing transactions, interactions, or other behaviors of people to derive insights and prescribe actions." In this category the 2013 median base salary for staff is $90,000; for managers, it's a cool $145,000.
Are big data pros ready for the IoT?
Most IT pros say they haven't started preparing for the Internet of Things -- even if they have. Spiceworks polled 440 IT professionals in April 2014 to get their take on the IoT and how they're preparing for it. Sixty-two percent of respondents were in North America and 38% in EMEA (Europe, the Middle East, and Africa). More than half (59%) of respondents said they're not taking specific steps to address the expected data deluge from sensors, cameras, and numerous other IoT devices. However, the survey also found that many IT pros are, in fact, preparing for the IoT by investing in infrastructure, security, applications, and analytics, and by expanding bandwidth.
Data scientists: still sexy
The eye-grabbing headline of an October 2012 article in the Harvard Business Review called the data science profession the "Sexiest Job of the 21st Century." That's debatable, but if "sexy" is synonymous with "in demand," data scientists haven't lost any of their mojo. According to Modis, a global IT staffing services provider, data scientists remain in "high demand but short supply," which translates into generous six-figure salaries for some PhDs with relevant big data experience.
Be afraid, data warehouse: Hadoop's in town
Should the data warehouse industry fear the rise of Hadoop? Embrace it? That question was posed to two Hadoop pioneers -- Doug Cutting of Cloudera and Arun Murthy of Hortonworks -- during a Q&A at Hadoop Summit 2014. While many enterprises are moving workloads from data warehouses to Hadoop, that's not happening en masse. But will it? "If you've got a lot of people no longer increasing the size of their data warehouse, but rather capping the size or potentially even decreasing their investment because they find they can do much of the processing as effectively and much more affordably in a Hadoop-based system, I think that's a threat," said Cutting.
Privacy fears won't stop big data
The cacophony of concerns rising from a seemingly endless series of privacy and security breaches isn't likely to thwart big data's advancement. The Economist reports in its June 2014 issue that "there is scant evidence that concern about privacy is causing a fundamental change in the way data are used and stored." Gartner analyst Carsten Casper tells the magazine that no "big privacy revolution" is brewing in the IT world. And while companies are asking more privacy-related questions, nine of 10 of those queries have to do with the location of data centers, Casper adds.
Big data drives software growth
The compound annual growth rate (CAGR) for the 2013-2018 worldwide software market will hover near 6%, research firm IDC predicts. But big data related categories, including collaborative applications and data access, analysis and delivery solutions, and structured data management software, will show a higher CAGR (around 9%) over that five-year period, says IDC.
A heightened interest in social media will help drive this growth. "This is complementary to the increased attention to big data and analytics solutions, which help enterprises understand and act on anticipated customer behavior and new insights into product reliability and maintenance," said IDC analyst Henry Morris in a statement.
Almost everything will be connected
The Internet of Things will include many strange and wondrous devices, many of which are new to the world of big data. That's why analysts at ABI Research predict more than 30 billion devices will be wirelessly connected by 2020. Health-related data collection will play a large role in the IoT, of course.
Here's a unique example: Microsoft, in conjunction with researchers from the University of Rochester (New York) and University of Southampton (UK), have designed a bra with sensors that detects the wearer's stress level by monitoring heart and skin activity, the BBC reported. Designed to see if wearable tech can help control stress-related overeating, the bra collects and sends data to a smartphone app to help the user control eating habits.
| }
|