You can’t pick up a trade magazine these days without seeing an article about Big Data and how companies are anxious to leverage this data for actionable insight. The CTO of ThingsRUs Ltd announces that, thanks to a successful Big Data initiative, customers need to service their Thing only once a year instead of twice. The VP Engineering of BricksNMortar Bank declares in an interview that “Hadoop let us down.” Another Big Data Startup announces that it is going public, with a valuation of $2B, based on losses of $400M.
What’s all the fuss about? After all, there has always been data, some of it very big, and there have always been established database products to handle that data. Is Big Data just hype, the next PR bubble?
- The Internet of Things (IoT): Once upon a time, companies made things that people used (think: shoes, pens, cars), and people happily used them. Today, thanks to a set of technological advances in areas like microprocessors and wireless communications, things are interconnected sources of vast amounts of data. Consumers expect the thing to not only work but also to monitor itself, service itself and provide insights.
- The Digital Economy: Consumers expect an online experience that is instantaneous, streamlined, relevant, personalized. Providing this level of service requires rapid processing of massive amounts of data become about users from multiple sources. The Digital Economy is pulling companies into Big Data, ready or not.
Big data is getting bigger at an exponential rate, and traditional data technologies are not designed for this scenario. In my next blog post, I’ll outline from a pragmatic perspective how Big Data technologies differ from classic database technologies.