Are we producing too much data? That may depend on your definition of “too much,” but it is certainly true that data is being produced at ever-increasing rates.
It has been calculated that the world's per-capita capacity to store information has roughly doubled every 40 months since the 1980s and, as of 2012, 2.5 quintillion bytes of data were being created every day.
Hong Kong is also experiencing exponential growth in data usage, especially in mobile data traffic. According to First Quarter Economic Report 2012 by HKSAR, total mobile data usage per month soared from a meagre 0.1 terabytes in 2003 to 69 terabytes in 2008 and further to 2,792 terabytes in 2011.
With the continuous growth of smart devices, including smartphones and tablets, it is expected that mobile data usage in Hong Kong will continue to rise1.
These days, big data is generally understood to mean high-volume, high-velocity, high-variety, and high-value information assets that require new forms of processing to enable enhanced decision making, insight discovery, and process optimization.
That “4-Vs” definition will be even more appropriate as we begin to ride the next big wave of data – machine data. There will almost certainly be a need for a further evolution in processing techniques to handle the unprecedented volumes that machine-data will create.
Big Data Tomorrow
And just how big will that next wave be? We estimate that machine-driven data will be an order of magnitude greater than the wave of human-driven data we’ve been generating over the last two decades.
IDC forecasts that machine-driven data will increase to 42% of all data by 2020, up from 11% in 20052. And, just as significantly, the third V – variety – will continue to expand.
To give you a hint of things to come, the latest generation of gene-sequencing technology creates data files of up to 4 terabytes. The new Boeing 747-8 generates nearly 2,000 terabytes of data in 24-hours of flight time. On the mobile frontier, other than basic dailing services, mobile data services are popular among consumers as well.
In June 2013, mobile data usage in Hong Kong recorded a remarkable surge to 9,422 terabytes (i.e. 9,422,056 gigabytes), or an average of 843 megabytes per 2.5G/3G/4G mobile user. This represents a growth of 1.62 times in data usage over the same period in 2012 and 4.3 times over that of 20113.
In fact, machine-to-machine (M2M) data is on the rise in almost every field, commercial or otherwise. Embedded digital sensors are now part of nearly every large system design; not just in new aircrafts but also in new buildings and automobiles.
In addition, previously stand-alone devices are becoming increasingly communicative so, for example, a runner’s heart rate monitor is connected to a smartphone via Bluetooth and could transmit data to a personal medical database, which in turn gets anonymized and combined with the data of thousands of others runners for large-scale analysis.
The reality is that the term “information-driven” is becoming a cliché in many industries, like financial institutions, healthcare, and pharmaceutical companies.
Business needs have automated data analysis – about customers, systems, materials, and performance – not only to execute well now and be more competitive in the future, but also to deliver societal improvements.
The social dividend of big data is easy to spot once you start to look. In call centers, for example, some companies are starting to apply “sentiment analysis” – based on big-data analysis of millions of customer calls and outcomes – to detect sentiment by tone of voice in real-time, and to quickly escalate dissatisfied customer calls for resolution and retention.
Recent big-data innovations in the field of video analysis have enabled automated analysis of practice evacuations in buildings, which identify and solve traffic flow issues and ultimately help architects to design safer buildings.
In healthcare big pharma – where they develop life-saving therapeutic drugs – is a prime example of big data in action, both at the level of research and in field trials.
Obviously, accumulating data does not do much good – for business or society – if an organization does not have the right technology to deal and capitalize that data. Turning data into useful information is what drives innovation or, as what we in our company say, “data sees what is happening but insight sees what is possible.”
Clearly our capacity to generate, transmit, and process digital information – increasingly in the M2M sphere – is not slowing down anytime soon. And it will only become “too much” when we do not have the technology to deal with it.
2 IDC, The Digital Universe in 2020 (2005-2020), December 2012
The views expressed in this column are the author's own and do not necessarily reflect this publication's view, and this article is not edited by Hongkong Business. The author was not remunerated for this article.
Do you know more about this story? Contact us anonymously through this link.
Andrew Sampson is Hitachi Data Systems’ general manager, Hong Kong and Macau, and director of managed and cloud services, Asia Pacific. A veteran of the local Information Technology and Telecommunications industries in Hong Kong, Andrew has held various sales, marketing and general management roles during his over 15-year stint in the territory.