The Brontobyte and Big Data: The Future of Data Storage



Predicting the future of stored data is anything but exact, however, if you look at the trendlines you will see a convergence of sophisticated new emerging technologies and methodologies that provide a glimpse into the foreseeable future. This will make storage management more flexible, smarter and easier to manage. 

While today most people are familiar with the terabyte, a handful of years ago few in the public knew what a terabyte was, now it’s the norm. 

What comes after the terabyte? According to the International System of Units:

Petabyte

Exabyte

Zettabyte

Yottabyte

To give you a comparison, the Yottabyte is approximately 1 septillion bytes or equivalent to a quadrillion gigabytes (GB).

So where is the Brontobyte, why doesn’t it appear on this list? It has yet to become a standard unit for data storage. This means that the Bronto is only a theoretical quantity of data at the present time. Does it mean that you should not pay attention to it, heck no. 

Unit

Abbreviation

Capacity

b

1 or 0 (on or off)

B

8 bits

KB

1024 bytes

MB

1024 kilobytes

GB

1024 megabytes

TB

1024 gigabytes

PB

1024 terabytes

EB

1024 petabytes

ZB

1024 exabytes

YB

1024 zettabytes

As the need for storage capacity grows, there have been some pretty sizable advances in data storage technologies, with flash media (non-volatile memory), cloud storage, shingled magnetic recording, and solid-state drives. IT managers are currently developing plans and strategies on how to deal with the massive wave of data set to come and led by the AI and IoT revolution. The applications that run AI and IoT require even larger volumes of high-performance storage options. 

Currently, data is stored in the main memory by the host server, leaders are looking for an approach where processing can be done at the storage layer. Edge technology enabled by maximum performance SSDs move computers closer to the storage devices and is a common trend we are seeing become increasingly more mainstream. This is why many are looking at computational storage systems. This delivers in-storage compute technology, bringing high-performance computing power to storage devices. 

It will process and analyze the data where it is created and stored. This empowers an organization and its users to perform a variety of actions at the device level like filtering data before sending it back to the cloud to store in arrays.

This is why more IT managers are planning to transition to NVMe and container-based infrastructures, and have already started identifying applications that could benefit from enhanced efficiencies of computational storage. 

“Software-defined storage (SDS) is a storage architecture that separates storage software from its hardware. Unlike traditional network-attached storage (NAS) or storage area network (SAN) systems, SDS is generally designed to perform on any industry-standard or x86 system, removing the software’s dependence on proprietary hardware.”

SDS has a number of key benefits and many companies are switching to this model. It allows administrators and IT managers to consolidate and manage all of their data centrally from one place while delivering features and functionality. 

SDS is virtualization software, permitting you to control and manage physical storage in ‘virtual pools’. It also offers thin provisioning for maximum storage capacity and once the pool is generated, “virtual disks” are also created in the pool as raw LUNs and then sent to your host servers to store data. The best feature of SDS controller software is that it works without concern about the capacity or usability of the hardware. It takes what it finds and maximizes capacity.

All of these trends have come about due to the unrelenting growth of data and gargantuan cloud data centers, driven by hyperscalers and convergence. Increasingly, software is expected to be the way we control the storage of data in the future. We are entering an era where we will substantially consolidate resources, based on the protocol convergence that underlie the physical infrastructure of computing, providing a stronger foundation for operational structure. 

What we do know is that the need for more data center storage is rapidly increasing and as people adopt IoT and AI, the need will continue to grow. Every day people use Alexa, add new features to their Apple Watch, install a Google Nest, Ring and on and on. The current trend in big data currently seems to have data coming from three primary sources. 

Consumer data from social media giants like Meta, TikTok, YouTube, Pinterest, LinkedIn and others consume a large portion of the data storage pie. Likes, Shares, Video Uploads, Comments and Messages equate to space on a drive. Equally important are public utilities and online services such as Google’s suite of free services. Many SaaS business apps are using the freemium subscription model and will offer free storage, trials, online support with tutorials and communities, hoping their subscription fees will grow with the user’s business. 

A less-talked-about sector of data storage comes from industry using sensors on equipment and devices. Machine data is expected to grow progressively with the increasing adoption of AI and IoT. From health to military applications and from space to automotive, these devices and sensors will capture data and send it to other devices across the internet which will track and store it. 

The third type of data is Transactional, stemming from all of the transactions that happen each day in your life and mine. Credit cards, bank transactions, grocery purchases, online purchases, points, they all get stored, likely in an array, safe and secure. 

Want to know the difference between data storage types? (link when posted)

While the Brontobyte may be theoretical, you can bet that it will get standardized in the not-to-distant future. The dimensions are enormous. In decimal notation, 1 brontobyte is 1,000,000,000,000,000,000,000,000,000 (1027) bytes. In binary notation, 1 “brobibyte” is 1,237,940,039,285,380,274,899,124,224 (290) byte. Currently, the largest quantity of data is the zettabyte. Just to give you some scope, in 2016, worldwide traffic exceeded one zettabyte, bringing us into the zettabyte era. Imagine, one brontobyte is equivalent to 1,000,000 zettabytes. 

Can you guess who is the world’s largest data company? As you may have guessed, it’s Google. They store about 10 exabytes (that’s 10 billion GBs!). Facebook on the other hand has about 2.5 billion individual content pieces, 2.7 billion likes and 300 million photos. 

You would think that the company with the most data, would have the most servers. If you guessed Google, you would be mistaken, the answer is Amazon. It is difficult to pinpoint exactly how much data they store, but in 2015 it was estimated to be around 1,000,000,000 gigabytes of data across more than 1,400,000 servers.

But what is bigger than a brontobyte?

There is now talk in Big Data circles about the next theoretical level above the brontobyte: the gegobyte or geopbyte. The equivalent of 1,000 brontobytes. 

From laptops to servers, the primary unit of measure for data is in the vicinity of the terabyte, which evolves with storage technology capacities. Apparently, if you had all of the hard drives in the world, you still would not have enough capacity to store one brontobyte. Like much in tech, we do not know exactly what emerging technologies will bring or how soon they will arrive 

source

Leave a Reply

Your email address will not be published.