How Does S3 Storage Work?

By March 3, 2020Cloud Backup

Backup specialist, Stephen Seagrave, gives us his analysis of S3 object storage including how it all works as well as what the different use cases are.

What is S3 object storage?

S3 is the API language that the storage uses to communicate with applications and store the data. It allows for the storage of trillions of objects and does not hit the max file limitations of traditional file systems. It has one scalable storage pool that scales uniformly and has very high redundancy and efficiency. The ability to scale to very large data sets is what allows S3 storage to bring down the cost of storage.

Any application that is compatible with Amazon’s S3 storage is fully compatible with Cloudian S3 as well. S3-enabled applications available from vendors such as Veeam, Veritas, Rubrik, Commvault, Pure Storage, Adobe, Splunk, Hadoop, Evertz and others can avail of Digital Planet’s S3 storage as a service with ease.

Depending on the level of integration from the application vendor, some or all of the S3 options can be used, such as Veeam’s new V10 object lock integration that allows for a second offsite copy of the backup data to be fully locked for a period of time. To the point that the data can in no way be deleted, even by the root admin of the storage.

This provides for full immutability configured directly from the backup application.

 

What is immutable object storage?

In the simplest of terms, immutability means data CANNOT be edited, changed or deleted. It doesn’t mean that it’s difficult to do or unlikely to happen. Nor does it mean that the file system is bespoke or developed in-house so it does not attract viruses – much like when people believed that Macs couldn’t have a virus simply because there were so few in use, that they were not worth attacking.

By its very definition, immutable data is fully protected from encryption and deletion.

 

Why is S3 immutable?

So, what makes S3 immutable and why is it different to other solutions?

Some solutions rely on air gapping – this means removing the storage from the network on a schedule or copy data to tape and sending it offsite.

Others call their filesystem ‘immutable’ and some lock their backups or archives with an extra slider or tick box or use legal hold as a crutch.

All of these are simply work arounds and can be undone. As soon as the data is available on the network again, it can be deleted. A disgruntled employee can log a request with your media vault to have your tapes destroyed. The backup admin can delete the backups. The PACs/RIS admin can remove the archive location and delete the images.

cloudian veeam s3 object storage backup solution

These solutions at best make it difficult or unlikely that your data can be deleted. But it is NOT immutable. Cloudian and Evros’s S3 storage is certified to SEC Rule 17a-4(f) standards for retaining records in compliance with the recording and non-rewritable, non-erasable storage of electric records. By locking the block of data at the storage level with an API call that cannot be reversed.

 

What is object storage used for?

Evros is currently the only Irish provider to offer S3 object lock as both a service from its own cloud, and for on-prem use. Once enabled, object lock CANNOT be disabled and the data is stored for the duration of the lock without fail.

The Cloudian Array is a multi-node, multi-site redundant storage array using erasure coding to provide 13 nines of availability.  When used with Veeam V10, the storage is supported by Veeam in compliance mode.

Compliance – this is the strictest mode, intended for regulatory compliance. Deletes of WORM-protected objects are disallowed even for the root account. Once in Compliance mode, retention configuration cannot be relaxed; in particular, the retention time period cannot be decreased and the mode cannot be changed.

However, the S3 storage service can be used by many applications.

cloudian s3 storage

  • Such as for offsite backup long-term retention to replace tape for Vendors including: Veeam, Veritas, Rubrik, Commvault etc.
  • Data archiving, for large medical data sets created by Cardiology PACS and Radiology PACS.
  • Cloud Snap, Pure Storage SAN Snapshot offload.
  • Nutanix Tiering or extended capacity.
  • Media Archive, for scalable and cost-effective capacity to accommodate formats such as 4K, 8K, and VR/360. To enable CCTV and Toll operators to store large data sets.
  • Big Data, such as Splunk and Hadoop all need large scale, affordable and elastic storage.

cloudian backup solution

Use cases for S3 object storage

The use cases for this storage are vast. however, most simply put, if you want secure S3 object storage to replace tape in your backup solution, or to enable long-term archiving of your S3 enabled application data, then Evros’s S3 can offer a highly redundant, highly scalable storage service at a cost much lower than ever offered by Evros, and the option to have your very own S3 storage array in your own site.

As a solution architect in the field of data protection, my personal opinion is that the game has changed. We no longer have to rely on 30-year-old tape technology or pay through the nose for overpriced deduplication storage. There is no need for bespoke backup vendor hardware appliances that promise the world but fall short in real world use. The roll out of vastly scalable and reliable storage is here – and I can’t wait!

 

Find out more

As partners with Cloudian, Evros can offer S3 object storage in the form of S3 storage as a service through Digital Planet. Evros is also offering customers the option to deploy their own S3 storage on their primary or secondary site – depending on what your needs are.

Or get in touch with Evros to find out more about the complete range of cloud and security services available.