The growth in volumes of unstructured data could mean it’s time to run a technology infrastructure review
By George Ralph, Managing Director of RFA – How does a modern business organise its technology estate so that it is meeting the needs of the workforce, while making best use of infrastructure and the resources that it has available to it?
It used to be the case that most organisations stored their data on site, in NAS filers or similar, using block storage for high-performance input/output, which is great on-premise or across locations operated by the same organisation. However, block storage doesn’t offer practical mobility features because it depends on the use of a file system above the block level to organise data and provide functionality. Legacy file systems are built in a tree-like hierarchical structure which mean that performance suffers as the file system gets deeper. They are not built to meet many of the demands of the modern organisation and are certainly not the best choice for a mobile workforce.
These legacy file systems are hard to integrate into cloud environments and Gartner forecasts that more than 80 per cent of enterprise data will be stored in scale-out storage systems in enterprise and cloud data centres by 2021, up from 30 per cent in 2017. Unstructured data is rapidly outgrowing structured data as the main cause of data growth and could form as much as 80-90 per cent of an organisation’s overall data. In addition, many firms are using more than one cloud service provider for IaaS, and many SaaS solutions to meet their organisation’s demands. This means that IT departments have a tough job in ensuring that data is safe across all environments, that it is accessible from any location and that it is performing as well as it possibly can. Not an easy task when you consider the traditional way that unstructured data has been managed, in a tree-like hierarchical file system, which is not scalable and not built to withstand the demands of a mobile, or disparate workforce.
The Rightscale State of the Cloud 2018 report sees the main roles of an enterprise IT team as advising on which applications move to cloud, then managing the cost of cloud services. So with the increase in the growth of unstructured data and in the use of multiple cloud services, what is the best way forward for firms when considering future services.
Public cloud services have evolved are growing in popularity amongst enterprise organisations as well as smaller businesses, and the State of the Cloud report found that public cloud adoption amongst all respondents had increased from 89 per cent in 2017, to 92 per cent in 2018, but more interestingly that there was in increase in the number of organisations which were prioritising the use of public cloud, with 38 per cent saying the public cloud would be their top priority in 2018, compared to 29 per cent in 2017. In my opinion, public cloud services are still the best option for firms to house their public facing applications, portals and non-sensitive data, and deliver great flexibility and scalability for organisations.
At the same time, the use of private cloud is also increasing, and adoption rose from 72 per cent in 2017 to 75 per cent in 2018, showing that use of all cloud is still on the increase. Private cloud is still perceived as being the most secure cloud, with the flexibility to use a firm’s own infrastructure, or that of a private cloud provider, with the assurance that only their data is housed there, and that the cloud provider can utilise the latest technology for optimal performance, security and reliability. Many public cloud providers use object storage, a flat storage system using meta data and a unique ID to identify data as objects, rather than blocks or files, for its scalability, but object storage is also being utilised in the private cloud, or even in a firm’s own data centre, allowing organisations to leverage all the scalability, agility and self-service functionality of the public cloud, without the security concerns.
Clearly cloud services are the future, and there is no need for firms to make a choice between public clouds, private clouds, or SaaS applications when the new breed of connected multi-cloud service can deliver the best of both worlds safely and securely. A multi-cloud wrap around service can add a layer of security and data protection that secures connections in to the network and secures the connections between cloud services. A multi-cloud solution means that firms can truly link up their application data, regardless of where it is hosted and processed, and centrally manage resources, for ease and simplicity. From a security perspective, a multi-cloud solution can enable data encryption for all data at rest and in transit, which is imperative for many firms when complying with GDPR and specific financial services regulations such as MiFID II. Firms can freely pick and choose the best location for their data, depending on its use, importance, sensitivity and speed of access. File backups for example could be stored in the public cloud, while a full server image might be stored on-site or in a private cloud, to enable a speedy restore following a DR incident.
Whatever cloud services you choose, and according to the Rightscale survey firms are using on average, 4.8 different clouds, taking the time to fully understand the pros and cons will ensure that you are maximising your investment and servicing your organisation properly.
New distributed file systems in particular are designed to cover applications that require very low latency to operate efficiently. These include traditional databases, high-performance analytics, financial trading and general high-performance computing applications, such as life sciences and media/entertainment.
By providing data mobility, these new distributed file systems allow end users and IT organisations to take advantage of cheap compute in public cloud, while maintaining data consistency across geographic boundaries.