With the growing adoption of cloud-based IT infrastructures, the proliferation of mobile and IoT devices, and the rise of social media, companies of all sizes, across all industries are amassing huge quantities of data in differing variety, velocity, veracity and validity.
Shifting to the Cloud
For this reason, as more organizations consider shifting their entire data platforms to the cloud, IT leaders need a carefully charted approach that ensures all data is well managed, governed, cleansed and protected. Oftentimes, companies make the mistake or assume the misconception that when your data is in the cloud, your chosen cloud service provider assumes responsibility or accountability for that data. This couldn’t be further from the truth. When migrating to the cloud, companies need to take even more ownership of not only its data quality but also its compliance, protection, accessibility and trustworthiness.
Another misnomer about moving data to the cloud is that it all lives in a single, easily accessible place—but that’s not always the case. No doubt, data is a company’s most valuable asset, but it provides little insight when stored in organizational or technology silos. Data only becomes highly valuable when combined with existing customer, product, and partner data to drive informed decision making.
With that in mind, here are a few key concepts CIOs should keep in mind as they strategize their transition to the cloud:
How do you transition to the cloud at scale (moving beyond the POC stage)?
Many often get caught flat-footed thinking short term rather than long-term. A short-term approach often handicaps innovation and incurs increasing costs in the long term, and could include running a cloud POC using a single, custom-coded application, rather than using a suite of tool-built apps that can be quickly and easily applied across different teams and systems. The latter approach will give organizations a far better indication of what it will take for a data platform to perform successfully in the cloud.
How do you keep your environment on the cutting edge?
Innovation in the cloud is incredibly rapid, and CIOs need an approach that enables agility to continually leverage the latest advances in big data cloud technology over time. It’s important to think about building cloud systems in a portable way, so emerging technologies can be plugged in as they arise, and allow cloud environments to keep pace with the growing needs of a business. For example, advances in machine learning, AI, and database technologies are accelerating time-to-outcome, and CIOs need to ensure that data platforms can adapt with new capabilities and to new standards as the latest technologies come to market and the industry continues to evolve.
How do you maintain necessary resources & control maintenance costs?
As you begin your journey to the cloud, you need to think about your support skills and maintenance costs over time. Will your developers be able to maintain the cloud environment you built over the long term? What skills and knowledge will they need to maintain the cloud environments over time? An important takeaway is that you wouldn’t want to do the initial cloud migration with the top five percent of coders at an organization—you need an approach that will scale. The great thing about easy-to-use tools like Talend is that your data engineers can be up and running with very little training. However, CIOs should also be thinking about creating standardized best practices that can be reused over time, which will make cloud environments easier to maintain in the long run. It will also make the implementation more scalable because you’ll be less at risk of being bottlenecked by your own resources.
Make sure to have a separate strategy for your data science team.
Data scientists are incredibly data hungry and would be eager to run the type of highly complex machine learning algorithms that require time and cloud resources. While it’s important for those data activities to happen as they seek to understand metrics—and which are the most important to the business—this is notable for two reasons:
As a CIO, you should strongly consider having a sandbox environment for data scientists to run their tests in a contained environment, otherwise, it may impact performance and resources for everyone else.
Be sure to have a data retirement/elimination plan. You don’t want to create a single cloud data lake, containing all your data and has machine learning tools on top—that just creates a multiplication table and a data swamp. We’ve seen a healthcare research and consulting company increase data volumes by 50 percent within just 10 days because of the ML algorithms they were running on their data lake. That amount of data and resources isn’t sustainable—not to mention expensive. Thus, it’s important to retire data at a rate that makes sense for your business objectives and compliance regulations.
Make data governance and quality the cornerstones of your cloud data strategy.
It’s critical to take a business outcome-based approach to your cloud journey, and that requires for data governance and data quality strategies to sit at the core. I think we can all agree that if you’re sitting on a trove of data that is not qualified, governed, or trusted, then you’re working with garbage. You can’t possibly scale any kind of enterprise data strategy if you don’t have quality data at the start—but qualifying data can be a time-consuming process. In fact, current industry stats indicate organizations report spending more than 60 percent of their time qualifying or preparing data, leaving little time for actual analysis. Hence, a new wave of self-service tools has emerged that enable business users to access, merge, cleanse, and qualify data faster than ever before. Using self-service tools can help companies take a more ‘collaborative approach’ to data quality wherein business stakeholders who are most familiar with the information, can help organize, cleanse, govern and keep it up-to-date.
These are just a few of the key considerations IT leaders should bear in mind as they look to migrate their data to the cloud. What’s most important is to make sure you start your cloud strategy with your desired business outcome in mind and work your way back from there. In doing so IT leaders can better ensure that the end result will be an environment that eliminates silos and leads to real-time, trusted insights that can help anticipate customer needs and keep pace with a constantly changing, dynamic marketplace.