In my previous article Cloud-first? Plan First! Why You Need A Cloud Governance Model And Where To Begin I introduced the concept of cloud governance models and explained why you need one. Today I will use our own example to demonstrate how you can put it into practice!
This post is second in the series of articles related to cloud governance and its implementation. You can find my previous entry here.
Let’s quickly recap the key points from last time (although I encourage you to read the whole article). The main issue was that your cloud governance model and deployments need to address three high-level components:
Be clear on what you want to achieve
You need to define the correct states and identified business objectives, mostly encompassing the following:
- Performance: defined by how your cloud adoption will translate to performance in terms of your business goals
- Cost optimization: i.e. optimization and control of costs related to cloud operations
- Compliance: how to meet requirements for your compliance regulations (be it internal or external)
- Security: how to keep your data and infrastructure safe and secure
- Risk management: what is your risk model and what risks are you trying to mitigate with your cloud deployment?
Based on these, you can apply technology solutions and choices in the following five key areas:
- Cost management
- Security baseline
- Identity baseline
- Resource consistency
- Deployment, auditing, and monitoring.
That’s about it in terms of the concept. But let’s make it practical. In this article, I want to share with you a simple example of the above approach that we often use in relation to cloud governance implementation. Let’s start with Azure resources for our projects and customers.
Business case – let’s define it!
In our case, we host all the infrastructure and resources in Azure. They might be hosted as internal resources, allocated either to the project or the customers as part of us being Tier 1 CSP partner for Microsoft.
As you can imagine, with 180+ employees and several hundreds of customers and projects, things might quickly get out of control in terms of cost and resource management.
However, as a data-driven company, we want to be able to track the full resource assignment which translates to several business objectives.
- In our model, we need to be able to assign each resource to a specific project or activity to make sure that we can track the overall cost assigned to delivery. In case of customer resources, we want to ensure that we are not overcharging our customers.
- We want to make sure that all data for completed projects is properly archived and not consuming resources.
- Additionally, we need to provide each Project Owner with clear visibility of the cost of resources consumed by a project.
- We need to be able to identify all resources related to a given project and customer to correctly manage access rights and be compliant with our legal obligations to the clients.
- We also need to be able to point out all the resources hosting data for a given customer in case we need to archive it, or indicate where such data are stored within our Azure resources.
(Both reasons above translate into a few risk related objectives.)
- We want to track all the project data to mitigate the risk of budget overrun and a high cost of consumption of Azure resources.
- We need to mitigate the risk of leaving customer data after project conflucdes. More specifically, scenarios where and we can’t find the data or shut down remaining project infrastructure.
These are the business objectives defined by our resource policies as part of our Azure Governance project. As you can see, we have not delved into any aspects of technical implementation or services in defining those objectives.
Now let’s translate all this into a tangible example.
Solution – let’s find it!
If you know Azure or other cloud services, you can probably come up with a solution of some kind right away. But for for those who are interested or those who have never done this before, let’s look into how we turned those business requirements into a practical implementation of cloud governance.
What were the requirements?
In our case, all resources need to be allocated to a project code that corresponds to one of our business or internal activities. The first requirement is to have a way to identify the following:
- resources assigned to a specific customer and projects
- who is responsible for them.
You can take a different approach here from the resource organization point of view:
- Split project resources across separate subscriptions for each project or customer
- In case of a single subscription hosting multiple projects, you can split it by resource groups.
In our case, all project codes are generated in Dynamics CRM when we start a project and assign a specific
- Customer, for whom we’re delivering the project
- Project Owner, who is responsible for delivering a project.
To be able to identify which resources are assigned to the right project we require that every resource group (and resources within it) are tagged with tags in Azure with three key elements:
- Project Code – simple text code coming out of our CRM as part of project initialization
- Project owner contact info – e-mail address of the Project Owner who is responsible for it
- Status of a project – active or closed.
Once you have this set, the next problem is to make sure that this is applied consistently to all resources you have to manage. Here are your options:
- Put automation in place to make sure that you have the correct tags on all resources
- Implement policies using Azure Policy and enforce them upon resource creation.
A note on Azure Policy
We had a small challenge when setting up our Azure Policy: some of our resources are created ad-hoc for testing purposes or quick and dirty PoC validation using Azure Portal.
Some time back, implementing a strict requirement for resources to be tagged with Azure Policy would mean that we blocked resource creation via portal on an ad-hoc basis. The portal was not always providing the tagging option for all the resources.
Also, our requirement was not that all resources had to be created with tags, but that we could eventually identify and assign all resources to the correct project and customer (via project).
How did we solve our case?
We settled on automation, with a simple but working solution:
- Our service desk gets a request to create project resources. Each request contains valid information, and resources include tags from the start
- In a case where resources appear outside of the process or need later modification:
- Automated job scans all the resources on a scheduled basis. If there is a new resource group or an existing one without proper tags assigned, it is reported to our IT/Consulting department
- Once the system finds a new resource, our IT works with Consulting to identify where it belongs. Then, they provide a proper project code
- Once the project code is assigned, automation keeps project data in sync with our CRM. For example, status and project owner contact information (e-mail).
Note for our Azure savvy friends – YES, we are going to update this process to Azure Policy!
What happens when a project ends?
We now have our resource correctly tagged and identified. The next stage is to make sure that it is correctly handled after the work on a project is done.
Again, here comes automation. Our business process in Dynamics CRM maintains project status. Based on this information, our automation task updates the project status tag on resources.
When a project is being closed, the Project Owner is notified that they need to archive it to avoid additional costs and issues with compliance in the process.
Why not automate the last phase and archive resources right away? It might be that the project is in a hyper-care period. In that case, we might need to restore it to an up-and-running state quickly. Alternatively, it might be that we have ongoing support or managed service for this customer, and we keep it for testing and validation purposes.
This knowledge stays with the Project Owner, and they make a decision based on the business case. With our governance process, we make sure that:
- All resources are correctly identified
- We can assign them to a specific customer and person responsible
- We can allocate the cost of these resources to a particular business activity.
Why automation? It is not the cutting edge of cloud tech!
One thing to notice is that in your solution you don’t always have to pursue the latest and greatest in the technology stack.
Azure-savvy people among our readers will quickly notice that fact. For some scenarios we wouldn’t use services or patterns to address them.
Additionally, here comes the iterative nature of the governance process: you have to assume that it is a cycle. You will have to review your business objectives and how you address them regularly, and apply changes and improvements.
It is the result of:
- changing business objectives and requirements
- evolving technology landscape and services
- changes in your processes and how your business operates.
Do not assume that what you have implemented will stay there forever – plan regular reviews and updates to your governance processes and tools.
Let’s see it – what is the outcome?
It sounds complicated. But, how the solution looks in practice and how it addresses our business objectives is much simpler than it may seem for business users.
We deploy the required resources in Azure to one of the subscriptions (depending on the model; it might be an internal subscription or one dedicated for the customer – CSP model).
Each subscription follows our strict naming convention and all resource groups also follow a naming convention defined in our policies.
In those subscriptions we have resource groups tagged with three pieces of information: Project Code, Project Owner and Project status. Here is an example:
Our internal automation keeps the information up to date.
Based on the billing details obtained from a billing API, we pull all the resources every month and allocate them to each business activity (project) – these details are blurred to keep project information confidential (sorry!).
The information connects to the relevant project. It is visible to each Project Owner. If a customer pays for those resources, it is also visible by the customer. The data is available through Excel and Power BI.
Upon the close of a project, the Project Owner receives a notification to decide what to do with the resources.
Why does it work?
This simple process meets our business objectives in the following ways:
- Cost optimization
- Each resource is assigned to our business activities (in case you’re wondering – we treat internal projects in the same way)
- When a project is closed, we notify the person responsible to mark resources to be closed. The system will follow up regularly to make sure there are no omitted resources
- Each Project Owner and customer have clear visibility of resource consumption through a smooth self-service flow.
- We can assign and quickly identify all resources related to every customer
- If required, we can promptly take action on any resource related to a given customer or project (take it down, report actions or access).
- Risk management
- We mitigate financial risk by providing clear cost assignment and visibility (of course this is not all that we do in this area 🙂 )
- We improve compliance and reduce data access risk through clear identification of the resources and their ownership.
What’s in it for you?
I hope that you can now translate Cloud Governance from concept and tools described in our first article into something more tangible and applicable to your organization.
In your case, it might be internal resources. You might allocate the cost to business units and departments, or a project. Another case could be that your source of projects is not a CRM but an EPM solution.
In all these cases if you think about the business objectives and what you want to achieve, then translate them into policy or governance statements, the last step is to implement those in a technology stack.
As an outcome, you get clear cloud resources with proper management, which makes cloud adoption easier.
In our next article, we will show you how you can support your Cloud Governance implementation with another critical element: business driven deployment and operations for automation and resource consistency. Stay tuned!
I encourage you also to join our upcoming webinar, during which my co-host Daniel and I will talk about how the combination of governance processes and DevOps allow to unleash the full potential of Azure cloud:
If you need help with this journey, get in touch with us. We will be happy to assist you.