Accelerate migration from Pivotal Cloud Foundry to AWS Kubernetes

Modern cloud providers offer compelling opportunities to modernize IT infrastructure and gain operational efficiency through cloud-native design practices, DevOps, containers and more. However, moving to the cloud isn’t easy.

Those who join the cloud movement soon learn that there are a lot of moving parts to be managed and coordinated in different ways in order to achieve the desired results. This usually leads to the realization that although cloud provides the ability to implement a lot of interesting techniques, it is ultimately a tool, and not a solution. And as it is for every solution that leverages new technologies, careful planning and design of your future platform is the key to success.

In this blog we will detail the steps you can take to migrate from Pivotal Cloud Foundry (now called VMware Tanzu) to AWS in a matter of days using the PCF (Pivotal Cloud Foundry) Kubernetes Migration Starter Kit we developed.

So, what is a microservices platform, really? As a first step toward understanding, let’s say that we want to accomplish the goal of going mainstream by running containerized applications in the cloud. The most popular platform for doing this nowadays is Kubernetes, available in many flavors as a managed service offering from all major cloud providers. However, as we sometimes joke within our company, “One cannot simply put a container in Kubernetes and expect it to work”. As soon as you try to facilitate such a simple “container drop”, you will be faced with dozens of questions that need to be addressed, and answering them one by one slowly reveals the target state of your cloud environment before deploying a single application with business logic:

The interesting thing about a microservices platform is the fact that there is no “holy grail” technology-wise for every placeholder. Instead, the market offers multiple solutions which can be chosen based on workload specifics, required external integrations, adoption complexity, or sometimes even team preference. But the general architecture always stays the same, and can be implemented on top of any modern cloud, or even on-premise system. With a careful choice of technologies, it can even be made portable enough to fulfill the concept of a true multi-cloud solution, although this is rarely required in practice.

The process of cloud migration requires a lot of steps, including the creation and implementation of a platform blueprint with technology choices for each box, which can typically take anywhere from 1 to 3 months based on a particular toolset and the development team’s capacity. It is only after that, finally, that you can start to move applications over. And then you get a new client and have to repeat this process from scratch again. Not an exciting or efficient approach is it? Especially when you consider that we haven’t even factored in the plan for the platform to become a live system, which requires additional features like IaC and CI/CD pipelines, deployment orchestration, and more.

Surely there must be a better way? Thankfully, there is! Building a reusable reference implementation for our platform, which includes all the “plumbing”, makes it possible to focus on business applications from day 1, while opening up additional time to revisit, fine-tune, or even replace some components as required later on. The result is a vastly accelerated time-to-market with little to no disruption to the wider business ecosystem, and much simpler change management control.

On our quest to find a practical use-case for implementation of our microservices platform, we developed the idea of using it for particular migration scenarios; in this case, migrating from PCF to AWS EKS. As we have observed, many of our customers do not start from the ground up, but instead, want to leverage modern-day technologies for their legacy workloads. One of the more popular platforms many of our customers use today is Pivotal Cloud Foundry, now known as VMware Tanzu Application Service. As a platform, it offers many great pre-configured components which deservedly increased its popularity; however, some customers do find it to be quite restricted in terms of customizability and scalability, and want to future-proof themselves, as well as cut down on the licensing costs.

While PCF has a very static view on application development and deployment, Kubernetes provides a robust but flexible platform. In our Kubernetes migration experience, our developers reveled in their newfound increased agility, greater sense of control, and the ability to deploy applications without having to involve support teams. This ultimately led to improved productivity, fewer defects during deployments, and a better DevOps culture.

Speaking of DevOps culture, in 2021, as toolchains moved from scripts or “Only infrastructure management tools” to widely used programming languages and libraries, we began to observe the assimilation of this culture into the DevOps and Application engineering teams – two completely separate units in the past, now operating more cohesively and efficiently.

Now that we have defined our Kubernetes and open-source stack migration goal, let’s take a look at the technology choices and some details of the implementation.

Grid Dynamics has a long history of implementing various solutions with AWS, ranging from custom application development (both tooling and code), leveraging cloud-native and AWS-specific technologies, and working together with Amazon as development partners. Along with our customers, we have gained significant confidence in the platform and the vision behind its roadmap.

Some of the common reasons to make a choice in favor of a cloud-based platform, and AWS in particular, are:

• Maintenance simplicity: Whether we need to deploy application workloads across the globe in a single click, or we want to build and deploy specific applications closer to our end-users with single-digit millisecond latency, AWS is able to provide us with the cloud infrastructure where and when we need it.
• Security: AWS investment into platform security is unmatched, compared even to large enterprises and other cloud providers. One notable feature of the AWS security toolkit is Identity and Access Management (IAM), which enables fine-grained access control across AWS. With IAM, it is easy to specify who can access which services and resources, and under which conditions.
• Robust infrastructure: AWS offers customers the largest infrastructure from data centers globally. We don’t have to support this infrastructure. Besides, there are a lot of helpful managed DevOps tools provisioning all your infrastructure using familiar programming languages.
• Adoption of community best practices: AWS has a large community built around it, so we can use almost any open source tool or find an answer to any question. AWS also offers helpful support that is divided into several plans, enabling customers to choose a plan that meets their specific needs.

EKS Blueprints is a new open-source framework from Amazon aimed to ease and simplify some commonly used configuration patterns. It provides a level of abstraction on top of the AWS Cloud Development Kit (CDK: framework for provisioning cloud application resources using familiar programming languages), and helps to translate reference architecture into infrastructure building blocks. More importantly, it incorporates Amazon’s best practices and methodology and provides necessary guardrails in order to help build well-architected EKS-based environments with built-in security enforcements. It can also easily be extended with the help of add-ons, which already include a couple of well-known AWS-backed and open-source technologies, and it is relatively easy to add new ones should the project requirements call for it. You can find more information and examples in the official EKS Blueprint quick start documentation.

We leveraged a CDK-based implementation for our exercise, however, it is also available as a Terraform module, which we intend to try out in the next version of our starter kit.

Pivotal Cloud Foundry has a few core mechanisms relevant to the scope of migration:

• Platform dependencies aka services
  PCF dependencies like DBs work via brokers which, when installed into PCF, provide an out-of-the box interface to create infrastructure around applications.
• Applications are designed to work with PCF
  Applications running in PCF gain access to the bound service instances via credentials stored in an environment variable called VCAP_SERVICES.
• Deployment manifests
  Manifest.yml, a file which defines your app using only a few rows, is an interface to manage apps in PCF.
• Buildpacks
  Buildpacks provide framework and runtime support for apps. Buildpacks typically examine your apps to determine what dependencies to download and how to configure the apps to communicate with bound services.

The general structure of the resulting solution, based on top of AWS-backed services, can be described as follows:

• CDK as the base framework for both EKS blueprints and custom extensions;
• EKS Blueprints, abstracting most of the CDK implementation into higher-level constructs;
• Custom modules to provide deeper integrations;
• And finally, the code to perform mapping from the PCF environments to the new cloud-based platform, and to stitch all the components together.

Core components of the starter kit include:

• GitHub: hosts the source code for applications
• Container registry: stores images
• CodeBuild: CI for applications, build container images with applications, test them and push new image into images repository
• Secrets Manager: stores credentials for DBs
• Lambda: serverless functions triggered on events(incoming request or external trigger like EventBridge)
• EventBridge: supports event flow(trigger of Lambda on ECR push in our case)
• ArgoCD: GitOps CD solution for kubernetes
• Kubernetes-external-secrets: tool for custom templating of secrets from AWS secrets manger to K8S secrets

Probably the most challenging and interesting aspect of the migration is supporting simultaneous deployments to both the old PCF environment and new cloud platform using the same codebase without code changes in PCF native apps – buildpacks, services (also their credentials) management and manifests.

At a glance, the migration process using the starter kit can be distilled into the following steps:

1. The Real PCF organization is analyzed first using conversion tools. These tools use the PCF API in order to detect all microservices/parts that the target application stack consists of. They look up all PCF-native services that the application has connection to, thus creating two lists: application parts and PCF-native services. They persist in configuration files for further use.
2. The configuration files generated can be enriched with external details like additional role mappings for AWS-only users or mapping between PCF and IAM users, and Git repositories with application source code.
3. Then we proceed with generating the Kubernetes manifests for the applications we detected, so ArgoCD will know what will need to be deployed.
4. Next, infrastructure deployment starts. In addition to the common resources, it creates specific resources too. For this purpose, the automation in the starter kit maps CF-native services to corresponding AWS-based counterparts, like RDS databases or other resources. Concurrently, secrets are replaced with new AWS-based service credentials using AWS Secrets Manager.
5. The starter kit uses its own ExternalSecrets add-on during provision. This add-on takes data about bounded services from PCF and  data about AWS-based services from AWS Secrets Manager for creating Kubernetes secrets. These secrets look like the VCAP_SERVICES variable to PCF applications so we don’t need to change the application code.
6. And now, our applications should be up and running in EKS!

Of course, a lot of action happens behind the scenes. For example, if we dig deeper into the implementation of CI/CD pipelines which are provisioned as a part of the migration, we would see the following picture:

If you’re thinking about moving from Pivotal Cloud Foundry to AWS Kubernetes, our PCF Kubernetes starter kit leverages AWS EKS Blueprints and Grid Dynamics’ microservices platform to:

• Save time and money by cutting infrastructure provisioning time by at least 60%;
• Co-innovate in a fully pre-integrated environment;
• Migrate without changing the existing code;
• Educate your team about AWS migration best practices.

If you’re interested in a more in-depth review of the implementation or specific features of the starter kit and technology, get in touch with us to start a conversation!