How Global Payments unified Kubernetes with Red Hat OpenShift
Transcript
Global Payments, a Fortune 500 company, transformed its container platform strategy by adopting Red Hat OpenShift. Initially managing Kubernetes manually, they scaled to 150 clusters by 2022, processing over 4 billion transactions monthly. Challenges included patching 6,000+ servers and integrating various tools. OpenShift's enterprise support, automated updates, and consistent environment simplified operations. They leveraged GitOps for deployment, Istio for service mesh, and Thanos for scalable monitoring. The transition improved developer experience, reduced deployment failures, and enhanced observability, enabling proactive issue resolution and meeting high availability standards.
Action Items
[ ] Explore running OpenShift control plane on virtual machines to optimize worker node capacity.
[ ] Discuss with Red Hat the possibility of a "hub of hubs" solution to manage policies and configurations across multiple data center hubs.
Outline
Introduction and Speaker Introductions
Speaker 1 introduces the meeting, mentioning Global Payments and their transformation of the container platform strategy with Red Hat and advanced cluster management for Kubernetes.
Joseph Rajan and Sunil, both directors at Global Payments, are introduced as the main speakers.
Speaker 3, Sunil, introduces himself as the director of the SRE team at Global Payments.
Speaker 2, Joseph Rajan, introduces himself as another director for SRE, focusing on automation and observability.
Overview of Global Payments and Kubernetes Journey
Speaker 3 provides an overview of Global Payments, a Fortune 500 company based in Atlanta, focusing on financial services and insured payments.
The company has 5 million merchant accounts, 73 billion transactions for the year, and 4000 technology partners.
Speaker 3 outlines the Kubernetes journey starting in 2018, initially managing Kubernetes from scratch and using SaltStack for OS and Kubernetes installation.
The company started with five clusters in one data center, mostly Java microservices and Web UI applications.
Scaling and Challenges in Kubernetes Management
Speaker 3 discusses the scaling to 20 clusters across multiple data centers in 2019, using KubeATM for Kubernetes installation.
In 2020, the company moved to managing over 50 clusters, processing 15 million transactions per month.
The company started running Kubernetes on AWS, using EKS and Terraform for cluster management.
Speaker 3 highlights the challenges of managing open-source Kubernetes, including constant patching and upgrading of 6000+ servers.
Adoption of OpenShift and Mitigation of Challenges
Speaker 3 explains the adoption of OpenShift in 2023, starting with a POC and moving to live workloads in 2024.
Joseph Rajan covers the observability and automation journey, starting with a self-managed observability stack using Prometheus, Grafana, and Alertmanager.
The company moved to a centralized Prometheus for monitoring all clusters and introduced synthetic monitoring tools.
Speaker 3 discusses the challenges of running open-source Kubernetes, including constant updates, patching, and compliance with security standards.
Benefits of OpenShift and Operator Framework
Speaker 3 highlights the benefits of OpenShift, including enterprise support, certified operators, and simplified OS management.
The company uses the assisted installer for bare-metal installation, making it easier to manage clusters.
OpenShift operators, such as the service mesh operator and the GitOps operator, simplify the exposure of services and deployment of applications.
The company uses GitOps extensively for managing clusters and applications, ensuring consistency across data centers and cloud environments.
Observability and Automation Improvements
Speaker 2 discusses the improvements in observability and automation, including multi-cluster observability and centralized monitoring.
The company uses Grafana for visualization and has integrated various data sources for comprehensive monitoring.
Speaker 2 highlights the benefits of the observability stack, including proactive monitoring, reduced deployment failures, and improved code quality.
The company has integrated various tools, such as Splunk and network tools, to enhance observability and security.
Lessons Learned and Future Plans
Speaker 3 shares lessons learned from the Kubernetes journey, emphasizing the importance of collaboration and creating a consistent template for OpenShift implementation.
The company has successfully set up OpenShift in multiple data centers, improving developer experience and reducing deployment time.
Future plans include working with Red Hat to enhance observability and network observability, and exploring predictive modeling using OpenShift.
The company aims to continue improving the observability stack, ensuring high availability and proactive monitoring to meet stringent SLA requirements.