Cloud Security – Part 2 – Kubernetes and Container Security

With thanks to friends at docker.com, Kubernetes.io, Wikipedia, Check Point, Sysdig

Let’s start with the text book

Container and Docker

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

A Docker is a software platform that allows you to build, test, and deploy applications quickly.

Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime.

Kubernetes

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

It groups containers that make up an application into logical units for easy management and discovery.

• Planet scale - Designed on the same principles that allow Google to run billions of containers a week, Kubernetes can scale without increasing your operations team.
• Never outgrow - Whether testing locally or running a global enterprise, Kubernetes flexibility grows with you to deliver your applications consistently and easily no matter how complex your need is.
• Run Anywhere - Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.

Kubernetes has become the de facto operating system of the cloud. This rapid success is understandable, as Kubernetes makes it easy for developers to package their applications into portable microservices.

However, Kubernetes can be challenging to operate. Teams often put off addressing security processes until they are ready to deploy code into production. Kubernetes requires a new approach to security.

After all, legacy tools and processes fall short of meeting cloud-native requirements by failing to provide visibility into dynamic container environments. Fifty-four percent of containers live for five minutes or less, which makes investigating anomalous behavior and breaches extremely challenging.

One of the key points of cloud-native security is addressing container security risks as soon as possible. Doing it later in the development life cycle slows down the pace of cloud adoption, while raising security and compliance risks.

The Cloud/DevOps/DevSecOps teams are typically responsible for security and compliance as critical cloud applications move to production. This adds to their already busy schedule to keep the cloud infrastructure and application health in good shape.

Kubernetes Security Considerations

The first area to protect is your applications and libraries. Vulnerabilities in your base OS images for your applications can be exploited to steal data, crash your servers or scale privileges. Another component you need to secure are third-party libraries. Often, attackers won't bother to search for vulnerabilities in your code because it's easier to use known exploits in your applications libraries.

The next vector is the Kubernetes control plane - your cluster brain. Programs like the controller manager, etc or kubelet, can be accessed via the Kubernetes API. An attacker with access to the API could completely stop your server, deploy malicious containers or delete your entire cluster.

Additionally, your cluster runs on servers, so access to them needs to be protected. Undesired access to these servers, or the virtual machines where the nodes run, will enable an attacker to have access to all of your resources and the ability to create serious security exposures.

OWASP Top 10 Kubernetes Cybersecurity Issues (2022)

When adopting Kubernetes, we introduce new risks to our applications and infrastructure. The OWASP Kubernetes Top 10 is aimed at helping security practitioners, system administrators, and software developers prioritize risks around the Kubernetes ecosystem. The Top 10 is a prioritized list of these risks.

K01: Insecure Workload Configurations
K02: Supply Chain Vulnerabilities
K03: Overly Permissive RBAC Configurations
K04: Lack of Centralized Policy Enforcement
K05: Inadequate Logging and Monitoring
K06: Broken Authentication Mechanisms
K07: Missing Network Segmentation Controls
K08: Secrets Management Failures
K09: Misconfigured Cluster Components
K10: Outdated and Vulnerable Kubernetes Components

Prometheus

Prometheus is a free software application used for event monitoring and alerting. It records metrics in a time series database (allowing for high dimensionality) built using an HTTP pull model, with flexible queries and real-time alerting.

It is an open source project for monitoring cloud-native applications and Kubernetes. It is a graduated member of the Cloud Native Computing Foundation (CNCF) and has an active developer and user community.

A typical monitoring platform with Prometheus is composed of multiple tools:

• Multiple exporters typically run on the monitored host to export local metrics.
• Prometheus to centralize and store the metrics.
• Alertmanager to trigger alerts based on those metrics.
• Grafana to produce dashboards.
• PromQL is the query language used to create dashboards and alerts.

Prometheus is typically used to collect numeric metrics from services that run 24/7 and allow metric data to be accessed via HTTP endpoints. This can be done manually or with various client libraries. Prometheus exposes data using a simple format, with a new line for each metric, separated with line feed characters.

Non-secure deployments of Prometheus, JFrog warns, may pose an even larger security risk, via an optional management API that can be used to delete metrics and close the monitoring server.

One solution would be to secure the Prometheus server accesses using a reverse proxy, encrypting the communications between clients and a server. We can implement this security layer with an Apache or Nginx server configured as a reverse proxy that implements TLS encryption, authentication and other security features.

Security considerations of cloud containers

It’s no secret that containerization has been one of the hottest tech trends of the last decade, and today containers are almost ubiquitous. In fact, Gartner projects that this year 75% of global enterprises will run containers in production.

With the rise in container popularity, there have been plenty of benefits. Containers are the cornerstone of microservices architectures that have enabled cloud native apps of all sizes. However, because of their popularity, containers are also a prime target for ransomware, hackers, and other threats.

As a result, enterprises that value a strong security posture must be able to address common container security issues. While there’s no single silver bullet for addressing container security challenges, taking a holistic approach and leveraging the right tools can go a long way.

Cloud Container Security - Objectives:

• Leveraging the ShiftLeft tool to create automatically secure containers.
• Using agentless security tooling to protect all enterprise cloud assets.
• Gain posture management and deep visibility across all containers, even in multi-cloud environments.
• Enforce the principle of least privilege and help stay compliant.
• Detect configuration issues like exposed credentials.
• Scan container images for vulnerabilities, malware, and weak configurations.
• Automatically deploy granular security controls.
• Reduce risk with image scanning integrated into CI/CD pipelines and runtime.
• Secure runtime with out-of-the-box managed policies based on Falco and ML.
• Know what happened and why with an audit trail.

7 Cloud Container Security Issues and Challenges

To address container security challenges, enterprises need to understand the security risks impacting container workloads. The following 7 container security issues demonstrate the wide range of strategic and tactical challenges related to container-based infrastructure.

1. Effectively shifting left

DevSecOps and the concept of shift left security emphasize the importance of integrating security throughout the software development lifecycle (SDLC) and eliminating friction in the process of developing secure software.

While DevSecOps tools and automation garner a lot of the “shift left” headlines, a big part of effectively shifting left is cultural. Different organizational units within enterprises must move away from the idea of “security as the team of no” and embrace cooperation. The organizations that are able to truly adopt a DevSecOps mindset and make security “everyone’s” responsibility are better positioned to improve security posture across an enterprise.

2. Managing ephemeral containers

Ephemeral containers are useful administrative and debugging tools in Kubernetes (K8s) clusters. For example, they can enable troubleshooting in environments that use distroless images. However, this also means ephemeral containers create an additional attack surface that wouldn’t otherwise exist. As a result, managing ephemeral containers is an essential aspect of K8s security.

While ephemeral containers can be powerful tools for capturing debug information, enterprises should implement security policies restricting their use to only necessary workloads and environments.

3. Addressing misconfigurations

According to a leading cybersecurity vendor’s recent cloud security survey, 27% of respondents reported a public cloud security incident. Of those incidents, 23% resulted from misconfigurations. That’s just one of many examples of the security risk posed by misconfigurations.

To ensure robust container security and workload protection, enterprises must be able to continuously detect — and correct — misconfigurations in container cluster configurations. That means ensuring only secure configurations are used in production, and no sensitive information or secrets are exposed.

4. Countering known vulnerabilities

Zero-day threats are a real risk facing enterprises today, but many breaches exploit known vulnerabilities. By scanning container images, dependencies, and workloads, enterprises can detect and implement a plan to address known vulnerabilities before they’re used in an exploit.

Integrating security tooling throughout the SDLC and CI\CD pipelines can go a long way in addressing this container security challenge. Enterprises that shift security left can often detect threats before they make it to production or mitigate them sooner than they otherwise could. For example, leveraging virtual patching to temporarily mitigate vulnerabilities until new containers are deployed.

5. Protecting against runtime threats

While signature-based detection works well to identify known exploits, many cloud workload security threats, such as zero-day exploits, require context to detect and mitigate. To provide enterprise-grade security for web applications and APIs, organizations need tooling that uses intelligence and context to detect new threats and limit false positives that hamstring productivity. Additionally, many cloud native applications can’t accommodate traditional endpoint security agents and instead require an agentless approach to runtime security.

6. Addressing human error

Human error is a common factor in many security incidents today. Manual processes leave room for typos, misconfigurations, and oversight that can lead to a breach. While IPS, IDS, and firewalling can help reduce risk after these misconfigurations occur, they don’t go far enough.

Enterprises should limit manual configuration and automate as much of their security configuration as practical. Additionally, they should implement scans that use policies to detect and help address misconfigurations before they’re exploited.

7. Passing compliance audits

Compliance risk is one of the single biggest risks facing modern enterprises. Failing an audit related to standards like GDPR, HIPAA, or SOX can damage an enterprise’s reputation and bottom line.

As a result, ensuring that container workloads and K8s clusters meet compliance requirements is a must. Cloud security posture management (CSPM) and Kubernetes security posture management (KSPM) tools can help automate compliance across cloud and container infrastructure.

Author Bio

Anthony Lim

MAISP

Fellow, Cybersecurity, Governance & Fintech, Singapore University of Social Sciences

Anthony is a pioneer of cyber-security and governance in Singapore and the Asia Pacific region, with over 25 years’ professional experience, as a business leader, consultant, advocate, instructor and auditor.

He has managed some national-level cybersecurity readiness assessment projects in Singapore and the region and was a co-author of an acclaimed international cloud security professional certification. He has held inaugural senior regional business executive appointments at Check Point, IBM and CA (now Broadcom), and was also client CISO at Fortinet and NCS. He has been active in industry association circles for nearly 2 decades, and is currently Advocate at (ISC)2 Singapore Chapter.

Anthony is an adjunct instructor and module developer for some tertiary academic & professional institutions. He has presented and provided content at many government, business, industry and academic seminars, committees, executive roundtables, workshops, trainings and media (print, broadcast, internet, including CNA, CNBC, Bloomberg, BBC) in Singapore, the region, and also for NATO, at Washington DC, Stanford University, ITU, Guangzhou Knowledge CIty and TsingHua University. He is a life alumni member of the University of Illinois, Urbana-Champaign.