GreenKube

GreenKube Architecture

This document describes the current technical architecture of GreenKube, a lightweight platform for estimating, reporting, and optimizing the carbon footprint and cost of Kubernetes workloads.

Architecture Diagram

flowchart TB
    subgraph K8s["Kubernetes Cluster"]
        Prom["Prometheus"]
        OC["OpenCost"]
        K8sAPI["K8s API"]
        EM["Electricity Maps API"]
        WN["Wattnet API"]
        Boavizta["Boavizta API"]
        GITOPS["Argo CD / Flux"]
    end

    subgraph External["Git providers"]
        GIT["GitHub / GitLab / Gitea"]
    end

    subgraph GreenKube["GreenKube"]
        direction TB
        subgraph Collectors["Collectors (Input Adapters)"]
            PC["PrometheusCollector"]
            OCC["OpenCostCollector"]
            NC["NodeCollector"]
            PodC["PodCollector"]
            EMC["ElectricityMapsCollector"]
            WNC["WattnetCollector"]
            BC["BoaviztaCollector"]
        end

        subgraph Core["Core (Business Logic)"]
            DP["DataProcessor<br/>(Facade)"]
            CO["CollectionOrchestrator<br/>(Parallel Fetch)"]
            Est["BasicEstimator<br/>(CPU → Joules)"]
            Calc["CarbonCalculator<br/>(Joules → CO₂e)"]
            MA["MetricAssembler<br/>(Build CombinedMetric)"]
            NZM["NodeZoneMapper<br/>(Cloud → eMaps Zone)"]
            PRM["PrometheusResourceMapper<br/>(Per-Pod Resources)"]
            CN["CostNormalizer<br/>(Per-Step Cost)"]
            ESvc["EmbodiedEmissionsService<br/>(Boavizta Cache + Fallback)"]
            SR["SummaryRefresher<br/>(Pre-computed Cache)"]
            HRP["HistoricalRangeProcessor<br/>(Chunked Range)"]
            Rec["OptimizationEngine<br/>(multi-source + evidence)"]
        end

        subgraph Storage["Storage (Output Adapters)"]
            PG["PostgreSQL"]
            SQLite["SQLite"]
        end

        subgraph Automation["GitOps automation"]
            Disc["Discovery<br/>Argo CD / Flux"]
            Bind["Workload<br/>bindings"]
            PlanSvc["Plan service<br/>snapshot + render + diff"]
            Renderer["Isolated<br/>renderer"]
            Worker["Plan worker<br/>PR + polling"]
            Sync["Controller sync<br/>+ apply detection"]
        end

        subgraph Presentation["Presentation"]
            API["FastAPI<br/>REST API"]
            CLI["Typer CLI"]
            Dash["SvelteKit<br/>Dashboard"]
            Grafana["Grafana<br/>(via Prometheus)"]
        end
    end

    Prom --> PC
    OC --> OCC
    K8sAPI --> NC
    K8sAPI --> PodC
    EM --> EMC
    WN --> WNC
    Boavizta --> BC

    PC --> CO
    OCC --> CO
    PodC --> CO
    NC --> DP

    CO --> DP
    EMC --> DP
    WNC --> DP
    BC --> ESvc

    DP --> Est
    DP --> NZM
    DP --> PRM
    DP --> CN
    DP --> MA
    DP --> HRP
    DP --> SR

    MA --> Calc
    MA --> ESvc
    MA --> Storage
    SR --> Storage

    Storage --> API
    Storage --> CLI
    API --> Dash
    API --> Grafana
    Storage --> Rec
    Rec --> API
    K8sAPI --> Disc
    GITOPS --> Disc
    Disc --> Bind
    Bind --> PlanSvc
    PlanSvc --> Renderer
    PlanSvc --> Storage
    Worker --> GIT
    GIT --> Sync
    GITOPS --> Sync
    Sync --> Storage
    Storage --> Worker
    Rec --> PlanSvc

Overview

GreenKube runs scheduled collection and recommendation lifecycles alongside an API and dashboard for querying stored data. An optional GitOps automation path discovers workload sources, renders and validates plans, executes approved pull requests, and observes controller sync and apply state. The CLI starts the collector or a local demo; it does not provide ad-hoc report or recommendation commands.

The system is designed around the principles of Clean Architecture and Hexagonal Architecture:

All I/O operations use Python’s asyncio for high-performance, non-blocking concurrent execution.

Architectural Principles

1. Database Agnosticism

The core business logic (src/greenkube/core) NEVER depends on a specific storage implementation. This is enforced through:

Supported backends:

2. Cloud Provider Agnosticism

Cloud-specific details (AWS, GCP, Azure, OVH, Scaleway) are isolated in:

3. Asynchronous & Non-Blocking

Core Components

Collectors (Input Ports)

All collectors are fully asynchronous and implement a common pattern.

PrometheusCollector

NodeCollector

PodCollector

OpenCostCollector (optional)

ElectricityMapsCollector

WattnetCollector

BaseElectricityProvider

BoaviztaCollector

Estimator (Business Logic)

BasicEstimator

Processor (Use Case Orchestrator)

DataProcessor

The main orchestrator that coordinates the data pipeline from collection to metric assembly.

Architecture: The processor acts as a facade, delegating specialized work to focused collaborators while managing the overall pipeline flow.

Key Responsibilities:

  1. Data Collection: Coordinates parallel collection from Prometheus, Kubernetes, OpenCost, and external APIs
  2. Energy Estimation: Converts resource usage into energy consumption (Joules)
  3. Zone Mapping: Resolves cloud regions to carbon intensity zones
  4. Carbon Calculation: Computes CO2e emissions from energy and grid intensity
  5. Metric Assembly: Combines energy, cost, resources, and metadata into unified metrics
  6. Embodied Emissions: Integrates hardware manufacturing emissions

Pipeline Stages:

Implementation Pattern:

Calculator (Business Logic)

CarbonCalculator

SummaryRefresher (Business Logic)

SummaryRefresher

Optimization Engine (Business Logic)

OptimizationEngine

Analyzes CombinedMetric data to identify optimization opportunities. It is the single orchestration path used by API recommendation requests and the startup scan (src/greenkube/core/optimization/). The CLI does not expose a recommendation command.

Pipeline: build an OptimizationContext (metrics window + node/HPA/PV/LB inputs) → run enabled sources → normalize/arbitrate/deduplicate → attach an evidence block and a risk/confidence/effort assessment → compute the multi-criteria ranking score → persist and reconcile.

Sources:

  1. Native (greenkube) — the built-in analyzers:
    • Zombie Pods: workloads consuming cost with minimal energy.
    • Rightsizing: over-provisioned CPU/memory based on observed usage percentiles and configured headroom.
    • Autoscaling Candidates: high coefficient of variation / spike ratio, skipping workloads already governed by an HPA.
    • Off-peak Scaling: idle windows suggesting scale-to-zero schedules.
    • Idle Namespaces: namespaces with minimal activity.
    • Carbon-Aware Scheduling: workloads running during high-intensity periods.
    • Node optimization: overprovisioned / underutilized nodes.
    • Orphaned PV / LoadBalancer cleanup.
  2. VPA (vpa, optional) — reads recommendation-mode VerticalPodAutoscalers (updateMode: Off) and replaces native CPU/memory rightsizing for the same workload, avoiding duplicate advice.

  3. Karpenter (karpenter, optional) — node-pool consolidation recommendations when compatible Karpenter CRDs are available.

Evidence, risk and ranking: every recommendation carries observation window, utilization distribution, current vs proposed resources, expected savings and method, confidence, reliability risk, rollback conditions, a machine-readable patch plan and an expiry date. Ranking profiles (balanced, carbon_first, cost_first, quick_wins, low_risk) combine impact, confidence, risk, effort, actionability and source authority.

For the user-facing automation workflow and supported source layouts, see GitOps automation.

Configuration: All thresholds, source flags and ranking settings are configurable via config.py and Helm values.

Repositories (Output Ports)

Repositories use asynchronous drivers for high-performance database interactions. All implement abstract base classes to ensure database agnosticism.

CarbonIntensityRepository

Abstract base class for carbon intensity storage.

Implementations:

NodeRepository

Abstract base class for node state snapshots.

Implementations:

CombinedMetricRepository

Stores final aggregated metrics (energy + carbon + cost + resources).

Schema (33 columns):

Migrations: All backends support automatic schema evolution (ADD COLUMN IF NOT EXISTS)

EmbodiedRepository

Caches Boavizta API responses for hardware embodied emissions.

Schema: provider, instance_type, gwp_manufacture, lifespan_hours, last_updated

SummaryRepository

Stores pre-computed KPI scalar totals per window.

Schema: window_slug, namespace, total_co2e_grams, total_embodied_co2e_grams, total_cost, total_energy_joules, pod_count, namespace_count, updated_at

TimeseriesCacheRepository

Stores pre-computed time-series buckets per window and granularity.

Schema: window_slug, namespace, bucket_ts, co2e_grams, embodied_co2e_grams, total_cost, joules

API & Presentation Layer

FastAPI Server

Endpoints:

Report export query parameters:

SvelteKit Dashboard

Pages:

Features:

CLI

The REST API is started by the separate greenkube-api entry point (greenkube.api.app:main).

Grafana Integration

Data Flow

Instant Collection (run())

Used for real-time monitoring and scheduled collection.

  1. Phase 1 — Node Discovery (sequential, single K8s API call):
    NodeCollector.collect()
    └─ Node metadata (instance type, zone, capacity, provider)
    └─ Build node_instance_map directly from Phase-1 data
    
  2. Phase 2 — Zone Resolution (depends on Phase-1 data):
    NodeZoneMapper.map_nodes(nodes_info)
    └─ Map cloud region → Electricity Maps zone per node
    
  3. Phase 3 — Parallel Collection + Boavizta (concurrent, Phase-1/2 data used for enrichment):
    asyncio.gather(
      ┌─ CollectionOrchestrator.collect_all(nodes_info):
      │   ├─ PrometheusCollector.collect()
      │   │   ├─ CPU usage (8 concurrent queries)
      │   │   ├─ Memory usage
      │   │   ├─ Network I/O (rx + tx)
      │   │   ├─ Disk I/O (read + write)
      │   │   ├─ Restart counts
      │   │   └─ Node labels (enriched with Phase-1 data)
      │   ├─ OpenCostCollector.collect()
      │   │   └─ Cost allocation data
      │   └─ PodCollector.collect()
      │       └─ Resource requests (CPU, memory, storage)
      └─ EmbodiedEmissionsService.prepare_embodied_data(nodes_info):
          ├─ Check EmbodiedRepository cache
          ├─ Fetch missing profiles from Boavizta API
          └─ Inject fallback profile (DEFAULT_EMBODIED_EMISSIONS_KG) for unknowns
    )
    
  4. Phase 4 — Assembly:
    ├─ BasicEstimator.estimate() → EnergyMetric per pod (Joules)
    ├─ MetricAssembler.prefetch_intensities()
    │   ├─ Group pods by zone
    │   └─ Prefetch intensity for (zone, timestamp) pairs
    └─ MetricAssembler.assemble()
        ├─ CarbonCalculator.calculate_emissions()
        ├─ EmbodiedEmissionsService.calculate_pod_embodied()
        │   └─ Mark metric is_estimated=True if fallback used
        └─ Build CombinedMetric (energy + carbon + cost + resources + metadata)
    
  5. Persistence:
    Repository.write_combined_metrics()
    └─ Batch insert to database (Postgres/SQLite)
    
  6. Background (hourly scheduler):
    SummaryRefresher.run()
    ├─ For each window (24h, 7d, 30d, 1y, ytd):
    │   ├─ aggregate_summary() → MetricsSummaryRow → upsert metrics_summary
    │   └─ aggregate_timeseries() → TimeseriesCachePoint[] → upsert metrics_timeseries_cache
    └─ Repeated for each namespace
    

Historical Analysis (run_range())

Used for reporting over time ranges with historical accuracy.

  1. Historical Node State Loading:
    NodeRepository.get_latest_snapshots_before(start)
    NodeRepository.get_snapshots(start, end)
    └─ Reconstruct node timeline
    
  2. Chunked Processing (day-sized chunks to prevent OOM):
    For each chunk (1 day):
      ├─ Prometheus range queries (concurrent):
      │   ├─ CPU usage over time
      │   ├─ Network I/O over time
      │   ├─ Disk I/O over time
      │   ├─ Restart counts over time
      │   └─ (5 queries via asyncio.gather)
      │
      ├─ Parse time-series data:
      │   └─ Build per-pod resource maps from range results
      │
      ├─ Energy estimation per timestamp:
      │   └─ Use historical node profile at each timestamp
      │
      ├─ Carbon calculation:
      │   └─ Historical intensity lookup
      │
      └─ Generate CombinedMetrics for chunk
    
  3. Aggregation:
    Collect all chunks → Filter by namespace → Return
    

Recommendation Flow

  1. Data Collection:
    Build OptimizationContext from repository (last N days) + node/HPA/PV/LB inputs
    
  2. Analysis:
    OptimizationEngine.generate()
    ├─ Run enabled sources (native analyzers, VPA)
    ├─ Normalize provenance and arbitrate conflicts by capability/priority
    ├─ Deduplicate and clamp values to configured minimums
    ├─ Attach evidence, risk/confidence/effort and expiry
    └─ Compute multi-criteria ranking score and factors
    
  3. Output:
    Return List[Recommendation] with:
    ├─ Type, source and provenance
    ├─ Affected resources (current vs recommended)
    ├─ Evidence (window, percentiles, rollback conditions, patch plan)
    ├─ Expected savings (cost + CO2e) and confidence
    └─ Risk level, effort and ranking score
    
  4. Persistence:
    Upsert active records by target identity; mark absent actives as stale
    

GitOps Automation Flow

  1. Discovery:
    Argo CD Application status + Flux inventories + live workloads
    └─ Persist generation-bound workload bindings
    
  2. Plan creation:
    Recommendation IDs (never repository/path/credentials)
    └─ Resolve binding → snapshot immutable revision → render → validate → expiring digest
    
  3. Approval and execution:
    Exact digest approval
    └─ Durable operation queue → worker revalidation
    └─ Atomic multi-file commit → deterministic branch → pull request
    
  4. Sync and verification:
    Webhook/polling update → controller observes exact source revision
    └─ actions sync_pending; plan merged → synced (or sync_failed after timeout)
    └─ Apply detection compares per-container requests
    └─ Normal post-apply health and savings verification
    

Only CPU and memory rightsizing recommendations have action planners. Unsupported sources and renderers fail closed with typed reasons. See GitOps automation for the supported matrix and security model.

Configuration & Deployment

Environment Variables (12-Factor App)

All configuration flows through src/greenkube/core/config.py:

Database:

External Services:

Collection:

Carbon:

Recommendations:

Automation:

Helm Chart Structure

helm-chart/
├── Chart.yaml                 # Chart metadata
├── values.yaml                # Standalone/default configuration
├── values-production.yaml     # Production profile overlay
└── templates/
    ├── deployment.yaml        # Standalone collector + API
    ├── production-deployments.yaml # API, controller, automation worker
    ├── api-service.yaml       # API/UI ClusterIP Service
    ├── automation-worker.yaml # Standalone worker mode
    ├── configmap.yaml         # Non-secret config
    ├── secret.yaml            # Tokens and credentials
    ├── migration-job.yaml     # Production schema migration hook
    ├── leader-lease.yaml      # Production controller Lease
    ├── renderer-sandbox.yaml  # Isolated Helm/Kustomize renderer
    ├── postgres-*.yaml        # PostgreSQL StatefulSet (optional)
    ├── serviceaccount.yaml    # K8s RBAC
    └── clusterrole.yaml       # Read permissions for nodes/pods/GitOps

Container Structure (Multi-Stage Build)

Dockerfile (multi-stage):
1. Frontend Builder (Node 22):
   ├─ npm ci (install deps)
   ├─ npm run build (SvelteKit static build)
   └─ Output: static files in build/

2. Python Builder:
   ├─ uv sync / build the wheel from pyproject.toml
   └─ Install into an isolated prefix

3. Final Image (Python 3.14-slim):
   ├─ Copy Python packages from builder
   ├─ Copy frontend build from frontend-builder
   ├─ Run as non-root user 'greenkube'
   └─ ENTRYPOINT: greenkube CLI

Normalization and Caching

Timestamp Normalization

Controlled by NORMALIZATION_GRANULARITY:

Caching Strategy

CarbonCalculator Cache:

Repository Cache:

Boavizta Cache:

Dashboard Summary Cache:

Performance Considerations

Concurrency

Memory Management

Scalability

Testing Strategy

Unit Tests

Integration Tests

Test Patterns

Design Principles & Guidelines

For Contributors

Database Agnosticism (Critical)

Error Handling

Async Best Practices

Code Quality

Logging Levels

Commit Messages

Follow conventional commits:

Future Architecture Evolution

Closed-loop reliability and recovery

The recommendation-to-GitOps composition is intentionally fail-soft at external boundaries:

Boundary Recovery contract Verification
Startup recommendation refresh Runs in a tracked background task; provider or database errors are logged and do not prevent liveness traffic. tests/integration/test_gitops_closed_loop.py::test_startup_refresh_failure_does_not_block_shutdown_or_health_state
Savings ledger write A transient ledger failure is logged and returns zero; the collection loop continues and can retry on its next period. tests/integration/test_gitops_closed_loop.py::test_savings_ledger_failure_is_recoverable
Rollback transition Recommendations in rollback_review or reverted stop producing new ledger attribution while their history remains queryable. tests/integration/test_gitops_closed_loop.py::test_rollback_review_stops_future_savings_attribution

The API liveness endpoint does not probe optional providers. Readiness is marked only after the database connection succeeds and is cleared before shutdown. This keeps restarts and provider outages observable without turning a transient dependency failure into a process-wide failure.

Planned Enhancements

Multi-Cluster Support:

Advanced Analytics:

Real-Time Streaming:

Extended Metrics:

Enhanced Reporting:

Extension Points

New Collectors:

  1. Implement collector interface
  2. Add to core/factory.py
  3. Update DataProcessor to use it
  4. Add tests

New Storage Backend:

  1. Implement CarbonIntensityRepository abstract
  2. Add to core/factory.py mapping
  3. Add schema migration logic
  4. Add tests

New Cloud Provider:

  1. Add region mappings to utils/region_mapping.py
  2. Add instance profiles to energy/instance_profiles.py
  3. Add PUE values to core/config.py
  4. Update tests

Resources

License

Apache 2.0 - See LICENSE file for details.