Streamline Your CI/CD Pipeline: Integrating Harpoon with ArgoCD for Effortless Deployments

In the modern software development landscape, continuous integration and continuous deployment (CI/CD) are crucial for delivering high-quality applications rapidly and reliably. This guide explores the powerful integration of ArgoCD and Harpoon, showing you how to create a sophisticated, automated deployment pipeline that maintains consistency and reliability across your Kubernetes infrastructure.

Understanding the Core Components

ArgoCD

A GitOps continuous delivery tool that automates Kubernetes deployments

Monitors and syncs application states defined in Git with the live environment

Provides declarative configuration management

Harpoon

Complements ArgoCD by handling dynamic deployments and scaling

Manages cluster reconstruction based on manifest changes

Ensures efficient resource utilization and deployment stability

Architecture and Workflow

The Integration Flow

1. Change Initiation

Developer commits changes to Git repository

ArgoCD detects changes in manifest files

2. Sync Process

ArgoCD validates new manifests

Calculates required changes

Initiates sync process

3. Harpoon’s Role

Detects ArgoCD sync activities

Manages dynamic scaling and reconstruction

Ensures smooth transition during changes

Navigating the Harpoon User Interface

Before diving into the practical implementation, let’s familiarize ourselves with Harpoon’s interface, which you’ll be working with throughout the deployment process.

1. Cluster and Service Information

When working with Harpoon, you’ll have access to important cluster information:

Service name configuration

Cluster IP address (e.g., 172.20.190.254)

DNS information for your AWS deployment

Namespace details (e.g., harpoon-99517-40126)

2. Deployment Configuration

The Harpoon deployment YAML shows several crucial configurations:

API Version and Deployment Kind.

Metadata including:

Kubernetes revision annotations

Creation timestamp

Unique identifiers

Deployment specifications:

Progress deadline (600 seconds)

Replica count

Revision history limits

Rolling update strategy (25% maxSurge and maxUnavailable)

3. Build and Deploy Interface

The main Harpoon interface demonstrates its powerful multi-service orchestration capabilities:

Database Service (MongoDB)
- Version selection (set to “latest”)
- Simple deployment controls
- Direct database configuration management
Backend Service (harpoondemoback)
- Branch selection (main branch)
- Commit reference tracking
- Build and Deploy pipeline controls
- Configuration management for backend services
Frontend Service (harpoondemofront)
- Separate branch management (dev branch)
- Independent commit tracking
- Dedicated build and deploy controls
- Frontend-specific configuration options
Global Controls and Configuration
- Cluster state monitoring (showing “Up” status)
- Project selection dropdown
- Saved configuration values
- Right-side panel providing access to:
  - Config maps
  - Volume management
  - Secret configuration
  - Ingress route setup
  - Node configuration

This interface showcases Harpoon’s ability to manage complex, multi-service architectures while maintaining clear separation and control over each component.

Key Features to Note

Service Independence
- Each service can be managed independently
- Different branches can be used for different services
- Separate build and deploy cycles
Configuration Management
- Centralized config value tracking
- Shared resource management
- Integrated secrets handling
Monitoring and Control
- Real-time cluster state monitoring
- Individual service build status
- Unified deployment overview

This comprehensive interface demonstrates how Harpoon simplifies the management of complex, multi-service applications while maintaining granular control over each component. With this understanding of the UI, let’s proceed to the practical implementation.

Practical Implementation

First, install ArgoCD in your cluster:


Copied!kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Install Harpoon:


Copied!bash
kubectl apply -f https://raw.githubusercontent.com/harpoon/harpoon/main/deploy.yaml
bash
kubectl apply -f https://raw.githubusercontent.com/harpoon/harpoon/main/deploy.yaml

2. Configure ArgoCD Application:


Copied!yaml
argocd-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sample-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/your-repo.git
    targetRevision: HEAD
    path: kubernetes/manifests
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
yaml
argocd-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sample-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/your-repo.git
    targetRevision: HEAD
    path: kubernetes/manifests
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

3. Define Your Application Manifests


Copied!yaml
 deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
        - name: app-container
          image: myapp:latest
          env:
            - name: ENVIRONMENT
              value: "production"
yaml
 deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
        - name: app-container
          image: myapp:latest
          env:
            - name: ENVIRONMENT
              value: "production"

Understanding Change Propagation

Manifest Changes and Reconstruction

When changes occur in manifests, the following sequence takes place:

1. Git Update


Copied!bash
git add deployment.yaml
git commit -m "Update configuration"
git push origin main
bash
git add deployment.yaml
git commit -m "Update configuration"
git push origin main

2. ArgoCD Detection

ArgoCD polls Git repository

Detects configuration changes

Initiates sync process

3. Harpoon Reconstruction

Monitors ArgoCD sync events

Evaluates resource requirements

Manages scaling and deployment updates

Example: Scaling Application

1. Update Manifest


Copied!yaml
spec:
  replicas: 5   Changed from 3 to 5
yaml
spec:
  replicas: 5   Changed from 3 to 5

2. Commit Changes


Copied!bash
git add deployment.yaml
git commit -m "Scale to 5 replicas"
git push origin main
bash
git add deployment.yaml
git commit -m "Scale to 5 replicas"
git push origin main

3. Verify Changes


Copied!kubectl get deployments sample-app
argocd app get sample-app
kubectl get deployments sample-app
argocd app get sample-app

Handling Configuration Changes

Environment Variables

When updating environment variables:


Copied!yaml
spec:
  template:
    spec:
      containers:
        - name: app-container
          env:
            - name: NEW_FEATURE_FLAG
              value: "enabled"
yaml
spec:
  template:
    spec:
      containers:
        - name: app-container
          env:
            - name: NEW_FEATURE_FLAG
              value: "enabled"

Harpoon manages this change by:

1. Detecting the configuration update

2. Implementing rolling updates

3. Maintaining service availability

Resource Allocation


Copied!yaml
spec:
  template:
    spec:
      containers:
        - name: app-container
          resources:
            requests:
              memory: "256Mi"
              cpu: "500m"
            limits:
              memory: "512Mi"
              cpu: "1000m"
yaml
spec:
  template:
    spec:
      containers:
        - name: app-container
          resources:
            requests:
              memory: "256Mi"
              cpu: "500m"
            limits:
              memory: "512Mi"
              cpu: "1000m"

Testing and Validation

1. Sync Status Verification


Copied!argocd app get sample-app
argocd app get sample-app

2. Deployment Health Check


Copied!kubectl describe deployment sample-app
kubectl get pods -l app=sample-app
kubectl describe deployment sample-app
kubectl get pods -l app=sample-app

3. Resource Validation


Copied!kubectl top pods -l app=sample-app
kubectl top pods -l app=sample-app

Best Practices

1. Version Control

Use semantic versioning for images

Maintain clear Git history

Document all configuration changes

2. Resource Management

Set appropriate resource limits

Implement horizontal pod autoscaling

Monitor cluster capacity

3. Security


Copied!yaml
Add network policies
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: sample-app-policy
spec:
  podSelector:
    matchLabels:
      app: sample-app
  policyTypes:
    - Ingress
    - Egress
yaml
Add network policies
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: sample-app-policy
spec:
  podSelector:
    matchLabels:
      app: sample-app
  policyTypes:
    - Ingress
    - Egress

4. Monitoring Setup

Implement comprehensive monitoring:


Copied!yaml
prometheus-servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: sample-app-monitor
spec:
  selector:
    matchLabels:
      app: sample-app
  endpoints:
    - port: metrics
yaml
prometheus-servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: sample-app-monitor
spec:
  selector:
    matchLabels:
      app: sample-app
  endpoints:
    - port: metrics

Troubleshooting Guide

Common Issues and Solutions

1. Sync Failures


Copied!argocd app logs sample-app
kubectl logs -n argocd deployment/argocd-application-controller
argocd app logs sample-app
kubectl logs -n argocd deployment/argocd-application-controller

2. Resource Constraints


Copied!kubectl describe nodes | grep -A 5 "Allocated resources"

kubectl describe nodes | grep -A 5 "Allocated resources"

3. Network Issues


Copied!kubectl exec -it sample-app-pod -- nc -vz service-name 80

kubectl exec -it sample-app-pod -- nc -vz service-name 80

Advanced Topics

1. Blue-Green Deployments


Copied!yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

2. Canary Deployments

Implement progressive traffic shifting:


Copied!yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: sample-app
spec:
  hosts:
  - sample-app
  http:
  - route:
    - destination:
        host: sample-app-v1
      weight: 90
    - destination:
        host: sample-app-v2
      weight: 10
yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: sample-app
spec:
  hosts:
  - sample-app
  http:
  - route:
    - destination:
        host: sample-app-v1
      weight: 90
    - destination:
        host: sample-app-v2
      weight: 10

The integration of ArgoCD and Harpoon creates a robust, automated deployment pipeline that combines the benefits of GitOps with dynamic infrastructure management. This setup provides:

Automated deployment processes

Consistent configuration management

Reliable scaling and updates

Clear audit trail of changes

Remember to always test changes in a development environment first and maintain comprehensive documentation of your deployment processes.

Resources and Further Reading

ArgoCD Documentation

Kubernetes Documentation

GitOps Principles