Deploy Prometheus and Grafana to AWS with GitHub Actions

Ever wanted to host your own system to track important metrics over time?

You're in luck! Bitovi's DevOps team created a new GitHub Action: Deploy Prometheus that makes hosting an instance of Prometheus and Grafana in your own AWS account super easy!

Whether you want to track the temperature of rooms in your house, website traffic visits, or even your company's financial health, Prometheus makes it easy, and with Bitovi's new action, you can now you can seamlessly and easily set up Prometheus and Grafana and up your monitoring game!

If you have a GitHub repo, an AWS account, and a desire to track anything you want, this post is for you!

What does the Deploy Prometheus Action do?

Our Deploy Prometheus Action will create the required infrastructure in AWS before installing Prometheus and Grafana and providing you with a URL.

Creates an EC2 instance behind a load balancer, among other resources.
Copies the Prometheus and Grafana config (Docker compose) the VM
Runs docker-compose up

After your workflow completes, a link to your deployment URL will be provided.

Prerequisites

AWS Account (yep, that’s it!)

You’ll need to store your secrets in Github. (Could be at repository or organization level)

Nice to Have

A domain in Route53 - Setting this up in AWS will make the final deployment a breeze. (See our blog on how to set this up! - It’s as simple as adding sub_domain and domain_name to your action)

Summary of Work

This example uses the files located here.

The example repository gives us a Sandbox Prometheus and Grafana instance with URL access to each.

In this article, you will:

Add Observability configuration files
Add a workflow (which calls the action)
Clean up the infrastructure when you’re done

This article assumes you are using branch based environments, so start by creating a new branch: sandbox.

1. Add Observability Configuration Files

Prometheus and Grafana each use file-based configuration.

To configure Prometheus, create a new file for the sandbox Prometheus:

sandbox/observability/prometheus/prometheus.yml

global:
  scrape_interval: 15s
  scrape_timeout: 10s
  evaluation_interval: 15s
alerting:
  alertmanagers:
  - static_configs:
    - targets: []
    scheme: http
    timeout: 10s
    api_version: v1
scrape_configs:
- job_name: prometheus
  honor_timestamps: true
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - localhost:9090

This is where you configure your external endpoint scrape targets and other configuration for Prometheus.

To configure Grafana, create a new file for the sandbox Prometheus:

sandbox/observability/grafana/datasources/sources.yml

apiVersion: 1

datasources:
# set the local prometheus instance as a datasource
- name: Prometheus
  type: prometheus
  url: http://prometheus:9090 
  isDefault: true
  access: proxy
  editable: true

2. Add a Workflow

To deploy Prometheus and Grafana with Bitovi’s deploy-prometheus action given the above configuration, all you need to do is create a workflow file under .github/workflows for each branch-based environment. For example, to create our sandbox deployment, create the following file:

.github/workflows/deploy-sandbox.yaml

name: Deploy Prometheus Sandbox

on:
    push:
      branches: [ sandbox ]
      paths:
      - '.github/workflows/deploy-sandbox.yaml'
      - 'observability/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    # Bitovi's Deploy Prometheus
    - id: deploy
      name: Deploy
      uses: bitovi/github-actions-deploy-prometheus@v0.1.0
      with:
        # AWS config
        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID}}
        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY}}
    
        # Secrets config
        env_ghs: ${{ secrets.DOT_ENV }}
     
        # observability config
        grafana_datasource_dir: sandbox/observability/grafana/datasources
        prometheus_config: sandbox/observability/prometheus/prometheus.yml

You can check all the possible variables you can use by going to our Github Action Marketplace Listing: Deploy Prometheus.

When you push code to the repo’s sandbox branch and the action completes, it will output a URL you can use to access your new instance. The URL will point to the Load Balancer in front of your instance, or if you also set up a domain name in Route53, the output will be that domain name.

Prometheus UI:

Grafana UI:

3. Cleanup

If you wish to destroy all of the resources created, you can add the following inputs at the bottom of your workflow.

Merging this to main will trigger the workflow again.

        tf_stack_destroy: true
        # If you wish to delete the auto-generated bucket
        tf_state_bucket_destroy: true

That's it! That covers the basics.

Conclusion

Congratulations! 🎉 You have deployed an instance of Prometheus! 🚀

With Bitovi’s Deploy Prometheus action, you now have a powerful tracking machine to keep an eye on all the metrics you love!

A Note on Platform Engineering with GitHub

Did you know that with this Deploy Prometheus GitHub Action, you’re already using GitHub as a Platform Engineering platform?

Want to know how to get more value from GitHub as a Platform Engineering platform? Bitovi loves working with Clients and Partners to build GitHub Actions for their products and services. Reach out to schedule a free Platform Engineering consultation!

We’re looking forward to a bright future helping you with GitHub Actions and Platform Engineering.

Need Help?

Got a quick question or want to meet the team? Drop into Bitovi’s Community Discord, and talk to us in the devops forum.