Scaling automation with ansible-pull

Ansible is a wonderful software to automate configure your systems. The default mode of using ansible is Push Model.

That means from your box, and only using ssh + python, you can configure your flee of machines.

Ansible is imperative. That means you define the tasks in your playbooks, roles and they will run in a serial manner on your remote machines either they need to run or not. For that reason ansible seems slow instead of other configuration tools. Ansible runs serial tasks in psedo-parallel mode with the remote servers, to increase the speed. But then, sometimes you need to gather_facts and that would cost in execution of the tasks. There are solutions to cache your facts in a redis (in memory key:value db) but even then you have to find work-around to speed your deployments.

But there is an another way, the Pull Mode!

Useful Reading Materials

to learn more on the subject, you can start reading these two articles on ansible-pull.

• Jan-Piet Mens

• Stavros Korokithakis

Pull Mode

So here how it looks like:

You will first notice, that your ansible repository is moved from you local machine to an online git repository. For me it is a GitLab. As my git repo is private, I have created a Read-Only Deploy Token.

With that scenario, our (ephemeral - or not) VMs will pull their ansible configuration (git repo) and run them locally. I usually build my infrastructure with Terraform by HashiCorp and make advance at the cloud-init to initiate their configuration.

Cloud-init

This is how my user-data.yml is:

...
# Install packages
packages:
  - ansible

# Remove cloud-init
runcmd:
  - ansible-pull -U https://gitlab+deploy-token-XXXXX:YYYYYYYY@gitlab.com/username/myrepo.git 

Playbook

You can either create a playbook with the hostname of the remote server, eg. node1.yml or use the local.yml as the default name.

Here is an example that will also put ansible-pull into a cron entry, that will change the git repo for new changes every 15 minutes:

- hosts: localhost

  tasks:
    - name: Ensure ansible-pull is running every 15 minutes
      cron:
        name: "ansible-pull"
        minute: "15"
        job: "ansible-pull -U https://gitlab+deploy-token-XXXXX:YYYYYYYY@gitlab.com/username/myrepo.git &> /dev/null"

    - name: Create a custom local vimrc file
      lineinfile:
        path: /etc/vim/vimrc.local
        line: 'set modeline'
        create: yes

    - name: Remove "cloud-init" package
      apt:
        name: "cloud-init"
        purge: yes
        state: absent

    - name: Remove useless packages from the cache
      apt:
        autoclean: yes

    - name: Remove dependencies that are no longer required
      apt:
        autoremove: yes

# vim: sts=2 sw=2 ts=2 et