Extending Ansible: Filter, Lookup, and Callback Plugins
Article 11 covered modules — code that runs on the host to do work. This article is the other direction of extension: plugins — code that runs on the control node to extend Ansible itself. Understand both and you have the full picture of how to customize Ansible.
Module vs plugin: the core difference
This is an important distinction, and an easy one to confuse:
MODULE PLUGIN
─────────────────────────────────────────────────────────────
runs ON THE TARGET HOST (over SSH) runs ON THE CONTROL NODE
does work: install pkgs, edit files extends Ansible itself
shipped to host (AnsiballZ — Art.1) runs locally in the ansible process
invoked as a task used in Jinja2 / inventory / output...
Put another way: module = "do something on the target machine"; plugin = "change how Ansible operates". You've already used plugins without noticing: the | default filter (Article 6) is a filter plugin, the aws_ec2 dynamic inventory (Article 3) is an inventory plugin, and the SSH connection is a connection plugin.
Common plugin types
filter transform data in Jinja2 ( {{ x | myfilter }} )
lookup fetch data from an EXTERNAL SOURCE ( {{ lookup('file', '/path') }} )
callback customize OUTPUT/logging (change how results are displayed)
connection how to connect to a host (ssh, docker, local, winrm...)
inventory inventory source (aws_ec2, k8s... — Article 3)
test a test in Jinja2 ( {{ x is mytest }} )
We'll dig into the three you'll most often write yourself: filter, lookup, callback.
Filter plugin: transforming data in Jinja2
A filter (| something) transforms a value in a template/expression (Article 6). Beyond the built-in filters (default, upper, length...), you can write your own. A filter plugin is a Python file in the filter_plugins/ directory:
# filter_plugins/custom_filters.py
def shout(value):
return str(value).upper() + "!"
def env_prefix(value, env="dev"):
return f"{env}-{value}"
class FilterModule(object):
def filters(self):
return {
'shout': shout,
'env_prefix': env_prefix,
}
Each filter is just a Python function (it takes the value on the left of | and returns a new value). The FilterModule.filters() class registers filter name → function. Use it right away in a playbook:
- hosts: localhost
vars:
name: "kkloud"
tasks:
- ansible.builtin.debug:
msg: "{{ name | shout }} / {{ name | env_prefix('prod') }}"
"msg": "KKLOUD! / prod-kkloud"
name | shout → KKLOUD!; name | env_prefix('prod') → prod-kkloud (the argument 'prod' is passed into the function). Filters run on the control node at render time — exactly the plugin definition. Custom filters are very handy for handling complex data cleanly in templates instead of cramming logic into the playbook.
Lookup plugin: fetching external data
A lookup pulls data from an external source into the playbook, used via lookup('name', ...). It's the standard way to read files, environment variables, secrets from an external vault, and so on. A few built-in lookups:
- ansible.builtin.debug:
msg: "{{ lookup('ansible.builtin.file', '/etc/hostname') }}" # file contents
- ansible.builtin.debug:
msg: "{{ lookup('ansible.builtin.env', 'HOME') }}" # environment variable
- ansible.builtin.debug:
msg: "{{ lookup('ansible.builtin.pipe', 'date +%F') }}" # command output
Lookups are especially powerful for fetching secrets from a secret manager (recall Article 10 — a replacement for static Vault at scale):
db_pass: "{{ lookup('community.hashi_vault.hashi_vault', 'secret/db:password') }}"
api_key: "{{ lookup('amazon.aws.aws_secret', 'prod/api_key') }}"
You can write your own lookup plugin (in lookup_plugins/) much like a filter — subclass LookupBase and implement run(), returning a list. Useful when you need to integrate an internal data source.
Note: lookups run on the control node, every time they're referenced (no caching by default). Don't put a heavy lookup (an API call) inside a large loop — it will run repeatedly.
Callback plugin: customizing output
A callback plugin changes how Ansible displays/records results — reacting to events (task started, task finished, play ended). This is how you change the look of output or integrate logging/monitoring. Enable a built-in callback in ansible.cfg:
[defaults]
stdout_callback = yaml # YAML output, more readable than the default
callbacks_enabled = profile_tasks, timer # time each task + the total
stdout_callback = yaml— displays results as YAML (more readable than JSON crammed onto one line).profile_tasks— prints the time for each task (find slow tasks — related to Article 13).timer— prints the total playbook time.
Custom callbacks are useful for: sending results to Slack, writing JSON logs for a monitoring system, formatting output for CI. You subclass CallbackBase and implement hooks like v2_runner_on_ok, v2_runner_on_failed...
Where to put plugins
Like modules (Article 11), plugins are discovered by directory convention next to the playbook, one directory per type:
filter_plugins/ filter plugin
lookup_plugins/ lookup plugin
callback_plugins/ callback plugin
library/ module (Article 11)
Or declare the path in ansible.cfg (filter_plugins = ...), or — the cleanest at scale — package them into a collection (Article 9): a collection has plugins/filter/, plugins/lookup/, plugins/modules/... directories that gather all your extensions in one place to share.
Wrap-up
A plugin extends Ansible itself (runs on the control node), unlike a module (does work on the host). The three you'll most often write: filter transforms data in Jinja2 (| myfilter, the FilterModule class); lookup fetches external data (lookup('file'/'env'/secret-manager...)); callback customizes output/logging (stdout_callback, profile_tasks). Put plugins in filter_plugins/, lookup_plugins/, callback_plugins/ next to the playbook, or package them into a collection. Together with modules in Article 11, plugins let you customize Ansible at every layer.
Now that we can write and extend, let's move to real-world operations at scale. Article 13: optimization and execution strategy — running Ansible fast and safely across hundreds of hosts.