Contents
Troubleshooting Puppet
Puppet normally runs every half hour on each Mageia server to ensure that its configuration is up-to-date. If a server is not updating, here are some things to check.
Has Puppet been explicitly disabled on the server?
Run this command on the server as root to see the disable reason; No such file or directory means it's not disabled:
cat $(puppet agent --configprint agent_disabled_lockfile)
If the disable reason is no longer relevant, enable Puppet again with:
puppet agent --enable
Is the Puppet master running?
On the master machine (duvel as of this writing), run this command as root and look for errors:
systemctl status puppetmaster
When did Puppet last run on the host?
The Puppet master logs all runs performed on a slave. Run this command as root on the puppet master to see the most recent log files for a slave:
ls -ltr /var/lib/puppet/reports/HOSTNAME.mageia.org/ | tail
You can also examine the file, but it's not really intended to be human-readable. Still, if you suspect a problem in one module, you can search the file and see if anything is relevant in that area. If you see less than one log file every half hour, it could be that something is slowing down each run and it simply takes a long time to apply the config. This happened once where puppet was configured to delete old files using tidy() from a directory tree containing hundreds of thousands of files which caused a single Puppet run to take 4 days (it was fixed by removing use of tidy() and deleting /var/lib/puppet/state/state.yaml).
Is the git repo being updated on the Puppet master?
It should be updated every 5 minutes from a cron job. If it stalls, then all nodes will have stale configurations. Run this as root on the puppet master to see if there are issues that may cause git pulls to fail:
cd /etc/puppet && git status
This might show that a file has been locally modified, causing git pull to fail. Editing files on the puppet master is sometimes the expeditious way to test a config change, but it is too easy to forget that it was done and so should be used sparingly. To preserve a local modification but still pick up the latest changes from git, run as root:
cd /etc/puppet && git stash && git pull && git stash pop
If it results in a merge conflict, immediately fix it by editing the merged file (or dropping the local change entirely) because otherwise servers will pick up the merge conflict markers in the Puppet configs and freak out. That's just one reason why you shouldn't generally modify files directly on the puppet master.
Is there an error in the Puppet configs?
Puppet errors are normally e-mailed to sysadmin-reports@ml.mageia.org so check the list archives for complaints. You can also perform some Puppet validation yourself by cloning the puppet repo and running this command (not as root) for the host:
puppet parser validate manifests/nodes/$(hostname --fqdn).pp
This will highlight some classes of errors, but not all.
Finally, you can apply perform a puppet run once in the foreground and see its log messages by running this as root:
puppet agent --test
Note that --test does not mean dry-run; this will apply any changes that are needed. Add --noop to just see what changes would be applied without doing it.
