I’ve been slowly converting my bash scripts to Python. This post details how I solved an issue with executing Python scripts from crontab. Most of my bash scripts are scheduled to run daily using crontab and work fine. The first script I converted to Python was a bash script used to backup the configurations of my Cisco SG300 switches.
The new Python script that uses Netmiko would run fine from a shell but not from crontab. Please note that I use Ubuntu. what follows may differ depending on the Linux distro your using.
The issue was proven to be environment paths required in crontab to run the script. The Python script itself executed fine but as the script was importing the Netmiko library I think thats where it was failing.
Troubleshooting the problem
To troubleshoot the issue a log file was configured in crontab for the entry to kick off the script. This would help provide some visibility into what was happening.
To speed up the testing the script was configured to fire every minute and output to a file /var/log/output.log
* * * * * python /Scripts/cisco_sg300_backup_tftp.py >> /var/log/output.log 2>&1
First check that the script was execute by cron:
grep CRON /var/log/syslog Mar 25 09:46:01 SV-NA1-VIR-22 CRON[8535]: (root) CMD (python /Scripts/cisco_sg300_backup_tftp.py >> /var/log/output.log 2>&1)
The contents of the log file:
more /var/log/output.log
Traceback (most recent call last):
File "/Scripts/cisco_sg300_backup_tftp.py", line 17, in
from netmiko import ConnectHandler
File "build/bdist.linux-x86_64/egg/netmiko/__init__.py", line 8, in
File "build/bdist.linux-x86_64/egg/netmiko/ssh_dispatcher.py", line 4, in
File "build/bdist.linux-x86_64/egg/netmiko/a10/__init__.py", line 2, in
File "build/bdist.linux-x86_64/egg/netmiko/a10/a10_ssh.py", line 4, in
File "build/bdist.linux-x86_64/egg/netmiko/cisco_base_connection.py", line 3, in
File "build/bdist.linux-x86_64/egg/netmiko/base_connection.py", line 24, in
File "build/bdist.linux-x86_64/egg/netmiko/utilities.py", line 8, in
ImportError: No module named serial.tools.list_ports
You can see from output that there was an issue with importing the Netmiko library.
Fixing the problem
Knowing that the script executes manually from a shell I took the environment variables and copied them into crontab.
nickm@SV-NA1-VIR-22:/Scripts$ sudo env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin LANG=en_GB.UTF-8 HOME=/home/nickm LANGUAGE=en_GB:en MAIL=/var/mail/root LOGNAME=root USER=root USERNAME=root SHELL=/bin/bash SUDO_COMMAND=/usr/bin/env SUDO_USER=nickm SUDO_UID=1000 SUDO_GID=1000
Through trial and error I found out which of the above output was required in crontab. For completeness I’ve include the full output of my crontab file:
nickm@SV-NA1-VIR-22:~$ sudo crontab -l PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin SHELL=/bin/bash SUDO_COMMAND=/usr/bin/env HOME=/home/nickm # Edit this file to introduce tasks to be run by cron. # # Each task to run has to be defined through a single line # indicating with different fields when the task will be run # and what command to run for the task # # To define the time you can provide concrete values for # minute (m), hour (h), day of month (dom), month (mon), # and day of week (dow) or use '*' in these fields (for 'any').# # Notice that tasks will be started based on the cron's system # daemon's notion of time and timezones. # # Output of the crontab jobs (including errors) is sent through # email to the user the crontab file belongs to (unless redirected). # # For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow command # # Run the reboot modem script every 15 minutes */15 * * * * /Scripts/reboot_modem.sh # Execute the VMware esxi host backups @ 1pm every day 0 13 * * * /Scripts/esxi_host_backups.sh # Execute the Cisco SG300 backups @ 2pm every day 0 14 * * * python /Scripts/cisco_sg300_backup_tftp.py # Execute the Palo Alto firewall backups @ 3pm every day 0 15 * * * /Scripts/palo_fw_backups.sh # Execute the Juniper SRX firewall backups @ 4pm every day # 0 16 * * * /Scripts/juniper_srx_backups.sh
I’m not sure why ‘HOME=/home/’ had to be set to nickm given that cron is run as root. If someone could enlighten me I’d appreciate it. I can drive linux pretty well but I’m no Linux mechanic.
Summary
Boom! with the environment variables the script now works fine. I’m hopeful executing Python scripts from crontab will work for all my future scripts too.