python-How to check website or url health periodically and automatically by using python script?
1. Purpose
In this post, I would demo how to create a python script to check a url(website/webservice)’s health periodically and automatically .
2. Environment
- Python 2.x and 3.x
- Linux system
3. The solution
3.1 The environment
Suppose we hava a web service which provides some sort of services to others, now we want it to run always and if it’s not healthy, we should restart the service automatically.
The service URL prefix is:
http://localhost:8080/myservice
And we also have a restart script for this service located at:
/opt/myservice/restart.sh
3.2 Add a health check endpoint for our service
For health check purpose, we should add a dedicated endpoint to be accessed by the health check script, it should be fast , here is our implementation in java language:
@RequestMapping(method = RequestMethod.GET,value = "hello",headers = "Accept=application/json")
public @ResponseBody String hello() {
return "world";
}
It means that if we caccess the below url:
http://localhost:8080/myservice/hello
We should get this result(http response status code equals 200):
world
Otherwise , the status code should not be 200 , just as the picture shows:
It health check fails, the restart script should be run immediately.
3.3 Setup the logger and imports for python script
Before coding the business, we should setup the environment for our script, just as follows:
import time
import os
import logging
from logging.handlers import TimedRotatingFileHandler
#setup the loggers for daily logging file rolling
logger = logging.getLogger()
logger.setLevel(logging.INFO)
file_handler = TimedRotatingFileHandler("healthcheck_myservice.log", when="midnight", interval=1)
file_handler.suffix = "%Y%m%d"
ch = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(formatter)
ch.setFormatter(formatter)
logger.addHandler(file_handler)
# global variables.
THE_URL = "http://localhost:8080/myservice/hello"
RESTART_SH = '/opt/myservice/restart.sh'
SLEEP_INTERVAL = 60
In the above code, we use the logging module in python to setup a daily rolling file handler for our program, it would create a log file everyday with the suffix year-month-day. You can view more information about the TimedRotatingFileHandler from this website.
The parameter when="midnight" in TimedRotatingFileHandler means that it would Roll over at midnight.
3.4 Implement the health check function with python
First, we need a function to check the status of the service, just as follows:
def health_check(url):
try:
logger.info("start access url %s" % (url))
import urllib
a=urllib.urlopen(url) #this is python2 version, the python3 compatible version is listed below
if a:
if a.getcode()==200:
lines = a.readlines()
if lines is not None and len(lines)==1:
if lines[0]=='world':
logger.info("health ok")
return True
else:
logger.info("warning code:"+str(a.getcode()))
except Exception as e:
logger.error('health check failed %s ' % (str(e)))
logger.info("health not ok")
return False
You can see that , the above function would return True only when the service response code is 200 and the responsecontent is world, or else, it would return False to the caller.
For python 2.x , we can use:
import urllib
urllib.urlopen(url)
For python 3.x , you can replace the urllib.urlopen with follows:
import urllib.request
with urllib.request.urlopen(url) as response:
html = response.read()
3.5 Implement the periodically checking logic
Now we should create a function to run the health check periodically and restart the service automatically if the check fails.
def check_periodically(url,failed_action_func):
while True:
try:
time.sleep(SLEEP_INTERVAL)
logger.info("check awake, start check")
if not health_check(url):
failed_action_func()
except Exception as e:
logger.error('check periodically failed %s ' % (str(e)))
pass
def restart_server():
do_script(RESTART_SH)
pass
def do_script(the_scripts):
logger.info("=========start script %s======"%the_scripts)
result = os.popen(the_scripts).read()
logger.info(result)
logger.info("=========end script %s======" % the_scripts)
pass
3.6 The main function
At last, we should provide the main function to execute the whole script:
if __name__ == '__main__':
check_periodically(THE_URL,restart_server)
pass
3.7 Save the script and execute it in background
Then we save the above script as healthcheck_myservice.py, and execute in background as follows:
$ nohup python healthcheck_myservice.py > healthcheck_myservice.out &
Then we can check the log as follows:
$ tail -f healthcheck_myservice.log
We can get these logs:
2021-06-16 15:54:23,775 - root - INFO - health ok
2021-06-16 15:55:23,834 - root - INFO - check awake, start check
2021-06-16 15:55:23,834 - root - INFO - start access url http://localhost:8080/myservice/hello
2021-06-16 15:55:23,839 - root - INFO - health ok
2021-06-16 15:56:23,899 - root - INFO - check awake, start check
2021-06-16 15:56:23,899 - root - INFO - start access url http://localhost:8080/myservice/hello
2021-06-16 15:56:23,906 - root - INFO - health ok
It works!
4. The whole script
#check if my service is running fine, or else, it would restart it
import urllib
import time
import os
import logging
from logging.handlers import TimedRotatingFileHandler
logger = logging.getLogger()
logger.setLevel(logging.INFO)
file_handler = TimedRotatingFileHandler("healthcheck_myservice.log", when="midnight", interval=1)
file_handler.suffix = "%Y%m%d"
ch = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(formatter)
ch.setFormatter(formatter)
logger.addHandler(file_handler)
#logger.addHandler(ch)
THE_URL = "http://localhost:8080/myservice/hello"
RESTART_SH = '/opt/myservice/restart.sh'
SLEEP_INTERVAL = 60
#perform the health check, return True if the url is accessible.
def health_check(url):
try:
logger.info("start access url %s" % (url))
a=urllib.urlopen(url)
if a:
if a.getcode()==200:
lines = a.readlines()
if lines is not None and len(lines)==1:
if lines[0]=='world':
logger.info("health ok")
return True
else:
logger.info("warning code:"+str(a.getcode()))
except Exception as e:
logger.error('health check failed %s ' % (str(e)))
logger.info("health not ok")
return False
#check the health for specified period
def check_periodically(url,failed_action_func):
while True:
try:
time.sleep(SLEEP_INTERVAL)
logger.info("check awake, start check")
if not health_check(url):
failed_action_func()
except Exception as e:
logger.error('check periodically failed %s ' % (str(e)))
pass
def restart_server():
do_script(RESTART_SH)
pass
def do_script(the_scripts):
logger.info("=========start script %s======"%the_scripts)
result = os.popen(the_scripts).read()
logger.info(result)
logger.info("=========end script %s======" % the_scripts)
pass
if __name__ == '__main__':
check_periodically(THE_URL,restart_server)
pass
5. Summary
In this post, I demonstrated how to use python to do health check job in linux.