Security /info/alibaba/ssl

Monitor SSL Certificates with Function Compute

Date Created: June 23, 2018
Last Update: June 24, 2018

Table of Contents

Introduction
Download
SSL Certificate Status Report
Program Configuration Parameters
Retrieving an SSL Certificate from an Internet Host
Checking an SSL Certificate
Creating the Function
Function Compute Authorization
Function Compute Debugging
Automatic Updates with FCLI
Remote Execution with FCLI
Create Time Trigger with FCLI

Introduction

After working with Alibaba Cloud services, you may find that you have dozens or hundreds of services that depend on SSL certificates. This can include ECS instances, websites, API Gateway services, Function Compute functions, and CDN endpoints. This article will discuss how to monitor SSL certificates and send emails on the status of your SSL certificates. We will use Function Compute and DirectMail for automation, monitoring and reporting.

This article is an add-on to my series of using Let's Encrypt SSL certificates. These certificates expire after 90 days, therefore, keeping track of certificate expiration dates is very important. However, tracking services that use SSL certificates is boring, tedious and we often forget about them. In another article we will develop software that can automatically renew Let's Encrypt SSL certificates.

The goal of this article is to show you how to do this. The code is not production quality, rather it is education quality. All software that you plan to deploy for production purposes needs to be reviewed and tested for quality and suitability for your requirements.

This article assumes that you have a basic understanding of Alibaba Function Compute and DirectMail. If not, I have written other articles on this website that will help you understand these services. The last part of this article shows how to speed up testing and updates using the Alibaba FCLI command line program.

Additional NeoPrime Documents:


Download

Last Update: June 24, 2018
Requirements: Python 3.6 or newer (Python 2 is not supported)
Platforms: Tested on Windows 10 and Function Compute
Download: SSL Check - Python 3 (Zip - 10 KB)
Note: Antivirus software will complain about this download because it is a zip file with Python source code.

SSL Certificate Status Report

Let's start by reviewing what this program generates. The following table is generated by this code and emailed to an email address that you specify. The report has 5 columns of information. Each row describes the status for one hostname. Notice that the last line is in yellow. This line includes the error message that the host is unreachable. I included the hostname "bad.neoprime.xyz" to generate this error on purpose.

The key columns are the "Status" and "Expires". As long as the status shows OK, all is good. Otherwise a message will be displayed such as "Expired" and "Time to Renew".



NeoPrime SSL Certificate Status Report

Sat, 23 Jun 2018 18:24:41 GMT

Domain Name Status Expires Issuer Subject Alt Names
neoprime.xyz OK 2018-08-28 05:40:35 Let's Encrypt Authority X3 api.neoprime.xyz, neoprime.xyz, www.neoprime.xyz
api.neoprime.xyz OK 2018-09-13 01:34:43 Let's Encrypt Authority X3 api.neoprime.xyz, cdn.neoprime.xyz, neoprime.xyz, www.neoprime.xyz
cdn.neoprime.xyz OK 2018-08-10 23:22:17 Let's Encrypt Authority X3 cdn.neoprime.xyz, neoprime.xyz, www.neoprime.xyz
www.neoprime.xyz OK 2018-08-28 05:40:35 Let's Encrypt Authority X3 api.neoprime.xyz, neoprime.xyz, www.neoprime.xyz
bad.neoprime.xyz [Errno 11001] getaddrinfo failed

Program Configuration Parameters

During initial testing, run the program from the command line. This software will extract your Alibaba credentials from your credentials file. Make sure that your credentials have rights to call DirectMail. Once you are ready to switch to Function Compute, change this line:

g_program_mode = PROGRAM_MODE_ACS_FUNC


PROGRAM_MODE_CMDLINE = 0	# The program operates from the command line
PROGRAM_MODE_ACS_FUNC = 1	# The program operates as a Alibaba Cloud Function Compute function

g_program_mode = PROGRAM_MODE_CMDLINE

g_days_left = 14		# Warn if a certificate will expire in less than this number of days

g_no_send = False		# if set, don't actually send an email. This is used for debugging

g_only_send_notices = False	# If set, only send emails if a certificate will expire soon or on error

g_email_required = False	# This is set during processing if a warning or error was detected

Configure the hostnames that you want to monitor. In this example, we are monitoring four hostnames.

g_hostnames = [
	"neoprime.xyz",
	"api.neoprime.xyz",
	"cdn.neoprime.xyz",
	"www.neoprime.xyz",
	]

Configure the report subject and send to email address.

email_params['Subject'] = 'NeoPrime SSL Cerificate Status Report'
email_params['To'] = 'someone@example.com'

Configure the Alibaba Cloud DirectMail account parameters. This example uses Singapore for the region.


# From the DirectMail Console
dm_account['Debug'] = 0
dm_account['Account'] = 'sender_address_from_directmail_console'
dm_account['Alias'] = 'my_alias_name_such_as_NeoPrime'
dm_account['host'] = "dm.ap-southeast-1.aliyuncs.com"
dm_account['url'] = "https://dm.ap-southeast-1.aliyuncs.com/"

Retrieving an SSL Certificate from an Internet Host

This is the function that returns an SSL certificate from an Internet host. This SSL certificate contains information about the certificate such as the domain name, expiration date, etc.

def ssl_get_cert(hostname):
	""" This function returns an SSL certificate from a host """

	context = ssl.create_default_context()

	conn = context.wrap_socket(
		socket.socket(socket.AF_INET),
		server_hostname=hostname)

	# 3 second timeout because Function Compute has runtime limitations
	conn.settimeout(3.0)

	try:
		conn.connect((hostname, 443))
	except Exception as ex:
		print("{}: Exception: {}".format(hostname, ex), file=sys.stderr)
		return False, str(ex)

	host_ssl_info = conn.getpeercert()

	return host_ssl_info, ''

Checking an SSL Certificate

This function loops thru each hostname and checks the SSL certificate for its expiration date (notAfter). This function also extracts other information from the SSL certificate such as the Issuer and Subject Alt Names (SAN). For each hostname, a row is added to the HTML table with "add_row()". This function returns the HTML body that we have built. This HTML body will be part of the email message that is sent.

def process_hostnames(msg_body, hostnames):
	""" Process the SSL certificate for each hostname """

	# pylint: disable=global-statement
	global g_email_required

	ssl_date_fmt = r'%b %d %H:%M:%S %Y %Z'

	for host in hostnames:
		f_expired = False

		print('Processing host:', host)

		ssl_info, err = get_ssl_info(host)

		if ssl_info is False:
			msg_body = add_row(msg_body, host, err, '', '', '', True)
			g_email_required = True
			continue

		#print(ssl_info)

		issuerName = get_ssl_issuer_name(ssl_info)

		altNames = get_ssl_subject_alt_names(ssl_info)

		l_expires = datetime.datetime.strptime(ssl_info['notAfter'], ssl_date_fmt)

		remaining = l_expires - datetime.datetime.utcnow()

		if remaining < datetime.timedelta(days=0):
			# cert has already expired - uhoh!
			cert_status = "Expired"
			f_expired = True
			g_email_required = True
		elif remaining < datetime.timedelta(days=g_days_left):
			# expires sooner than the buffer
			cert_status = "Time to Renew"
			f_expired = True
			g_email_required = True
		else:
			# everything is fine
			cert_status = "OK"
			f_expired = False

		msg_body = add_row(msg_body, host, cert_status, str(l_expires), issuerName, altNames, f_expired)

	return msg_body

Complete Example

############################################################
# Version 0.90
# Date Created: 2018-06-11
# Last Update:  2018-06-23
# https://www.neoprime.io
# Copyright (c) 2018, NeoPrime, LLC
# Author: John Hanley
############################################################

""" Alibaba Cloud Function Compute Example """

import	sys
import	datetime
import	socket
import	json
import	ssl
import	time
import	myemail
import	myhtml

PROGRAM_MODE_CMDLINE = 0	# The program operates from the command line
PROGRAM_MODE_ACS_FUNC = 1	# The program operates as a Alibaba Cloud Function Compute function

g_program_mode = PROGRAM_MODE_ACS_FUNC
#g_program_mode = PROGRAM_MODE_CMDLINE

g_days_left = 14		# Warn if a certificate will expire in less than this number of days

g_no_send = False		# if set, don't actually send an email. This is used for debugging

g_only_send_notices = False	# If set, only send emails if a certificate will expire soon or on error

g_email_required = False	# This is set during processing if a warning or error was detected

g_hostnames = [
	"neoprime.xyz",
	"api.neoprime.xyz",
	"cdn.neoprime.xyz",
	"www.neoprime.xyz",
	]

email_params = {
	'To': '',
	'Subject': '',
	'Body': '',
	'BodyText': ''
}

email_params['Subject'] = 'NeoPrime SSL Certificate Status Report'
email_params['To'] = 'someone@example.com'

dm_account = {
	'Debug': 0,	# Debug flag
	'Account': '',	# DirectMail account
	'Alias': '',	# DirectMail alias
	'host': '',	# HTTP Host header
	'url': ''	# URL for POST
	}

# From the DirectMail Console
dm_account['Debug'] = 0
dm_account['Account'] = ''
dm_account['Alias'] = ''
dm_account['host'] = "dm.ap-southeast-1.aliyuncs.com"
dm_account['url'] = "https://dm.ap-southeast-1.aliyuncs.com/"

def ssl_get_cert(hostname):
	""" This function returns an SSL certificate from a host """

	context = ssl.create_default_context()

	conn = context.wrap_socket(
		socket.socket(socket.AF_INET),
		server_hostname=hostname)

	# 3 second timeout because Function Compute has runtime limitations
	conn.settimeout(3.0)

	try:
		conn.connect((hostname, 443))
	except Exception as ex:
		print("{}: Exception: {}".format(hostname, ex), file=sys.stderr)
		return False, str(ex)

	host_ssl_info = conn.getpeercert()

	return host_ssl_info, ''

def add_row(body, domain, status, expires, issuerName, names, flag_hl):
	""" Add a row to the HTML table """

	#build the url
	url = '<a href="https://' + domain + '">' + domain + '</a>'

	# begin a new table row
	if flag_hl is False:
		body += '<tr>\n'
	else:
		body += '<tr bgcolor="#FFFF00">\n'	# yellow

	body += '<td>' + url + '</td>\n'
	body += '<td>' + status + '</td>\n'
	body += '<td>' + expires + '</td>\n'
	body += '<td>' + issuerName + '</td>\n'
	body += '<td>' + names + '</td>\n'

	return body + '</tr>\n'

# Email specific

def send(account, credentials, params):
	""" email send function """

	# pylint: disable=global-statement
	global g_only_send_notices
	global g_email_required

	# If set, only send emails if a certificate will expire soon or on error
	if g_only_send_notices is True:
		if g_email_required is False:
			print('')
			print('All hosts have valid certificates')
			print('Sending an email is not required')
			return

	myemail.sendEmail(credentials, account, params, g_no_send)

def get_ssl_info(host):
	""" This function retrieves the SSL certificate for host """
	# If we receive an error, retry up to three times waiting 10 seconds each time.

	retry = 0
	err = ''

	while retry < 3:
		ssl_info, err = ssl_get_cert(host)

		if ssl_info is not False:
			return ssl_info, ''

		retry += 1
		print('    retrying ...')
		time.sleep(10)

	return False, err

def get_ssl_issuer_name(ssl_info):
	""" Return the IssuerName from the SSL certificate """

	issuerName = ''

	issuer = ssl_info['issuer']

	# pylint: disable=line-too-long
	# issuer looks like this:
	# This is a set of a set of a set of key / value pairs.
	# ((('countryName', 'US'),), (('organizationName', "Let's Encrypt"),), (('commonName', "Let's Encrypt Authority X3"),))

	for item in issuer:
		# item will look like this as it goes thru the issuer set
		# Note that this is a set of a set
		#
		# (('countryName', 'US'),)
		# (('organizationName', "Let's Encrypt"),)
		# (('commonName', "Let's Encrypt Authority X3"),)

		s = item[0]

		# s will look like this as it goes thru the isser set
		# Note that this is now a set
		#
		# ('countryName', 'US')
		# ('organizationName', "Let's Encrypt")
		# ('commonName', "Let's Encrypt Authority X3")

		# break the set into "key" and "value" pairs
		k = s[0]
		v = s[1]

		if k == 'organizationName':
			if v != '':
				issuerName = v
				continue

		if k == 'commonName':
			if v != '':
				issuerName = v

	return issuerName

def get_ssl_subject_alt_names(ssl_info):
	""" Return the Subject Alt Names """

	altNames = ''

	subjectAltNames = ssl_info['subjectAltName']

	index = 0
	for item in subjectAltNames:
		altNames += item[1]
		index += 1

		if index < len(subjectAltNames):
			altNames += ', '

	return altNames

def process_hostnames(msg_body, hostnames):
	""" Process the SSL certificate for each hostname """

	# pylint: disable=global-statement
	global g_email_required

	ssl_date_fmt = r'%b %d %H:%M:%S %Y %Z'

	for host in hostnames:
		f_expired = False

		print('Processing host:', host)

		ssl_info, err = get_ssl_info(host)

		if ssl_info is False:
			msg_body = add_row(msg_body, host, err, '', '', '', True)
			g_email_required = True
			continue

		#print(ssl_info)

		issuerName = get_ssl_issuer_name(ssl_info)

		altNames = get_ssl_subject_alt_names(ssl_info)

		l_expires = datetime.datetime.strptime(ssl_info['notAfter'], ssl_date_fmt)

		remaining = l_expires - datetime.datetime.utcnow()

		if remaining < datetime.timedelta(days=0):
			# cert has already expired - uhoh!
			cert_status = "Expired"
			f_expired = True
			g_email_required = True
		elif remaining < datetime.timedelta(days=g_days_left):
			# expires sooner than the buffer
			cert_status = "Time to Renew"
			f_expired = True
			g_email_required = True
		else:
			# everything is fine
			cert_status = "OK"
			f_expired = False

		msg_body = add_row(msg_body, host, cert_status, str(l_expires), issuerName, altNames, f_expired)

	return msg_body

def main_cmdline():
	""" This is the main function """

	# My library for processing Alibaba Cloud Services (ACS) credentials
	# This library is only used when running from the desktop and not from the cloud
	import mycred_acs

	# Load the Alibaba Cloud Credentials (AccessKey)
	cred = mycred_acs.LoadCredentials()

	if cred is False:
		print('Error: Cannot load credentials', file=sys.stderr)
		sys.exit(1)

	now = datetime.datetime.utcnow()
	date = now.strftime("%a, %d %b %Y %H:%M:%S GMT")

	msg_body = ''

	msg_body = myhtml.build_body_top()
	msg_body += '<h4>NeoPrime SSL Cerificate Status Report</h4>'
	msg_body += date + '<br />'
	msg_body += '<br />'
	msg_body = myhtml.build_table_top(msg_body)

	#
	# This is where the SSL processing happens
	#
	msg_body = process_hostnames(msg_body, g_hostnames)

	msg_body = myhtml.build_table_bottom(msg_body)
	msg_body = myhtml.build_body_bottom(msg_body)

	email_params['Body'] = msg_body
	email_params['BodyText'] = ''

	#print(msg_body)

	send(dm_account, cred, email_params)

def main_acs_func(event, context):
	""" This is the main function """

	cred = {
		'accessKeyId': '',
		'accessKeySecret': '',
		'securityToken': '',
		'Region': ''
	}

	cred['accessKeyId'] = context.credentials.accessKeyId
	cred['accessKeySecret'] = context.credentials.accessKeySecret
	cred['securityToken'] = context.credentials.securityToken

	now = datetime.datetime.utcnow()
	date = now.strftime("%a, %d %b %Y %H:%M:%S GMT")

	msg_body = ''

	msg_body = myhtml.build_body_top()
	msg_body += '<h4>NeoPrime SSL Cerificate Status Report</h4>'
	msg_body += date + '<br />'
	msg_body += '<br />'
	msg_body = myhtml.build_table_top(msg_body)

	#
	# This is where the SSL processing happens
	#
	msg_body = process_hostnames(msg_body, g_hostnames)

	msg_body = myhtml.build_table_bottom(msg_body)
	msg_body = myhtml.build_body_bottom(msg_body)

	email_params['Body'] = msg_body
	email_params['BodyText'] = ''

	#print(msg_body)

	send(dm_account, cred, email_params)

	return msg_body

def handler(event, context):
	""" This is the Function Compute entry point """

	body = ""

	body = main_acs_func(event, context)

	res = {
		'isBase64Encoded': False,
		'statusCode': 200,
		'headers': {
			'content-type' : 'text/html'
		},
		'body': body
	}

	return json.dumps(res)

# Main Program
if g_program_mode == PROGRAM_MODE_CMDLINE:
	main_cmdline()

Creating the Function

The following screenshot shows the parameters to set for your function. The second screenshot shows the parameters to set for the "Time Trigger" as this function will be call periodically. I set the Time Trigger to once per day at 8 AM PST (16:00 GMT).

Security

Security

Function Compute Authorization

Function Compute requires permission to send email using DirectMail. There are two methods to do this. Hard code your credentials in the source code (very bad idea) or use RAM (Resource Access Manager) to create a "role" that you assign to your Function Compute service (very good idea).

Steps to create an RAM role for Function Compute:

  1. Login to the Alibaba Console
  2. Go to Resource Access Manager
  3. Click on Roles
  4. Click Create Role button
  5. Select Service Role
  6. Select FC Function Compute
  7. Enter a role name and description
  8. Click Create

This role is created but no permissions have been granted to the role.

Steps to grant permissions (authorize) to a role:

  1. Click Authorize button
  2. Click Edit Authorization Policy button
  3. In the Search Keywords box enter the word "Direct"
  4. Select AliyunDirectMailFullAccess
  5. Click the right arrow button to copy the policy to the right side
  6. Click OK button

An important concept with Function Compute and RAM Roles, is that roles are assigned to Function Compute Services. All functions under a service inherit this role. This means that if you have a Function Compute service with several functions, the RAM Role will need the sum of the required permissions for each functions. If you need tigher security, create separate services based upon role permissions.

The RAM Policy will create a JSON document that describes the granted permissions. In this case the role is granting all actions that start with dm (Action: dm:*) on all resources (Resource: *).

{
  "Version": "1",
  "Statement": [
    {
      "Action": "dm:*",
      "Resource": "*",
      "Effect": "Allow"
    }
  ]
}

An often overlooked component of assigning a RAM Role to a service is that the service requires permissions to assume that role. A RAM Role has two components, the STS (Security Token Service) permissions to assume a role and the role permissions.

This JSON desribes the permissions that the Function Compute service itself has to assume a role via the AssumeRole action. Notice the service name "fc.aliyncs.com" and the Action "sts:AssumeRole".

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "fc.aliyuncs.com"
        ]
      }
    }
  ],
  "Version": "1"
}

Function Compute Debugging

Manually invoke your function in the Alibaba Function Compute Console. If you see the following type of error, then you forgot to assign a RAM Role with DirectMail permissions to your service.

Error Code: 404
Error: {
	"Recommend":"https://error-center.aliyun.com/status/search?Keyword=InvalidAccessKeyId.NotFound&source=PopGw",
	"Message":"Specified access key is not found.",
	"RequestId":"31BEFC34-DD4F-4916-927A-773A7C4F26C5",
	"HostId":"dm.ap-southeast-1.aliyuncs.com",
	"Code":"InvalidAccessKeyId.NotFound"
}


Automatic Updates with FCLI

Alibaba FCLI Documentation

The Function Compute example consists of serveral files. Rather than going to the Alibaba Console and uploading the code changes, I like to use the FCLI command line program to update my Function Compute functions from the command line. The following is the Windows Cmd Prompt batch script that I use.

This command creates a new package called index.zip and adds the source files. Then using fcli.exe the package is uploaded to Function Compute. Very easy and straightforward. Another example of good DevOps - remove as many manual steps as possible.

del index.zip
pkzipc -add index.zip index.py myemail.py myhtml.py

fcli function update --code-file index.zip -s service_name -f function_name

Creating the function with FCLI

This command with create and upload the code in one step. You will need to manually create the Time Trigger for the function in the Alibaba Console. This example uses the service name "ssl" and the function name "ssl_check".

fcli function create --code-file index.zip -t python3 -h index.handler -s ssl -f ssl_check

Remote Execution with FCLI

This command "invokes" the function remotely. This is a convenient method for testing.

fcli function invoke -s service_name -f function_name

Create Time Trigger with FCLI

This command creates a time trigger for Function Compute with FCLI. There are two components: the command and the yaml configuration file.

triggerConfig:
    payload: ""
    cronExpression: "0 0/60 * * * *"
    enable: true

fcli trigger create -t OncePerHour -s ssl -f ssl_check -c TimeTrigger.yaml --type timer

Additional Ideas

You could change the Time Trigger to invoke this function more often such as every 15 minutes. Then change the source code parameter "g_only_send_notices = True" to only receive an email if there is a problem. This would be a service check feature that can report to you if any of the HTTPS services are failing.

Another idea is to create multiple functions in different regions around the world to detect problems that regional customers might experience.

You could even add code to reboot an ECS instance that was not responding.

Do not specify too many hostnames to check. Function Compute has a max time limit of 300 seconds. This will limit the function to about 10 hostnames, allowing for failure timeouts of 30 seconds. If you reduce the failure timeout then you can process more hostnames with each funcion. You can also create multiple functions in Function Compute for processing many hostnames. If you do not retry failures, then the limit is around 100 hostnames per function. The Alibaba Console has an "Invoke" button to manually invoke a function. Near the bottom of the console window will be stats on how long the function executed. This can help you adjust the number of hosts per function.




15220 Main Street, Bellevue, WA 98007
T: 425-528-8500 - F: 425-528-8550 - E: neoprime@neoprime.io

Copyright 2018 NeoPrime LLC