Security /info/alibaba/ssl

Let's Encrypt Part 3

Date Created: June 12, 2018
Last Update: July 5, 2018

Table of Contents

Introduction
Let's Encrypt API Endpoints
Example code to get the ACME directory
Example code to create a new account

Article Series

NeoPrime - Let's Encrypt Part 1
NeoPrime - Let's Encrypt Part 2
NeoPrime - Let's Encrypt Part 3
NeoPrime - Let's Encrypt Part 4
NeoPrime - Let's Encrypt Part 5

Introduction

In this multipart article, we will learn how to use the Let's Encrypt ACME version 2 API using Python to develop software that can create, install, renew and revoke SSL certificates for Alibaba Cloud. The same principles apply to any computing service that supports X.509 SSL certificates.

In this we will explain about ACME endpoints, how to get the ACME directory, create an ACME account and retrieve your ACME account information.

Let's Encrypt API Endpoints

Let's Encrypt ACME supports two modes by using different endpoints. A production mode that will issue real certificates and is rate limited and a staging mode for testing which issues test certificates. The production endpoint limits the number of requests that you can make per day. While developing software for Let's Encrypt make sure that you are using the staging endpoint otherwise you will reach your API limit and need to wait until the next day to resume testing.


Staging Endpoint:
https://acme-staging-v02.api.letsencrypt.org/directory

Production Endpoint:
https://acme-v02.api.letsencrypt.org/directory

ACME Directory

The first API call should be to get the ACME directory. The directory is a list of URLs that you call for various commands. The response from get drectory is a JSON structure that looks like this:

{
	"LPTIN-Jj4u0": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
	"keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
	"meta": {
		"caaIdentities": [
			"letsencrypt.org"
		],
		"termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
		"website": "https://letsencrypt.org/docs/staging-environment/"
	},
	"newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
	"newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
	"newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
	"revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}

The first line should be ignored. ACME generates a random key and value to dissuade developers from hard coding JSON expectations into their code. The URL provides more information.

Let's look at each of the returned data parts:

keyChange
This URL is used to change the public key associated with the account. This is used to recover from a key comprimise.

meta.caaIdentities
An array of hostnames which the ACME server recognizes as referring to itself for the purposes of CAA record validation. We do not use these records in the example code.

meta.termsOfService
A URL identifying the current terms of service. Take the time to read the referenced document on the Terms of Service. In the example code we set the fields accepting the Terms of Service by default.

meta.website
A URL locating a website providing more information about the ACME server. We do not use this record in the example code.

newAccount
This endpoint is used to create a new account.

newNonce
This is an important API and should be the second call that you make to the ACME API. A nonce is a unique random value that protects agains replay attacks. Each API call (except for directory) requires a unique nonce value.

newOrder
This endpoint is used to request this issuance of an SSL certificate.

revokeCert
This endpoint is used to revoke an existing certificate that was issued by the same account.


Example code to get the ACME directory

This is the simplest ACME API example in the examples package. This example just calls the ACME main endpoint. The returned data is a JSON structure that defines the API endpoints as described above. Review the output from this program and become familiar with the varous URLs as they will be used in most of the follwowing examples.


Source: get_directory.py

""" Let's Encrypt ACME Version 2 Examples - Get Directory"""

# This example will call the ACME API directory and display the returned data
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.1.1

import	sys
import	requests
import	helper

path = 'https://acme-staging-v02.api.letsencrypt.org/directory'

headers = {
	'User-Agent': 'neoprime.io-acme-client/1.0',
	'Accept-Language': 'en'
}

try:
	print('Calling endpoint:', path)
	directory = requests.get(path, headers=headers)
except requests.exceptions.RequestException as error:
	print(error)
	sys.exit(1)

if directory.status_code < 200 or directory.status_code >= 300:
	print('Error calling ACME endpoint:', directory.reason)
	sys.exit(1)

# The output should be json. If not something is wrong
try:
	acme_config = directory.json()
except Exception as ex:
	print("Error: Cannot load returned data:", ex)
	sys.exit(1)

print('')
print('Returned Data:')
print('****************************************')
print(directory.text)

acme_config = directory.json()

print('')
print('Formatted JSON:')
print('****************************************')
helper.print_dict(acme_config, 0)

Example code to create a new account

The next step is to create a new account on the ACME server. This uses the account.key that we create in Part 2. The ACME server does not track information such as company name, etc in the account database. Only a contact email address for notifications.

In this example, change the EmailAddress parameter to be your email address. The example shows how to include more than one email address. The ACME server does not require that you include an email address as this is optional. The ACME server does not verify the email address.

Let's review a couple of key points about the code.


1: Get the ACME directory.

	acme_config = get_directory()


2: Get the "newAccount" URL:

	url = acme_config["newAccount"]


3: Request a nonce for the first ACME API call.
After the first ACME API call, a new nonce in returned in the header "Replay-Nonce" after each ACME API call.

	nonce = requests.head(acme_config['newNonce']).headers['Replay-Nonce']


4: Assemble the HTML headers.
The important item is: Content-Type: application/jose+json

	headers = {
		'User-Agent': 'neoprime.io-acme-client/1.0',
		'Accept-Language': 'en',
		'Content-Type': 'application/jose+json'
	}


5: Assemble the HTML body with the ACME API parameters.
Creating the HTTP body will be covered in detail in the Part 4.

	payload = {}

	payload["termsOfServiceAgreed"] = True
	payload["contact"] = EmailAddresses

	body_top = {
		"alg": "RS256",
		"jwk": myhelper.get_jwk(AccountKeyFile),
		"url": url,
		"nonce": nonce
	}


6: Assemble the HTML body "jose" data structure.
Notice how everything is base64 encoded.

	body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))
	payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

	jose = {
		"protected": body_top_b64,
		"payload": payload_b64,
		"signature": myhelper.b64(signature)
	}


7: Finally, call the ACME API.
This is done with an HTTP POST with a JSON body.

	resp = requests.post(url, json=jose, headers=headers)


8: After the ACME API two items are returned in the HTTP response headers:
Location is the URL for your account.
Replay-Nonce is the "nonce" value for the next ACME API call.

	resp.headers['Location']
	resp.headers['Replay-Nonce']


Most ACME API calls require the HTTP headers include:

Content-Type: application/jose+json

Source: new_account.py

""" Let's Encrypt ACME Version 2 Examples - New Account"""

# This example will call the ACME API directory and create a new account
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.3.2

import	os
import	sys
import	json
import	requests
import	myhelper

# Staging URL
path = 'https://acme-staging-v02.api.letsencrypt.org/directory'

# Production URL
# path = 'https://acme-v02.api.letsencrypt.org/directory'

AccountKeyFile = 'account.key'

EmailAddresses = ['mailto:someone@eexample.com', 'mailto:someone2@eexample.com']

def check_account_key_file():
	""" Verify that the Account Key File exists and prompt to create one if it does not exist """
	if os.path.exists(AccountKeyFile) is not False:
		return True

	print('Error: File does not exist: {0}'.format(AccountKeyFile))

	if myhelper.Confirm('Create new account private key (y/n): ') is False:
		print('Cancelled')
		return False

	myhelper.create_rsa_private_key(AccountKeyFile)

	if os.path.exists(AccountKeyFile) is False:
		print('Error: File does not exist: {0}'.format(AccountKeyFile))
		return False

	return True

def get_directory():
	""" Get the ACME Directory """
	headers = {
		'User-Agent': 'neoprime.io-acme-client/1.0',
		'Accept-Language': 'en',
	}

	try:
		print('Calling endpoint:', path)
		directory = requests.get(path, headers=headers)
	except requests.exceptions.RequestException as error:
		print(error)
		return False

	if directory.status_code < 200 or directory.status_code >= 300:
		print('Error calling ACME endpoint:', directory.reason)
		print(directory.text)
		return False

	# The following statements are to understand the output
	acme_config = directory.json()

	return acme_config

def main():
	""" Main Program Function """
	headers = {
		'User-Agent': 'neoprime.io-acme-client/1.0',
		'Accept-Language': 'en',
		'Content-Type': 'application/jose+json'
	}

	if check_account_key_file() is False:
		sys.exit(1)

	acme_config = get_directory()

	if acme_config is False:
		sys.exit(1)

	url = acme_config["newAccount"]

	# Get the URL for the terms of service
	terms_service = acme_config.get("meta", {}).get("termsOfService", "")
	print('Terms of Service:', terms_service)

	nonce = requests.head(acme_config['newNonce']).headers['Replay-Nonce']
	print('Nonce:', nonce)
	print("")

	# Create the account request
	payload = {}

	if terms_service != "":
		payload["termsOfServiceAgreed"] = True

	payload["contact"] = EmailAddresses

	payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

	body_top = {
		"alg": "RS256",
		"jwk": myhelper.get_jwk(AccountKeyFile),
		"url": url,
		"nonce": nonce
	}

	body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

	data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

	signature = myhelper.sign(data, AccountKeyFile)

	#
	# Create the HTML request body
	#

	jose = {
		"protected": body_top_b64,
		"payload": payload_b64,
		"signature": myhelper.b64(signature)
	}

	try:
		print('Calling endpoint:', url)
		resp = requests.post(url, json=jose, headers=headers)
	except requests.exceptions.RequestException as error:
		resp = error.response
		print(resp)
	except Exception as ex:
		print(ex)
	except BaseException as ex:
		print(ex)

	if resp.status_code < 200 or resp.status_code >= 300:
		print('Error calling ACME endpoint:', resp.reason)
		print('Status Code:', resp.status_code)
		myhelper.process_error_message(resp.text)
		sys.exit(1)

	print('')
	if 'Location' in resp.headers:
		print('Account URL:', resp.headers['Location'])
	else:
		print('Error: Response headers did not contain the header "Location"')

main()

sys.exit(0)


Example code to get account information

Now that we have created an account using our account.key, let's communicate with the ACME server to see what information is stored on the server. This example introduces a configuration file "acme.ini" to contain the configurartion parameters instead of hard coding them into the source code.

Modify acme.ini to include your specific information such as email address.


Source: acme.ini

 
[acme-neoprime]
UserAgent = neoprime.io-acme-client/1.0

# [Required] ACME account key
AccountKeyFile = account.key

# Certifcate Signing Request (CSR)
CSRFile = example.com.csr

ChainFile = example.com.chain.pem

# ACME URL
# Staging URL
# https://acme-staging-v02.api.letsencrypt.org/directory
# Production URL
# https://acme-v02.api.letsencrypt.org/directory

ACMEDirectory = https://acme-staging-v02.api.letsencrypt.org/directory

# Email Addresses so that LetsEncrypt can notify about SSL renewals
Contacts = mailto:example.com;mailto:someone2@example.com

# Preferred Language
Language = en

Source: get_acount_info.py

""" Let's Encrypt ACME Version 2 Examples - Get Account Information """

############################################################
# This example will call the ACME API directory and get the account information
# Reference: https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.7.3.3
#
# This program uses the AccountKeyFile set in acme.ini to return information about the ACME account.
############################################################

import	sys
import	json
import	requests
import	helper
import	myhelper

############################################################
# Start - Global Variables

g_debug = 0

acme_path = ''
AccountKeyFile = ''
EmailAddresses = []
headers = {}

# End - Global Variables
############################################################

############################################################
# Load the configuration from acme.ini
############################################################

def load_acme_parameters(debug=0):
	""" Load the configuration from acme.ini """

	global acme_path
	global AccountKeyFile
	global EmailAddresses
	global headers

	config = myhelper.load_acme_config(filename='acme.ini')

	if debug is not 0:
		print(config.get('acme-neoprime', 'accountkeyfile'))
		print(config.get('acme-neoprime', 'csrfile'))
		print(config.get('acme-neoprime', 'chainfile'))
		print(config.get('acme-neoprime', 'acmedirectory'))
		print(config.get('acme-neoprime', 'contacts'))
		print(config.get('acme-neoprime', 'language'))

	acme_path = config.get('acme-neoprime', 'acmedirectory')

	AccountKeyFile = config.get('acme-neoprime', 'accountkeyfile')

	EmailAddresses = config.get('acme-neoprime', 'contacts').split(';')

	headers['User-Agent'] = config.get('acme-neoprime', 'UserAgent')
	headers['Accept-Language'] = config.get('acme-neoprime', 'language')
	headers['Content-Type'] = 'application/jose+json'

	return config

############################################################
#
############################################################

def get_account_url(url, nonce):
	""" Get the Account URL based upon the account key """

	# Create the account request
	payload = {}

	payload["termsOfServiceAgreed"] = True
	payload["contact"] = EmailAddresses
	payload["onlyReturnExisting"] = True

	payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

	body_top = {
		"alg": "RS256",
		"jwk": myhelper.get_jwk(AccountKeyFile),
		"url": url,
		"nonce": nonce
	}

	body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

	#
	# Create the message digest
	#

	data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

	signature = myhelper.sign(data, AccountKeyFile)

	#
	# Create the HTML request body
	#

	jose = {
		"protected": body_top_b64,
		"payload": payload_b64,
		"signature": myhelper.b64(signature)
	}

	#
	# Make the ACME request
	#

	try:
		print('Calling endpoint:', url)
		resp = requests.post(url, json=jose, headers=headers)
	except requests.exceptions.RequestException as error:
		resp = error.response
		print(resp)
	except Exception as error:
		print(error)

	if resp.status_code < 200 or resp.status_code >= 300:
		print('Error calling ACME endpoint:', resp.reason)
		print('Status Code:', resp.status_code)
		myhelper.process_error_message(resp.text)
		sys.exit(1)

	if 'Location' in resp.headers:
		print('Account URL:', resp.headers['Location'])
	else:
		print('Error: Response headers did not contain the header "Location"')

	# Get the nonce for the next command request
	nonce = resp.headers['Replay-Nonce']

	account_url = resp.headers['Location']

	return nonce, account_url

############################################################
#
############################################################

def get_account_info(nonce, url, location):
	""" Get the Account Information """

	# Create the account request
	payload = {}

	payload_b64 = myhelper.b64(json.dumps(payload).encode("utf8"))

	body_top = {
		"alg": "RS256",
		"kid": location,
		"nonce": nonce,
		"url": location
	}

	body_top_b64 = myhelper.b64(json.dumps(body_top).encode("utf8"))

	#
	# Create the message digest
	#

	data = "{0}.{1}".format(body_top_b64, payload_b64).encode("utf8")

	signature = myhelper.sign(data, AccountKeyFile)

	#
	# Create the HTML request body
	#

	jose = {
		"protected": body_top_b64,
		"payload": payload_b64,
		"signature": myhelper.b64(signature)
	}

	#
	# Make the ACME request
	#

	try:
		print('Calling endpoint:', url)
		resp = requests.post(url, json=jose, headers=headers)
	except requests.exceptions.RequestException as error:
		resp = error.response
		print(resp)
	except Exception as error:
		print(error)

	if resp.status_code < 200 or resp.status_code >= 300:
		print('Error calling ACME endpoint:', resp.reason)
		print('Status Code:', resp.status_code)
		myhelper.process_error_message(resp.text)
		sys.exit(1)

	nonce = resp.headers['Replay-Nonce']

	# resp.text is the returned JSON data describing the account

	return nonce, resp.text

############################################################
#
############################################################

def load_acme_urls(path):
	""" Load the ACME Directory of URLS """
	try:
		print('Calling endpoint:', path)
		resp = requests.get(acme_path, headers=headers)
	except requests.exceptions.RequestException as error:
		print(error)
		sys.exit(1)

	if resp.status_code < 200 or resp.status_code >= 300:
		print('Error calling ACME endpoint:', resp.reason)
		print(resp.text)
		sys.exit(1)

	return resp.json()

############################################################
#
############################################################

def acme_get_nonce(urls):
	""" Get the ACME Nonce that is used for the first request """
	global	headers

	path = urls['newNonce']

	try:
		print('Calling endpoint:', path)
		resp = requests.head(path, headers=headers)
	except requests.exceptions.RequestException as error:
		print(error)
		return False

	if resp.status_code < 200 or resp.status_code >= 300:
		print('Error calling ACME endpoint:', resp.reason)
		print(resp.text)
		return False

	return resp.headers['Replay-Nonce']

############################################################
# Main Program Function
############################################################

def main(debug=0):
	""" Main Program Function """
	acme_urls = load_acme_urls(acme_path)

	url = acme_urls["newAccount"]

	nonce = acme_get_nonce(acme_urls)

	if nonce is False:
		sys.exit(1)

	nonce, account_url = get_account_url(url, nonce)

	# resp is the returned JSON data describing the account
	nonce, resp = get_account_info(nonce, account_url, account_url)

	info = json.loads(resp)

	if debug is not 0:
		print('')
		print('Returned Data:')
		print('##################################################')
		#print(info)
		helper.print_dict(info)
		print('##################################################')

	print('')
	print('ID:        ', info['id'])
	print('Contact:   ', info['contact'])
	print('Initial IP:', info['initialIp'])
	print('Created At:', info['createdAt'])
	print('Status:   ', info['status'])

def is_json(data):
	try:
		json.loads(data)
	except ValueError as e:
		return False
	return True

acme_config = load_acme_parameters(g_debug)

main(g_debug)

In Part 4 we will go deeper into the ACME API and learn how to construct each part of the JSON body, sign the payload and process the results.

Let's Encrypt Part 4.




15220 Main Street, Bellevue, WA 98007
T: 425-528-8500 - F: 425-528-8550 - E: neoprime@neoprime.io

Copyright 2018 NeoPrime LLC