Recommended deployment architecture
We recommend deploying the control plane with 6 server nodes within the Consul cluster. The 6 server nodes should be evenly distributed across three Availability zones inside of AWS and Consul Autopilot should be configured to enable the Redundancy Zone feature with each AWS Availability Zone being mapped to a Consul Redundancy Zone. The Redundancy Zones feature will have Autopilot maintain 1 voting server per zone with the remaining server in that zone left as non-voting. These non-voting servers act to enhance read scalability, and can be promoted to voting servers in the case of a voting server failure.
Note
This section will describe the architecture components and best practices when it comes to deploying Consul on VMs within AWS for a production environment. By combining the security and high availability services of AWS with the following recommended architecture, this will ensure availability, scalability, and security of the Consul control and data plane.
Component compatibility
| Component Name | Supported Implementation | Technologies and Versions |
|---|---|---|
| Operating System | EC2 | Ubuntu 22.04; RedHat Enterprise Linux 9 |
| Consul Enterprise Version | Linux Binary | Latest Consul Enterprise version - consul_1.16.X+ent |
| Object Storage | S3 compatible | AWS S3 - Consul Snapshot backups |
| Key Management | KMS | AWS KMS - Initial secrets and certs storage |
Deployment sequence
The infrastructure code written in Terraform comprises a multi step deployment. The first step is creating the initial resources to set up the infrastructure required to support Consul Enterprise on AWS. The second step is the deployment of Consul Enterprise onto VMs in various configurations (server, agent, gateway).
This solution design guide will focus on two separate terraform modules that work independently or in combination.
- Terraform-aws-consul-prerequisites: Module is intended to assist with the provisioning of prerequisite resources that are required for a Consul Enterprise on AWS (EC2) deployment. This is an optional module or can be utilized or tweaked to meet the organization security standards.
- Terraform-aws-consul: Module deploys Consul Enterprise in AWS.
Note: you can refer to the Terraform Modules section for the granular resources details.
Consul agent configuration
You can create one or more configuration files to configure the Consul agent. The configuration files are formatted as HCL or JSON. The solution design guide recommends HCL and will deploy server configuration in HCL.
The below configuration is required to start the Consul agents in server (server.hcl) and client (client.hcl) mode respectively.
Consul server example configuration
The following is an example Consul server agent configuration.
server.hcl
data_dir = "/opt/consul"
log_level = "${consul_log_level}"
datacenter = "${consul_datacenter}"
encrypt = "${gossip_encryption_key}"
encrypt_verify_incoming = true
encrypt_verify_outgoing = true
server = true
license_path = "${consul_license_path}"
# Configure Redundancy Zones
autopilot {
redundancy_zone_tag = "zone"
}
node_meta {
zone = "${consul_server_availability_zone}"
}
# Enable ACLs
acl {
enabled = true
default_policy = "deny"
}
client_addr = "0.0.0.0"
ports = {
"grpc" = -1
"grpc_tls" = 8503
"http" = -1
"https" = 8501
}
enable_central_service_config = true
advertise_addr = "$INTERNAL_IP_ADDRESS"
retry_join = ["provider=aws tag_key=Environment-Name tag_value=primary-consul"]
# TLS config
tls {
defaults {
verify_incoming = false
verify_outgoing = true
ca_file = "/etc/consul.d/tls/consul-ca.pem"
ca_path = "/etc/consul.d/tls"
cert_file = "/etc/consul.d/tls/consul-cert.pem"
key_file = "/etc/consul.d/tls/consul-key.pem"
}
# overrides tls.defaults path, if specified.
https {
verify_incoming = true
verify_outgoing = true
}
grpc {
verify_incoming = false
}
internal_rpc {
verify_server_hostname = true
}
auto_encrypt {
allow_tls = true
}
telemetry {
prometheus_retention_time = "480h",
disable_hostname = true
metrics_path = "/v1/agent/metrics"
}
performance {
raft_multiplier = 1
}
# Server performance config
limits {
rpc_max_conns_per_client = 100
http_max_conns_per_client = 200
}
ui_config {
enabled = true
}
Consul client example configuration
The following is an example Consul client agent configuration.
client.hcl
data_dir = "/opt/consul"
node_name = "consul-client-`hostname`"
log_level = "${consul_log_level}"
datacenter = "${consul_datacenter}"
encrypt = "${gossip_encryption_key}"
server = false
client_addr = "0.0.0.0"
ports = {
"grpc" = -1
"grpc_tls" = 8503
"http" = -1
"https" = 8501
}
advertise_addr = "$INTERNAL_IP_ADDRESS"
retry_join = ["provider=aws tag_key=Environment-Name tag_value=primary-consul"]
# TLS config
tls {
defaults {
verify_incoming = false
verify_outgoing = true
ca_file = "/etc/consul.d/tls/consul-ca.pem"
ca_path = "/etc/consul.d/tls"
cert_file = "/etc/consul.d/tls/consul-cert.pem"
key_file = "/etc/consul.d/tls/consul-key.pem"
}
# overrides tls.defaults path, if specified.
https {
verify_incoming = true
verify_outgoing = true
}
grpc {
verify_incoming = false
}
internal_rpc {
verify_server_hostname = true
}
auto_encrypt {
allow_tls = true
}
}
#Telemetry
telemetry {
prometheus_retention_time = "480h",
disable_hostname = true
}