What is Consul?

HashiCorp Consul is a service networking solution that enables teams to manage and secure network connectivity between services and across on-prem and multi-cloud environments and runtimes. Consul offers service discovery, service mesh, traffic management, and automated updates to network infrastructure devices. You can use these features individually or together in a single Consul deployment.

Consul Concepts

Service discovery

A popular use case for Consul is its service discovery capabilities. Service instances are registered to a central registry that monitors them through health checks. Utilizing a service identity, rather than traditionally an IP address and port, the service catalog can dynamically track and report service instance changes within a service catalog. Consumers then utilize DNS, HTTP APIs and Consul’s CLI to dynamically retrieve other service’s information from the service catalog.

Consul provides a control plane that enables you to register, query, and secure services deployed across your network. The control plane is the part of the network infrastructure that maintains a central registry to track services and their respective IP addresses.

Service mesh

Consul’s service mesh provides secure service-to-service connection, authorization, and encryption within the mesh. Leveraging mutual transport layer security (mTLS), sidecar proxies run in the mesh to ensure service communication security without requiring changes to application logic. Consul service mesh allows you to divide application layer traffic between different subsets of service instances. You can leverage L7 traffic management capabilities to perform complex processes, such as configuring backup services for failover scenarios, canary and A-B testing, blue-green deployments. In addition, service mesh intentions enable secure access control. Intentions control traffic between services at both the networking layer (Layer 4) and the application layer (Layer 7).

When using Consul’s service mesh capabilities, Consul interacts with the data plane, the part of the network infrastructure that processes data requests. Consul dynamically configures sidecar and gateway proxies in the request path, which enables you to authorize service-to-service connections, route requests to healthy service instances, and enforce mTLS encryption without modifying your service’s code. This ensures that communication remains performant and reliable.

Core components

Control plane

The Control plane is responsible for the duties pertaining to health, management and configuration of Consul agents (server, clients) and consul-dataplane. This includes, but is not limited to:

• Cluster state and replication
• Cluster membership between Consul Server and clients.
• Managing data plane proxy configurations:
  • Application proxies
  • Consul Gateway proxies.
• Health checks for Consul Servers, Consul clients, applications and application proxies.
• Cluster peering relationships and which services are shared between peered clusters.

Consul server

Consul servers store all the configuration data for your infrastructure. The Consul server maintains a catalog of each service, with its IP address and port number, allowing services to discover and communicate with one another. The server also provides APIs for managing nodes, services, health checks, and more.

We recommend deploying 6 servers in a cluster. The more servers you deploy, the greater the resilience and availability in the event of a failure. More servers, however, slow down consensus, which is a critical server function that enables Consul to efficiently and effectively process information.

Consul server clusters elect a single server to be the leader through consensus. The leader processes all queries and transactions, which prevents conflicting updates in clusters containing multiple servers.

Servers that are not currently acting as the cluster leader are called followers. Followers forward write requests from client agents to the cluster leader. The leader replicates the write requests to all other servers in the cluster. Replication ensures that if the leader is unavailable, other servers in the cluster can elect another leader without losing any data. Read requests can be serviced by followers depending on the specified consistency mode.

When using redundancy zones, you can configure non-voting members to improve Consul resiliency and to enhance read scalability. If voting followers become unavailable, Consul would attempt to keep server quorum by promoting a non-voting member to become a voting follower. In normal operations, non-voting members enhance read scalability by increasing the number of servers that can service a read request.

In standard deployments, Consul uses a control plane that contains both server agents and client agents. Server agents maintain the service catalog and service mesh, including its security and consistency, while client agents manage communications between service instances, their sidecar proxies, and the servers. This model is optimal for applications deployed on virtual machines or bare metal servers.

You must use this model if you have services running on VMs.

Control Plane for Kubernetes (consul-k8s)

The consul-k8s-control-plane binary includes first-class integrations between Consul and Kubernetes. The project encapsulates multiple use cases such as syncing services, injecting Consul sidecars, and more.

Following features are covered in this solution design guide:

• Catalog Sync: Sync Consul services into first-class Kubernetes services and vice versa. This enables Kubernetes to easily access external services and for non-Kubernetes nodes to easily discover and access Kubernetes services.

• Consul Service Mesh: Run Consul Service Mesh on Kubernetes. This feature injects Envoy sidecars and registers your Pods with Consul.

Dataplane

Consul clients

For standard VM, bare metal, and Nomad workloads, Consul uses an agent-based model that relies on client agents. You must run client agents on every compute node in your datacenter. Consul clients accept service registrations and report node and service health status to the Consul cluster.

Consul-dataplane on kubernetes

Consul on Kubernetes (as of Consul v1.14) relies on consul-dataplane (referenced as Dataplane in the diagram below) which is a lightweight process for managing Envoy proxies. Consul Dataplane removes the need to run client agents on every node in a cluster for service discovery and service mesh. Instead, Consul deploys sidecar proxies that provide lower latency, support additional runtimes, and integrate with cloud infrastructure providers.

Enterprise features

This Hashicorp Validated Design is prescriptive technical content for HashiCorp Enterprise products and solutions.

The following features are available in Consul Enterprise and will be part of our recommendations.

Availability and performance

• Sameness groups: Define partitions and cluster peers as a member of a group to streamline configuration or easily configure failover to members of the group.
• Redundancy zones: Deploy backup voting Consul servers to efficiently improve the Consul fault tolerance.
• Server request rate limit per source IP: Limit gRPC and RPC traffic to servers by source IP addresses.

Scalability

• Read replicas: Deploy non-voting Consul servers to enhance the scalability of the read requests.

Operational excellence

• Autopilot/automated upgrades: Assist in upgrades of the Consul cluster via version updates.
• Automated backups: Configure and assist in the backup of the cluster state.
• Admin partitions: Define administrative boundaries between tenants within a single consul datacenter.
• Namespaces: Define resource boundaries within a single admin partition for further organizational flexibility.

Security

• OIDC auth method: Manage user access to Consul through an OIDC identity provider instead of Consul ACLs directly.
• Audit logging: Capture Consul access and usage patterns by reviewing access to Consul HTTP API.
• FIPS 140-2 compliance: Leverage FIPS builds of Consul Enterprise to ensure Consul deployments are secured with BoringCrypto and CNGCrypto, and compliant with FIPS 140-2.