567a84095e
The BGP controller now supports announcing routes to multiple BGP peers for redundancy and resilience. If one peer fails, route announcements continue to succeed for other healthy peers.
```yaml
bgp:
local_as: 12345
local_ip: 192.168.1.100 # optional
peers:
- peer_ip: 10.10.10.1
peer_as: 6789
communities: # per-peer communities (optional)
- 100:100
- peer_ip: 10.10.10.2
peer_as: 6789
communities:
- 100:101
multi_hop: true # optional, defaults to true for eBGP
communities: # global communities applied to all peers
- 1000:1000
origin: igp
```
```yaml
bgp:
local_as: 12345
peer_as: 6789
peer_ip: 10.10.10.1
communities:
- 100:100
origin: igp
```
Legacy configurations are automatically converted to the new format internally, ensuring backward compatibility.
Routes are announced to all configured peers. If announcement to one peer fails, the operation continues for other peers. Errors are aggregated and returned, but partial success is allowed.
Communities are merged in the following order:
1. **Global communities** (defined at `bgp.communities`)
2. **Per-peer communities** (defined at `bgp.peers[].communities`)
3. **Per-route communities** (defined at `apps[].vip_config.bgp_communities`)
Example: If global communities are `[1000:1000]`, peer communities are `[100:100]`, and route communities are `[5000:5000]`, the announced route will have all three: `[1000:1000, 100:100, 5000:5000]`.
- **Default behavior**: Multi-hop is disabled by default
- **Enable**: Set `multi_hop: true` per peer to explicitly enable multi-hop BGP
The `/info` endpoint now returns an array of peer information instead of a single peer object:
**Before:**
```json
{
"conf": {
"neighbor_address": "10.10.10.1",
"peer_as": 6789
},
"state": {...}
}
```
**After:**
```json
[
{
"conf": {
"neighbor_address": "10.10.10.1",
"peer_as": 6789
},
"state": {...}
},
{
"conf": {
"neighbor_address": "10.10.10.2",
"peer_as": 6789
},
"state": {...}
}
]
```
- `config/config.go`: Added `PeerConfig` struct and `Peers` slice to `BgpConfig`
- `controller/bgp.go`: Refactored to support multiple peers with best-effort semantics
- `controller/monitor.go`: Updated `GetInfo()` to return slice of peers
- `server/server.go`: Updated info handler to return array of peers
1. **Controller struct** now stores `[]PeerConfig` instead of single peer fields
2. **Announce/Withdraw** methods loop through all peers with error aggregation
3. **getApiPath** accepts a `PeerConfig` parameter for per-peer community merging
4. **addPeer** determines multi-hop settings per peer
5. **PeerInfo** returns information for all configured peers
6. **Shutdown** gracefully shuts down all peer sessions
The implementation includes comprehensive test coverage:
1. **TestLegacyConfigConversion** - Verifies backward compatibility by testing that legacy single-peer configs are automatically converted to multi-peer format
2. **TestMultiPeerConfig** - Tests that new multi-peer configurations are properly loaded with multiple peers
3. **TestNoPeersConfigError** - Ensures proper error handling when no peers are configured
4. **TestCommunityMerging** - Validates that global, per-peer, and per-route communities are correctly merged in order
5. **TestMultiHopConfiguration** - Tests multi-hop BGP settings with various scenarios:
- Default behavior (multi-hop disabled)
- Explicit multi-hop disable
- Explicit multi-hop enable
6. **TestBestEffortAnnouncement** - Verifies that announcements succeed even when individual peers may have issues
7. **TestWithdrawMultiplePeers** - Tests route withdrawal across multiple peers
8. **TestPeerInfoMultiplePeers** - Validates that peer information is correctly returned for all configured peers
- **TestBgpNew** - Full integration test with actual BGP listeners (requires root, skipped in CI)
- **TestMultiPeerAnnouncement** - Tests actual route announcements to multiple BGP listeners (requires root, skipped in CI)
Existing configurations using `peer_ip` and `peer_as` continue to work without modification.
To add a second peer for resilience:
```yaml
bgp:
local_as: 12345
# Keep existing config for backward compatibility, or remove these lines
# peer_as: 6789
# peer_ip: 10.10.10.1
# Add new multi-peer config
peers:
- peer_ip: 10.10.10.1
peer_as: 6789
- peer_ip: 10.10.10.2 # redundant peer
peer_as: 6789
communities:
- 100:100
origin: igp
```
All operations (Announce, Withdraw, Shutdown) use best-effort error handling:
- Operations continue even if individual peers fail
- Errors are collected and returned as aggregated error messages
- Format: `"announcement errors: [peer 10.10.10.1: error message, peer 10.10.10.2: error message]"`
These changes were authored via AI LLM.
Authored-By: Claude Code (Sonnet 4.5)
GoCast
Gocast is a tool that does controller BGP route advertisements from a host. It runs custom defined healthchecks and announces or withdraws routes (most commonly VIPs or Virtual IPs) to a BGP peer. The most common use case for this is anycast (vip) based load balancing for infrastructure services such as DNS, Syslog etc where several instances are available in geographically diverse regions that announce the same anycast VIP, and clients then get sent to the closest instance.
For some practical examples and more details, check out this blog post : https://mayuresh82.github.io/2020/11/28/automatic_service_discovery_anycast/
Looking for code reviewers
If you are interested in being a reviewer and/or co-maintainer, please reach out to @mayuresh82 !
Installation
Use the docker container at mayuresh82/gocast or compile from source:
- Install Go
- Setup your GOPATH
- Run
go get -d github.com/mayuresh82/gocast - Run
cd $GOPATH/src/github.com/mayuresh82/gocast - Run
make
Design
GoCast uses GoBGP as a library to peer with remote neighbors and announce/withdraw prefixes. It really is just a healthcheck based wrapper around GoBGP. Remote peers can be autodiscovered or statically configured. A peer will most commonly be a Top-Of-Rack (TOR) switch.
Typically you would run GoCast on the same hosts as the service that needs to be monitored. Once an application "registers" with GoCast, GoCast then runs the predefined health monitors/checks and if they fail (e.g a service listening on a specific port), the routes are withdrawn thereby taking the node out of service.
GoCast uses a config file to define agent parameters (http addr, consul server addr, timers etc) and BGP parameters (local/peer ASN, peer IP, origin/communnities). See example config.yaml.
Registration
An application can register with the GoCast instance running on the same host using one of the following methods:
- http call : Make an http get call with the required parameters. For example:
http://gocast-addr/register?name=<appName>&vip=<addr/mask>&monitor=port:tcp:5000
Multiple monitors can be defined and the healthcheck succeeds only when all the monitors pass.
-
Custom defined apps in config.yaml. See the example config.yaml for syntax examples
-
Consul based auto-discovery (see below)
Monitors
A health monitor can either be a port monitor, an exec monitor or consul. Port monitors are specified as port:protocol:portnum , exec monitors run a script or arbitrary command and pass on successful exit (status code 0), specified as exec:command and consul monitors use consul's own healthchecks, specifed simply as consul
Consul Integration
GoCast supports consul for automatic service discovery and healthchecking. For this to work, the following needs to be setup:
-
The host running GoCast needs to have the environment variable CONSUL_NODE set to the hostname in consul
-
The following tags must be set in consul for autodiscovery to work:
enable_gocast : required
gocast_vip=<addr/mask>: required
gocast_monitor=monitor:params: optional
If gocast_monitor=consul is specified, then GoCast uses the defined healthchecks in consul as the health monitors for the service.
If gocast_nat=protocol:listenPort:destinationPort is specified, then GoCast will create NAT rules, via iptables, and map traffic destined to the assigned VIP and the specified listenPort to the physical IP and destinationPort.
Example: gocast_nat=tcp:53:8053 and gocast_nat=udp:53:8053
Alternatively, if gocast_nat=protocol:port is specified, then GoCast will create NAT rules, via iptables, and map traffic destined to the assigned VIP and the specified port to the physical IP and port.
Example: gocast_nat=tcp:53 and gocast_nat=udp:53
Docker support
The docker image at mayuresh82/gocast can be used to run GoCast inside a container. In order for GoCast to manipulate the host network stack correctly, the container needs to run with NET_ADMIN capablity and host mode networking. For example:
docker run -d --cap-add=NET_ADMIN --net=host -v /path/to/host-config:/path/to/container-config mayuresh82/gocast -config=/path/to/config.yaml -logtostderr
Caveats and workarounds
The service to be monitored can also be run inside a container, provided the published service ports are set to listen on 0.0.0.0 (not a specific IP.) Certain orchestration solutions such as Nomad run the docker containers with published ports listening only on the physical IP address. This will cause all requests to the app to fail, because the host does not listen on the loopback interface any more (which GoCast uses and assigns the VIP IP to). To work=around this there are 2 options:
-
Start the service container in host networking mode OR
-
Register NAT rules for your service with GoCast for the required protocol/port(s). GoCast will then create iptables NAT rules that map traffic destined to the assigned VIP to the physical IP address. This is achieved by adding the
nat=protocol:listenPort:destinationPortin the http query orgocast_nat=protocol:listenPort:destinationPorttag(s) in consul, as shown in the Consul integration section above.
Why not just use ExaBGP or something similar ?
ExaBGP is commonly used for this purpose, with bash scripts and such. However, I found that there no standard way of doing things and there is little to no support for containerized services. Also ExaBGP's API is clunky and documentation is almost non existent. GoCast provides an out of the box solution without hacking together a bunch of scripts.