FreeIPA Replication

Topology design, ipa-replica-install, agreement management, promoting replicas to CA/KRA, segment design, agreement states, and recovering from split-brain.

Topology: star, chain, mesh

An IPA topology is a graph whose nodes are servers and whose edges are replication agreements. Three common shapes:

Star                Chain                Mesh (3-host full)
                                          ipa1
   ipa1             ipa1---ipa2          /    \
   /|\               |                  /      \
  / | \              ipa3             ipa2----ipa3
ipa2 ipa3 ipa4       |
                    ipa4

A worked three-host example

Three servers in one site, full mesh:

ipa1.example.internal   10.0.0.11   CA + KRA + DNS
ipa2.example.internal   10.0.0.12   CA + DNS
ipa3.example.internal   10.0.0.13   DNS only

Segments (ca suffix and domain suffix each):
  ipa1 <---> ipa2
  ipa1 <---> ipa3
  ipa2 <---> ipa3

Step 1 — First server (ipa1)

# On ipa1 — the first CA
ipa-server-install \
  --realm EXAMPLE.INTERNAL --domain example.internal \
  --hostname ipa1.example.internal \
  --ds-password 'REDACTED' --admin-password 'REDACTED' \
  --setup-dns --auto-forwarders \
  --setup-kra \
  --unattended

Step 2 — Second replica (ipa2, also CA)

# On ipa2 — join the realm first with a client install, or let replica-install do it.
ipa-client-install --domain example.internal --server ipa1.example.internal \
  --realm EXAMPLE.INTERNAL --unattended \
  --principal admin --password 'REDACTED'

ipa-replica-install \
  --setup-ca --setup-dns --auto-forwarders \
  --principal admin --admin-password 'REDACTED' \
  --unattended

Step 3 — Third replica (ipa3, DNS only)

# On ipa3
ipa-client-install --domain example.internal --server ipa1.example.internal \
  --realm EXAMPLE.INTERNAL --unattended \
  --principal admin --password 'REDACTED'

ipa-replica-install --setup-dns --auto-forwarders \
  --principal admin --admin-password 'REDACTED' \
  --unattended

By default, ipa-replica-install creates a single agreement to the server it was installed from. That gives you a chain. Add the missing edges to make it a mesh:

# From any server — these commands use the topology plugin on the server itself.
ipa topologysegment-add realm       ipa2-ipa3 \
    --leftnode=ipa2.example.internal --rightnode=ipa3.example.internal
ipa topologysegment-add ca          ipa2-ipa3 \
    --leftnode=ipa2.example.internal --rightnode=ipa3.example.internal

There are two suffixes you typically care about: realm (the domain suffix, dc=example,dc=internal) and ca (o=ipaca). Add segments for each suffix that the pair of servers both host.

ipa-replica-install basics

Common flags

ipa-replica-manage and friends

ipa-replica-manage predates the topology plugin. It still works and is often the fastest way to see what a host thinks is going on locally.

# What does this server replicate with?
sudo ipa-replica-manage list
sudo ipa-replica-manage list ipa1.example.internal

# Push an initial sync from this server to a peer
sudo ipa-replica-manage re-initialize --from=ipa1.example.internal

# Force a sync for a stuck agreement
sudo ipa-replica-manage force-sync --from=ipa1.example.internal

# Remove an agreement (single direction)
sudo ipa-replica-manage del ipa-bad.example.internal

Important: ipa-replica-manage del on its own does not clean the whole topology — it removes agreements on this host only. For full decommissioning, use the topology plugin and ipa server-del:

ipa topologysegment-find realm
ipa topologysegment-del realm ipa1-ipa-bad
ipa server-del ipa-bad.example.internal

Topology segments (the modern way)

# See all segments for the realm suffix
ipa topologysegment-find realm

# Add / remove a segment (this is bidirectional by default)
ipa topologysegment-add realm ipa1-ipa3 \
  --leftnode=ipa1.example.internal --rightnode=ipa3.example.internal

ipa topologysegment-del realm ipa1-ipa3

# Add a one-way segment (rarely useful — only for intentional asymmetry)
ipa topologysegment-add realm ipa1-ipa2-oneway \
  --leftnode=ipa1.example.internal --rightnode=ipa2.example.internal \
  --direction=left-to-right

The topology plugin guards against mistakes: you cannot delete the last segment that keeps the topology connected. If you need to, it refuses until you add an alternative path or explicitly ask for --force.

Visualise

# ASCII-ish
for h in $(ipa server-find --raw | awk '/cn:/ {print $2}'); do
  echo "== $h =="; ipa topologysegment-find realm --server=$h
done

Promoting a replica to CA / KRA

You installed the first replicas without --setup-ca and now you want a second CA. Do not reinstall — promote:

# On the replica, with admin Kerberos ticket
kinit admin
ipa-ca-install --unattended --principal admin --admin-password 'REDACTED'

# Separately, if you also want KRA here:
ipa-kra-install --unattended --principal admin --admin-password 'REDACTED'

After ipa-ca-install finishes, check:

ipa-csreplica-manage list                 # agreements on the ca suffix
ipa topologysegment-find ca
getcert list | head -50                    # certmonger is now tracking CA certs here

Agreement states

A 389-DS replication agreement publishes a status string you can read out of the agreement entry. The states you'll see:

# Read raw agreement status
ldapsearch -Y GSSAPI -b cn=config \
  '(objectclass=nsds5replicationagreement)' \
  dn nsDS5ReplicaHost nsds5replicaLastUpdateStatus nsds5ReplicaUpdateInProgress

Split-brain recovery

Split brain in IPA usually means: a replica was offline long enough for its changelog window to expire, came back, and both sides have diverged changes that can't merge. Symptoms:

• Users created on "the other side" don't show up; users created here don't show up there.
• Agreement status shows LDAP error 32 or similar "changelog gap".
• ipa-replica-manage list says agreements exist; getent passwd somebody returns different results depending on which server you ask.

Recovery (assumes ipa1 is the authoritative side)

# 1. Stop writes on the diverged side.
ssh ipa2 'sudo ipactl stop'

# 2. Back up just-in-case.
ssh ipa2 'sudo ipa-backup'

# 3. On the authoritative side, mark agreements to the broken peer as halted
#    (avoid corrupt updates while we re-init).
#    Then initiate re-initialisation FROM the authoritative side:
ssh ipa2 'sudo ipactl start'
ssh ipa2 'sudo ipa-replica-manage re-initialize --from=ipa1.example.internal'

# 4. Verify convergence:
kinit admin
ipa user-add sanitycheck-$(date +%s) --first=Sanity --last=Check
# ...on ipa2 a minute later:
ssh ipa2 "ipa user-show sanitycheck-$(date +%s)"

If the split is between CAs (the ca suffix), the equivalent is ipa-csreplica-manage re-initialize --from=ipa1. Do CA re-init before letting clients request new certs, not during.

The 389-DS server ID gotcha

Every IPA server has a replica ID assigned at first install (a small integer baked into its CSN stamps). You cannot change it without reinstall. Practical consequences:

• Do not rename a FreeIPA host, full stop. Decommission it and install a new replica with the new name.
• If you clone a VM and boot two copies, both have the same replica ID — replication will break subtly, with missing updates rather than loud errors. Don't clone live IPA VMs.
• Replica IDs are visible in nsDS5ReplicaId under cn=replica,cn=dc\=example\,dc\=internal,cn=mapping tree,cn=config. Sanity-check them when you add new servers.

Backup implications

Replication is not a backup. A schema change that deletes a field will happily propagate to every replica. So will a kinit admin; ipa user-del --continue '*'.

• Full backups with ipa-backup on a hidden replica: no client impact, consistent snapshot.
• Take backups on a schedule, keep them offline (different disk, different host), and test the restore on a throwaway VM quarterly.
• For the CA: ipa-backup --data is not enough if Dogtag DB was modified mid-renewal. Full ipa-backup locks long enough to be consistent.
• Retention: 14 days of daily + 3 months of weekly is a reasonable default. Longer if compliance requires.

Verification

• [ ] ipa topologysegment-find realm shows the expected edges for every server pair that should replicate
• [ ] ipa topologysegment-find ca matches the realm topology for every CA-bearing pair
• [ ] ipa-replica-manage list on each server enumerates only the expected peers
• [ ] A test user created on ipa1 appears on ipa2 and ipa3 within 10 seconds
• [ ] A test host added on ipa3 appears on ipa1 and ipa2 within 10 seconds
• [ ] ipa-healthcheck (or ipa-healthcheck --failures-only) is clean on every server
• [ ] At least two servers have --setup-ca
• [ ] ipa-backup runs nightly on a hidden replica and the restore has been tested in the last 90 days

See also: FreeIPA, FreeIPA Dogtag Certs, Kerberos.