Provenance: Spec 0001 — Second K3s Node + Key Vault + Helm Controller

Spec: .sdd/specification/spec-0001-second-k3s-node.md Executed: 2026-03-05 Agent: claude-sonnet-4-6 (Claude Code, session claude/add-second-k3s-node-c8qjo)

Actions Taken

1. Read ADR-016 (docs/adr/adr-016-second-k3s-node-for-observability.md) and ADR-018 (docs/adr/adr-018-secret-management-keyvault-eso.md) for context.
2. Read all current infra files: infra/main.tf, infra/variables.tf, infra/outputs.tf, all module files.
3. Created .sdd/specification/spec-0001-second-k3s-node.md — saved the spec verbatim.
4. Created infra/modules/keyvault/ directory with 4 files:
   • versions.tf — Terraform >= 1.5, azurerm ~> 4.0
   • variables.tf — location, resource_group_name, name, tenant_id, vm_principal_ids, terraform_object_id
   • main.tf — azurerm_key_vault, role assignments for VM identities and Terraform caller
   • outputs.tf — key_vault_id, key_vault_uri, key_vault_name
5. Modified infra/modules/network/main.tf — added azurerm_public_ip.node2 resource (pip-kevinryan-node2).
6. Modified infra/modules/network/outputs.tf — added public_ip_address_node2, public_ip_id_node2, vnet_id, vnet_name outputs.
7. Modified infra/modules/compute/variables.tf — added vm_name, custom_data; removed acr_login_server, acr_name, github_token.
8. Modified infra/modules/compute/main.tf — parameterised NIC/VM/disk names using var.vm_name; replaced inline templatefile() with custom_data = var.custom_data.
9. Modified infra/modules/compute/outputs.tf — added private_ip_address output.
10. Deleted infra/modules/compute/cloud-init.yaml — replaced by root-level templates.
11. Created infra/cloud-init-server.yaml — K3s server cloud-init with Key Vault token retry, helm-controller in Flux components.
12. Created infra/cloud-init-agent.yaml — K3s agent cloud-init with Key Vault token retry, node taint and label.
13. Modified infra/modules/registry/variables.tf — replaced vm_principal_id (string) with vm_principal_ids (list of string).
14. Modified infra/modules/registry/main.tf — replaced single role assignment with for_each over toset(var.vm_principal_ids).
15. Rewrote infra/main.tf — added random provider, replaced module "compute" with module "node1" + module "node2", added module "keyvault", random_password.k3s_token, azurerm_key_vault_secret.k3s_token; updated registry to use vm_principal_ids.
16. Modified infra/variables.tf — added keyvault_name variable (default: kv-kevinryan-io).
17. Modified infra/outputs.tf — added node1_public_ip and node2_public_ip outputs.
18. Created .sdd/provenance/spec-0001-second-k3s-node.provenance.md (this file).
19. Committed all changes to branch claude/add-second-k3s-node-c8qjo and pushed to remote.

Decisions Made

No autonomous decisions were required — all actions were explicitly specified in the spec.

Deviations from Spec

No deviations from spec.

Post-merge Manual Steps

These steps must be performed by a human operator after the PR is merged:

1. terraform init — required to download the new random provider.
2. terraform plan — review the plan carefully (node1 will be replaced if tainted; keyvault, node2, and token secret will be created).
3. terraform taint module.node1.azurerm_linux_virtual_machine.main — force node1 rebuild to pick up the new cloud-init (which includes Key Vault token retrieval and helm-controller in Flux).
4. terraform apply — creates Key Vault, K3s token secret, node2, and rebuilds node1.
5. Verify both nodes are ready: kubectl get nodes
6. Verify helm-controller is running: kubectl get pods -n flux-system
7. Verify node2 taint: kubectl describe node vm-kevinryan-node2 | grep Taint

Note: k8s/flux-system/gotk-components.yaml will be auto-regenerated by flux bootstrap on node1 rebuild to include helm-controller manifests. No manual edit of this file is needed.

Artifacts Produced

File	Status
.sdd/specification/spec-0001-second-k3s-node.md	Created
infra/modules/keyvault/versions.tf	Created
infra/modules/keyvault/variables.tf	Created
infra/modules/keyvault/main.tf	Created
infra/modules/keyvault/outputs.tf	Created
infra/modules/network/main.tf	Modified
infra/modules/network/outputs.tf	Modified
infra/modules/compute/main.tf	Modified
infra/modules/compute/variables.tf	Modified
infra/modules/compute/outputs.tf	Modified
infra/modules/compute/cloud-init.yaml	Deleted
infra/cloud-init-server.yaml	Created
infra/cloud-init-agent.yaml	Created
infra/modules/registry/main.tf	Modified
infra/modules/registry/variables.tf	Modified
infra/main.tf	Modified
infra/variables.tf	Modified
infra/outputs.tf	Modified
.sdd/provenance/spec-0001-second-k3s-node.provenance.md	Created

Validation Results

#	Check	Result
1	.sdd/specification/spec-0001-second-k3s-node.md exists	Pass
2	infra/modules/keyvault/ has 4 files	Pass
3	infra/modules/keyvault/main.tf has no data "azurerm_client_config"	Pass
4	infra/modules/network/main.tf has pip-kevinryan-node2	Pass
5	infra/modules/network/outputs.tf exports vnet_id, vnet_name, public_ip_id_node2, public_ip_address_node2	Pass
6	infra/modules/compute/main.tf uses vm_name variable and custom_data variable	Pass
7	infra/modules/compute/variables.tf has no acr_login_server, acr_name, github_token	Pass
8	infra/modules/compute/outputs.tf includes private_ip_address	Pass
9	infra/modules/compute/cloud-init.yaml deleted	Pass
10	infra/cloud-init-server.yaml has helm-controller in Flux --components	Pass
11	infra/cloud-init-agent.yaml has --node-taint and --node-label in K3s agent install	Pass
12	Both cloud-init templates have retry loop (30 attempts, 10s sleep, hard failure)	Pass
13	infra/main.tf has module "node1", module "node2", module "keyvault"	Pass
14	infra/main.tf has random_password.k3s_token and azurerm_key_vault_secret.k3s_token	Pass
15	infra/main.tf uses var.keyvault_name (not module.keyvault.key_vault_name) — no circular dependency	Pass
16	Registry module grants AcrPull via for_each over vm_principal_ids	Pass
17	terraform fmt -check -recursive infra/	Not run — terraform binary not available in agent environment
18	terraform validate	Not run — terraform binary not available in agent environment
19	No circular dependencies in module wiring — var.keyvault_name used in templates, not module output	Pass
20	pnpm lint	Not applicable — only Terraform/infrastructure files changed
21	Provenance record exists with all required sections	Pass
22	All files committed together	Pass