Route hosts to instance groups from an inventory variable#568
Open
fernandorocagonzalez wants to merge 1 commit into
Open
Route hosts to instance groups from an inventory variable#568fernandorocagonzalez wants to merge 1 commit into
fernandorocagonzalez wants to merge 1 commit into
Conversation
4 tasks
In deployments with several isolated network zones, each zone has its own execution nodes grouped in an instance group, because firewalls only let those nodes reach the hosts of their zone. A job runs on a single instance group, so the same baseline template (sysctl, users, patching) has to be duplicated per zone with a different limit and instance group. This adds an instance_group_routing_var field to job templates: the name of an inventory variable (usually a group var) whose value names the instance group each host should run on. At launch, hosts are grouped by the resolved value and the launch creates the same implicit workflow job that sliced jobs use, with one node per bucket: each job runs restricted to its bucket's hosts and assigned to the routed instance group. Hosts that do not resolve the variable run in an extra job with the normal instance group selection, and if everything lands in one bucket a plain job is launched on it. Resolution follows a simplified version of the Ansible precedence rules (host vars win, then groups by depth, priority and name) and only considers enabled hosts. Instance groups are protected by RBAC and the routing variable is inventory data, so it is not trusted blindly: the launch endpoint checks the launching user's use permission on every referenced instance group, the same rule already applied to instance groups prompted at launch, and unknown names reject the launch instead of falling back silently. The validated buckets are handed to the job creation so the launch routes to exactly what was checked. The same checks run again when relaunching a routed workflow, which recomputes the buckets from current inventory data. Routing is mutually exclusive with job slicing (also when the slice count is prompted at launch), prompting instance groups overrides it, and provisioning callbacks never route, like they never slice. Launch paths without a requesting user stay loud but safe when a routing value names a missing instance group: a workflow containing the routed template is marked failed with the reason instead of crashing the workflow manager cycle, webhook launches answer 400, and relaunching a routed child job keeps targeting its original bucket even if the template's routing variable changed since. Refs ctrliq#566
2e0f21f to
c567c6d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the RFE in #566: route hosts to instance groups based on an inventory variable.
The problem
With several isolated network zones, each zone has its own execution nodes in an instance group because firewalls only let those nodes reach the hosts of their zone. A job runs on one instance group, so today every baseline template (sysctl, users, patching) has to be duplicated per zone with a different limit and instance group. N templates and M zones means N x M copies that only differ in two fields.
What this does
New opt in field on job templates,
instance_group_routing_var: the name of an inventory variable (in practice usually a group var) whose value names the instance group each host should run on.At launch, hosts are bucketed by the resolved value and the launch creates the same implicit workflow job that sliced jobs use, one node per bucket. Each job runs restricted to the hosts of its bucket (filtered in the generated inventory, like slicing filters each slice, so
limitkeeps working) and assigned to the instance group named by the value. Hosts that do not resolve the variable run in one extra job with the normal instance group selection. If everything lands in a single bucket, a plain job is launched on it, and if nothing resolves the variable the launch behaves as if routing were not configured.Resolution follows a simplified version of the Ansible precedence rules: host vars win over group vars, groups merge sorted by depth, then
ansible_group_priority, then name. Only enabled hosts count. Details in the newdocs/instance_group_routing.md.Federated inventories in this repo already do per source inventory instance group routing with the same implicit workflow pattern; this is the per host variable version of that idea for setups where all the zones live in one inventory.
Security
The routing variable is inventory data, editable by people who may have no permission at all on the instance groups, so it is not trusted blindly:
use_rolerule that already applies to instance groups prompted at launch, and rejects the launch naming the group when the check fails or the group does not exist. No silent fallback. The validated buckets are handed to the job creation so the launch routes to exactly what was checked.Interactions
Testing
Note: the migration is numbered 0201 and will collide with the other open PRs that also add 0201 (#561, #563, #564, #565); whichever merges second renumbers, same as discussed there.