VCAP — VCF Automation (3V0-21.25)

VCF Automation Identity & Multi-tenancy Architecture#

VCF Automation implements a hierarchical identity and isolation model essential for enterprise multi-tenancy. Understanding the organization → project → workspace relationship is critical for VCDX-level deployment.

Organization & Project Hierarchy

Organizations

are top-level isolation boundaries. A VCF Automation instance supports multiple orgs; each org has independent:

• Project hierarchies (unlike VCF Operations, which has no org concept)
• Identity source bindings
• Shared image libraries (org-scoped)
• Custom naming policies
• Billing/cost attribution

Projects

are workspaces within an org. Each project scopes:

• Cloud templates (design-time)
• Deployments (runtime)
• Service catalog items (may be shared org-wide)
• RBAC (project admin, designer, user roles)
• Approval policies
• Resource quotas (per user/org/project)
• Network domains and storage profiles

Identity Source Configuration

VCF Automation 8.x+ supports four identity models:

1. VCF Identity Services (vIDM-as-a-service in VCF 9.0)

   → Default for VCF deployments
   → LDAP/AD backend
   → Org/project group membership synced via SCIM

2. Legacy vRealize Automation Identity Provider (deprecated 9.0)

   → Still functional but no new org features
   → Local user database (not recommended at scale)

3. OIDC / Third-party IdP (Azure AD, Okta, Keycloak)

   → Enterprise federation
   → Token-based (no password sync)
   → Group claims mapped to org/project groups

4. SAML 2.0 (SP-initiated flow)

   → Active Directory / Shibboleth
   → Less flexible than OIDC

Role Inheritance & Custom Roles

Roles are assigned at org and project level. Built-in roles are:

• Role
• Scope
• Permissions
• Org Admin
• Org-wide
• Create/delete projects, manage org identity sources, view all deployments, set org quotas
• Project Admin
• Project
• Manage project cloud templates, service items, approvals, quotas, RBAC
• Designer
• Project
• Create/edit cloud templates, define ABX actions, configure service catalog
• User
• Project
• Request from service catalog, manage own deployments, day-2 actions only
• Viewer
• Project
• Read-only access to deployments, templates (audit role)

Custom Roles

(new in VCF 8.x) allow fine-grained permissions. Example:

customRole:
  name: "Network Designer"
  org: "Engineering"
  permissions:
    - resource.create.networking.*
    - resource.update.networking.*

• resource.read.networking.*
• blueprint.edit.network_profiles
• blueprint.read # can see all templates but not edit compute

Group-Based vs Direct Assignment

Group-based:

Assign role to AD group (via SCIM sync), user inherits role on group membership. Scales to thousands of users.

Direct assignment:

Assign role to specific user. Use for privileged roles (org admin, project admin).

Best Practice:

Use groups for team/functional roles (developers, network admins). Use direct assignment only for escalation (principal eng, on-call admin). Keep group membership in HR system (AD/Azure), auto-sync via SCIM.

Service Account Tokens & API Access

Service accounts (new in VCF 8.x) allow headless automation. Unlike users, service accounts:

Have no interactive login

Can be assigned to one org + one project only

Generate API tokens (refresh + access tokens)

Token TTL is configurable (default 1 hour access, 30-day refresh)

Audit log shows which service account performed each API call

curl -X POST https://vcf-automation.example.com/iaas/api/login \
  -H "Content-Type: application/json" \
  -d '{

• "username": "automation-bot@example.com",
• "password": "serviceAccountToken"
• }' \
• -c cookies.txt

Then use refresh token to get access token

curl -X POST https://vcf-automation.example.com/iaas/api/tokens \
  -H "Authorization: Bearer $REFRESH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"refreshToken":"'"$REFRESH_TOKEN"'"}'

Audit Trail & Compliance

Every identity action is logged:

• User login/logout (successful + failed)
• Role assignment/removal
• API token generation/revocation
• Service account creation/deletion
• Group membership changes (SCIM sync events)
• Org/project creation/deletion

Logs retained 365 days by default. Export to SIEM via syslog or webhook for compliance audits (SOC 2, PCI, HIPAA).

References:

VCF Automation Identity Services Integration (VMware docs)

SCIM Provisioning for VCF Automation Groups

Service Account Token Lifecycle Best Practices

Key Takeaways

• Production Tip:For VCF 9.0+, always deploy VCF Identity Services alongside VCF Automation. Use SCIM provisioning (not manual group sync) to keep AD/Azure/Okta group membership in sync with Automation org/project groups. Set up service accounts for API automation with 24hr token TTL max.
• Blueprint Design for Multi-Cloud: YAML templates abstract infrastructure. Use 'Cloud.vSphere.Machine' for vSphere, 'Cloud.AWS.Machine' for AWS same template (conditional resource blocks). Inputs define cloud selection; resources adapt. Benefit: single source of truth for hybrid deployments. Challenge: network addressing differs per cloud (vSphere VLANs vs AWS subnets).
• ABX Extensibility Patterns: ABX actions extend blueprints for custom logic (approval gating, external API calls, remediation). Cold-start actions (<100/day) save resources; keep-warm (>100/day) reduce latency. Languages: Python (most), Node.js (async), PowerShell (AD integration). Secrets via HashiCorp Vault (never hardcoded). Timeout: 30s for blocking actions, 5-10s aggressive for non-blocking.
• Catalog Management: Organize templates by environment (dev/staging/prod). Share at org-level for consistency, keep project-private for team-specific. Version templates in Git; use semantic versioning (v1.0, v1.1, v2.0). Breaking API changes → major version. Additive features → minor version. Recommendation: pin projects to v1.0 until tested on v2.0.
• Approval Policies: Tiered approach based on cost/risk. <$1K auto-approve, $1-5K dept manager + finance (parallel), >$5K VP review (sequential). Non-blocking: email approvers. Blocking: prevents deployment until approved. IMPORTANT: approval policy must be YAML-declarative (not ad-hoc), version-controlled, and audited.

Advanced Cloud Templates (YAML) — Design Patterns#

Cloud templates in VCF Automation 8.9+ use YAML-based Infrastructure as Code. Mastering design patterns enables complex multi-tier provisioning with reusability and DRY principles.

Template Structure & Resource Nesting

A cloud template defines resources (VMs, networks, disks, load balancers) using resourceType declarations and inputs. Resources can have dependencies:

formatVersion: 1
name: Multi-Tier Web App
inputs:
  tier:
    type: string
    enum: [dev, staging, prod]
  app_version:
    type: string
    default: "1.0"
  environment_name:
    type: string
    minLength: 3
    maxLength: 20
    pattern: "^[a-zA-Z0-9-]*$"  # JSON Schema validation

resources:
  web_network:
    type: Cloud.Network
    properties:
      name: web-net-${input.tier}
      networkType: existing
      constraints:
        - tag: "network-type:web"

  db_network:
    type: Cloud.Network
    properties:
      name: db-net-${input.tier}

  load_balancer:
    type: Cloud.LoadBalancer
    properties:
      name: lb-${input.tier}
      routes:
        - protocol: HTTPS
          port: 443
          instanceProtocol: HTTP
          instancePort: 8080
      instances:
        - '${web_server_1.id}'
        - '${web_server_2.id}'

  web_server_1:
    type: Cloud.vSphere.Machine
    properties:
      name: web-${input.tier}-01
      image: CentOS-7-Template
      cpus: 2
      memory: 4096
      networks:
        - network: '${web_network.id}'
          assignment: static
          address: 10.0.1.10

  web_server_2:
    type: Cloud.vSphere.Machine
    properties:
      name: web-${input.tier}-02
      image: CentOS-7-Template
      cpus: 2
      memory: 4096
      networks:
        - network: '${web_network.id}'
          assignment: static
          address: 10.0.1.11

  db_server:
    type: Cloud.vSphere.Machine
    properties:
      name: db-${input.tier}-01
      image: RHEL-8-DB-Template
      cpus: 4
      memory: 16384
      storage:
        bootDiskCapacityInGB: 100
        disks:
          - capacityInGB: 500
            name: db-data
            storageProfile: "HighPerf-SSD"
      networks:
        - network: '${db_network.id}'
          assignment: dhcp
      cloudConfig: |
        #cloud-init
        bootcmd:
          - [ sh, -c, "mkdir -p /var/lib/mysql" ]
        runcmd:
          - [ systemctl, start, mysqld ]

YAML Anchors & Reusable Snippets

Use YAML anchors (&) and aliases (*) to reduce duplication across templates:

formatVersion: 1
name: Shared Network Stack Template

Define a reusable network segment

define:
  network_template: &default_network_spec
    networkType: existing
    constraints:
      - tag: "network-security:standard"

resources:
  frontend_network:
    type: Cloud.Network
    properties:
      <<: *default_network_spec  # Merge anchor properties
      name: frontend-net
      networkCIDR: 10.0.1.0/24

  backend_network:
    type: Cloud.Network
    properties:
      <<: *default_network_spec
      name: backend-net
      networkCIDR: 10.0.2.0/24

  management_network:
    type: Cloud.Network
    properties:
      <<: *default_network_spec
      name: mgmt-net
      networkCIDR: 10.0.3.0/24

Cross-Project Template Sharing

Templates can be shared at org level or kept project-private. To share:

• Tag template with shared:true
• Set read-only access for other projects
• Document inputs and outputs in template description
• Version templates in Git; use branch-per-major-version

Parameter Validation with JSON Schema

Input validation happens before provisioning. Define constraints inline:

inputs:
  instance_count:
    type: integer
    minimum: 1
    maximum: 10
    default: 3
    description: "Number of app servers to provision"

  disk_size_gb:
    type: number
    default: 100.0
    pattern: "^[1-9][0-9]*(\.[0-9]+)?$"  # Positive decimal

  subnet_list:
    type: array
    items:
      type: string
      pattern: "^([0-9]{1,3}\.){3}[0-9]{1,3}/[0-9]{1,2}$"  # CIDR validation
    minItems: 1
    maxItems: 5

  deployment_region:
    type: string
    enum: [us-west, us-east, eu-west, ap-southeast]

  tags:
    type: object
    properties:
      cost_center:
        type: string
      owner:
        type: string
    required: [cost_center]

Software Components & Day-2 Configuration

VCF Automation 8.x uses "software components" (formerly Content & Kubernetes) to configure in-guest software. Syntax:

resources:
  app_server:
    type: Cloud.vSphere.Machine
    properties:
      name: app-01
      image: Ubuntu-20.04-LTS
      cloudConfig: |
        #cloud-init
        package_update: true
        packages:
          - python3-pip
          - git
          - curl
        runcmd:
          - [ git, clone, 'https://github.com/myorg/app.git', '/opt/app' ]
          - [ pip3, install, -r, /opt/app/requirements.txt ]

• [ systemctl, start, app-service ]

Alternative: use software components for complex multi-step

  kubernetes_node:
    type: Cloud.vSphere.Machine
    properties:
      name: k8s-nod

Key Takeaways

• Scalability Pattern:For 50+ templates, extract common patterns into a library. Use ABX actions to read snippets from a content library and inject into templates at runtime. This enables a "template marketplace" within your org.
• YAML Schema Validation: Use JSON Schema in template inputs to enforce constraints (minLength, maxLength, pattern, enum). Validation happens before provisioning; invalid input rejected immediately (fast feedback). Example: email pattern "^[a-zA-Z0-9._%+-]+@company\\.com$" prevents external users from being provisioned.
• Resource Dependencies: Explicit dependency graph via resource references (${network.id} in VM network field). Implicit dependencies via resource types (VM must wait for network creation). VCF Automation topologically sorts resources; parallelizes independent ones. Useful for async operations: provision all VMs in parallel, then configure in sequence.
• Software Components & Cloud-Init: Use cloud-init (standard) or custom scripts for guest OS config. Limitations: cloud-init requires cloud-init service enabled on template, custom scripts need parameterization. Recommendation: use Ansible post-provision instead of cloud-init (more robust, idempotent). Integration pattern: VCF provisions VM, triggers Ansible playbook via Day-2 action.

ABX Actions Production Patterns#

ABX (Automation by Example) actions are event-driven functions that extend VCF Automation. Production deployments require careful attention to language choice, container lifecycle, secrets, error handling, and monitoring.

Language & Runtime Selection

VCF Automation 8.9+ supports three runtimes:

• Runtime
• Version
• Use Case
• Cold Start
• Libraries
• Python
• 3.10 (8.9) / 3.11 (9.0)
• Data processing, HTTP calls, scripting
• ~2-3s
• requests, boto3, jinja2, paramiko
• Node.js
• 18.x LTS
• Async operations, webhooks, real-time
• ~1-2s
• axios, node-ssh, lodash
• PowerShell
• 7.4
• Windows/Hyper-V automation, AD integration
• ~3-5s
• native: Get-ADUser, Invoke-RestMethod

Selection criteria:

Use Python for most operations (largest ecosystem, vSphere SDK). Use Node.js for parallel async operations (e.g., provision 100 VMs in parallel). Use PowerShell only for Windows-specific (AD, Exchange, SCCM integration).

Container Lifecycle: Cold-Start vs Keep-Warm

ABX actions run in ephemeral containers. Two strategies:

Cold-start (default):

Container created on demand, destroyed after action completes. Pros: cost-efficient, no persistent state. Cons: 2-5s latency per invocation.

Keep-warm (pinned):

Container stays alive for 24h after last invocation. Pros: <100ms latency, connection pooling. Cons: resource reservation, potential memory leaks.

ABX Action Configuration:

---
name: sync-inventory-to-cmdb
runtime: python3.10
memory: 512  # MB
timeout: 300  # seconds
keepWarm: true  # Enable pinned container
concurrency: 5  # Max parallel invocations
environment:
  CMDB_API_ENDPOINT: "https://cmdb.example.com"
  LOG_LEVEL: "DEBUG"

Secrets Management via Vault Integration

Never hardcode credentials in ABX code. Use HashiCorp Vault (or vRealize Secrets Manager) to inject credentials:

Python ABX Action Example:

---
import os
import json
from hvac import Client

def handler(context, inputs):

Vault endpoint auto-injected via VCF Automation secret store

vault_token = os.environ.get('VAULT_TOKEN')
vault_addr = os.environ.get('VAULT_ADDR', 'https://vault.example.com')

client = Client(url=vault_addr, token=vault_token)

Read secrets for this deployment

deployment_id = inputs['resourceId']
secret_path = f"secret/data/vcf-automation/{deployment_id}"

    try:
        secret_data = client.secrets.kv.v2.read_secret_version(path=secret_path)
        db_password = secret_data['data']['data']['db_password']

• api_token = secret_data['data']['data']['api_token']
• except Exception as e:
• raise Exception(f"Failed to retrieve secrets: {str(e)}")

Use secrets in automation

• return {
• "status": "success",
• "deployment": deployment_id,
• "secret_retrieved": True
• }

Vault Integration Best Practice:

For each deployment, create a unique secret path with TTL = deployment lifetime. Include rotation policy (quarterly). Never log secrets; use redaction filters in audit logs.

HTTP Ingress/Egress & Proxy Configuration

ABX actions often need to reach external APIs. VCF Automation requires proxy configuration for egress:

ABX Configuration (YAML):

---
name: webhook-receiver
runtime: nodejs18
httpIngress: true  # Enable external webhook calls
httpPort: 8080
proxyConfig:
  enabled: true
  proxyHost: proxy.example.com
  proxyPort: 3128
  noProxyList:

• "vault.example.com"
• "*.internal.example.com"
• "10.0.0.0/8"

---

Node.js ABX Handler:

const axios = require('axios');
const https = require('https');

exports.handler = async (context, inputs) => {
const proxyUrl = http://${process.env.PROXY_HOST}:${process.env.PROXY_PORT};

  const httpAgent = new (require('http')).Agent({
    httpProxyUrl: proxyUrl
  });
  const httpsAgent = new https.Agent({
    httpsProxyUrl: proxyUrl
  });

• const response = await axios.post('https://external-api.example.com/provision',
• inputs,
• { httpAgent, httpsAgent, timeout: 30000 }
• );

return { status: 'webhook-sent', responseCode: response.status };
};

Timeout Tuning & Error Handling

ABX actions have two timeout levels:

Action timeout:

Max duration of action execution (default 300s, max 3600s). Set per action in config.

HTTP request timeout:

Individual HTTP call timeout (default 30s). Set in application code.

For long-running operations (>60s), implement async polling pattern:

Python ABX Action - Async Polling Pattern:

---
import requests
import time
from datetime import datetime, timedelta

• def handler(context, inputs):
• max_wait = 300 # 5 minutes
• poll_interval = 5 # seconds
• start_time = datetime.now()

Initiate long-running operation

• response = requests.post('https://api.example.com/provision/async',
• json=inputs,
• timeout=10)
• task_id = response.json()['taskId']

Poll for completion

    while (datetime.now() - start_time).total_seconds() < max_wait:
        status_response = requests.get(
            f'https://api.example.com/provision/{task_id}/status',

Key Takeaways

• Production Pattern:Use keep-warm for high-frequency actions (>100/day). Use cold-start for scheduled/rare actions. Monitor memory usage; set memory limit to actual need + 100MB buffer to avoid OOM kills.

Event Broker & Subscription Model#

VCF Automation's event broker is the nervous system of cloud-native IaC. Understanding event topics, subscription ordering, filtering, and debug techniques is essential for building reliable automation workflows.

Event Topics & Topic Hierarchy

VCF Automation 8.9+ exposes dozens of event topics. Key categories:

Compute Events:

• com.vmware.cac.compute.allocation.request # User requests VM from catalog
• com.vmware.cac.compute.provisioning.pre # Before VM provisioning starts
• com.vmware.cac.compute.provisioning.post # VM provisioned, before Day-0 config
• com.vmware.cac.compute.machine.created # VM fully online
• com.vmware.cac.compute.machine.reconfigured
• com.vmware.cac.compute.machine.deleted

Network Events:

• com.vmware.cac.network.interface.created
• com.vmware.cac.network.interface.configured
• com.vmware.cac.network.ip.assigned
• com.vmware.cac.network.security.applied

Storage Events:

com.vmware.cac.storage.disk.allocated
com.vmware.cac.storage.snapshot.created

Blueprint Events:

• com.vmware.cac.blueprint.request # User submits request
• com.vmware.cac.blueprint.request.approved # Admin approves request
• com.vmware.cac.blueprint.request.rejected

Deployment Lifecycle:

• com.vmware.cac.deployment.request.created # New deployment request
• com.vmware.cac.deployment.created # Deployment object exists
• com.vmware.cac.deployment.update.request
• com.vmware.cac.deployment.update.completed
• com.vmware.cac.deployment.delete.request
• com.vmware.cac.deployment.resource.removed # When VM/network removed

Day-2 Actions:

com.vmware.cac.day2.action.requested
com.vmware.cac.day2.action.completed

Resource Removal:

com.vmware.cac.resource.destroy.request
com.vmware.cac.resource.destroy.completed

Blocking vs Non-Blocking Subscriptions

Subscriptions control workflow.

Blocking:

Action must complete successfully before provisioning continues. Used for pre-provisioning validation, quota checks, policy enforcement.

Non-blocking:

Action executes in parallel; provisioning continues regardless of success/failure. Used for notifications, logging, external sync.

Subscription Configuration (YAML):

---
name: "validate-deployment-quota"
topicId: "com.vmware.cac.blueprint.request"
blocking: true  # <-- Blocking: must pass to continue
priority: 1     # <-- Priority 1 executes first
condition: |
  ${blueprint.name} contains 'prod'
eventHandler:
  actionPath: "/actions/check-quota"
  timeoutSeconds: 30

---
name: "log-deployment-creation"
topicId: "com.vmware.cac.deployment.created"
blocking: false  # <-- Non-blocking: always continues
priority: 5
eventHandler:
  actionPath: "/actions/log-to-siem"
  timeoutSeconds: 5

Critical:

Blocking actions failing will block deployment. Always set aggressive timeouts (5-30s max) and implement circuit-breaker patterns. If validation action is down, entire provisioning pipeline stops.

Priority Ordering & Execution Sequence

Multiple subscriptions to same topic execute in priority order (1 = highest). Within same priority, execution order is undefined (parallel).

• Example: Blueprint request triggers 3 blocking actions in sequence:
• Priority 1 (blocking): Check quota (execute first, fail-stop)
• Priority 2 (blocking): Validate network CIDR (execute second, fail-stop)
• Priority 3 (blocking): Create CMDB record (execute third, fail-stop)
• Priority 10 (non-block): Email approval team (fire-and-forget)

If Priority 1 fails → deployment rejected, Priority 2/3 never run
If Priority 1 passes, Priority 2 fails → deployment rejected, Priority 3 never runs
If Priority 1-3 pass → deployment continues, Priority 10 fires in parallel

Filter Expressions for Targeted Subscriptions

Use filters to reduce noise and target specific events:

Filter Expression Examples:

---

Only handle production deployments

${blueprint.name} matches '^prod-.*'

Only handle deployments with specific tag

${deployment.tags['environment']} == 'prod'

Size-based filtering

${inputs.num_vms} > 5

Cost threshold

${deployment.estimatedCost} > 5000

Complex AND/OR

${blueprint.project} == 'Production' AND
${inputs.tier} == 'critical' AND
(${deployment.cpus} > 16 OR ${deployment.memory} > 32768)

Negation (NOT in prod)

NOT (${blueprint.name} contains 'prod')

Subscription-to-Action Binding & Error Handling

When binding subscriptions to ABX actions, handle failures gracefully:

Subscription Error Handling Policies:

---
retryPolicy:
  enabled: true
  maxAttempts: 3
  backoffMultiplier: 2  # 1s, 2s, 4s retries

fallbackAction: "/actions/log-subscription-failure" # Fallback on all retries exhausted

timeout: 30 # seconds (short for blocking)

onFailure: ROLLBACK # For blocking: reject deployment if action fails

For non-blocking: ignored (always continues)

---

ABX Action - Graceful Error Handling:

def handler(context, inputs):
    try:
        result = external_api_call(inputs)
        return {
            'status': 'succe

Service Broker Catalog Design#

Service Broker enables self-service IT through a curated, policy-driven catalog. Advanced design patterns enforce governance while maintaining usability.

Content Sharing Policies (Project vs Org Level)

Cloud templates can be shared at two scopes:

Project-level sharing:

Template visible only within a project. Team isolation, but duplicates templates across projects.

Org-level sharing:

Template visible across all projects. Requires careful change management (breaking changes affect many users). Use versioning to mitigate.

Shared Library Organization Pattern:

• ---
• Organization/
• SharedLibrary/
• v1.0/
• 3Tier-App
• VDI-Pool
• K8s-Cluster
• v2.0/
• 3Tier-App # Breaking API change (new input schema)
• K8s-Cluster # Minor update (additive change)

  ProjectA/
    uses: SharedLibrary/v1.0/3Tier-App
    uses: SharedLibrary/v2.0/K8s-Cluster

ProjectB/

uses: SharedLibrary/v1.0/* # Pinned to v1.0

Not ready for v2.0 migration yet

Custom Forms with JSON Schema & Conditional Fields

Service Catalog items expose forms to end-users. Control form behavior with JSON Schema and form hints:

Cloud Template Inputs with Form Overrides:

---
formatVersion: 1
inputs:
  deployment_name:
    type: string
    title: "Deployment Name"
    minLength: 3
    maxLength: 30
    pattern: "^[a-z0-9-]*$"
    description: "3-30 lowercase letters, numbers, hyphens"

  environment:
    type: string
    enum: [dev, staging, prod]
    title: "Environment"
    default: dev

Conditional field: only show if environment == prod

  approval_ticket:
    type: string
    title: "Change Request Number"
    description: "Required for production deployments"
    $visible: "${input.environment} == 'prod'"  # VCF Automation 8.9+

  database_enabled:
    type: boolean
    title: "Include Database"
    default: true

Conditional nested field

  db_version:
    type: string
    enum: [mysql-5.7, mysql-8.0, postgres-13, postgres-14]
    title: "Database Version"
    $visible: "${input.database_enabled}"  # Hide if DB not selected

Dropdown populated from external API

  storage_profile:
    type: string
    title: "Storage Profile"
    $datasource: "${api.get('https://vcenter.example.com/api/storage-profiles')}"
    description: "Fetched dynamically from vCenter"

Multi-select with validation

  network_segments:
    type: array
    items:
      type: string
    minItems: 1
    maxItems: 4
    title: "Network Segments"
    description: "Select 1-4 networks for deployment"

Number with range

  instance_count:
    type: integer
    minimum: 1
    maximum: 50
    default: 3
    title: "Number of App Server Instances"
    description: "1-50 instances"

Custom validation with pattern + description

  email:
    type: string
    format: email
    title: "Owner Email"
    pattern: "^[a-zA-Z0-9._%+-]+@example\\.com$"
    description: "Must be @example.com domain"

Dropdown from External API

Populate dropdowns dynamically from external data sources:

Cloud Template - Dynamic Dropdown Example:

---
formatVersion: 1
inputs:
  team_name:
    type: string
    title: "Team"
    $datasource: "${api.get('https://orgchart.example.com/teams')}"

Returns: ["Team-A", "Team-B", "Team-C", ...]

  cost_center:
    type: string
    title: "Cost Center"

Depends on selected team

$datasource: "${api.get('https://finance.example.com/cost-centers?team=' + input.team_name)}"

Dynamic filtering based on previous input

resources:
  web_server:
    type: Cloud.vSphere.Machine
    properties:
      name: ${input.team_name}-web-${resource.name}
      tags:
        team: ${input.team_name}
        cost_center: ${input.cost_center}

Approval Policies: Sequential vs Parallel

Design approval flows for governance and speed.

Sequential Approval (slowest, most risk-averse):

---

Request → Dept Manager → Finance → IT Security → Deployment

Each approver has 24hr SLA. Can reject at any stage.
Total time: 4 × 24h = 96h minimum

Parallel Approval (faster, requires quorum):

---

Request → {Dept Manager, Finance, IT Security} [2 of 3 approve]

Approvers have 24h SLA. Majority vote decides.
Total time: 1 × 24h = 24h minimum + quorum logic

Tiered Approval (cost-based):

---

If cost < $1000 → Dept Manager auto-approves
If cost $1000-$5000 → Dept Manager + Finance (parallel)
If cost > $5000 → Dept Manager + Finance + VP (sequential)

VCF Automation 8.9+ approval policy syntax:

name: "tiered-approval-policy"
approvalLevels:
  - level: 1
    approverType: ROLE
    approverValue: "ProjectAdmin"
    condition: ${request.estimatedCost} < 1000
    autoApprove: true

  - level: 1
    approverType: ROLE
    approverValue: "Finance"
    condition: ${request.estimatedCost} >= 1000 AND ${request.estimatedCost} < 5000
    parallelWith: [previous]  # Parallel with level 1
    slaDays: 1

  - level: 2
    approverType: ROLE
    approverValue: "VP"
    condition: ${request.estimatedCost} >= 5000
    slaDays: 2
    autoEscalateTo: "CISO"

Key Takeaways

• Form UX Best Practice:For production deployments, require approval ticket number as a gating mechanism. Use conditional fields to simplify form: show only relevant options. Validate email domains to prevent external-user provisioning.

Infrastructure as Code at Scale — GitOps with VCF Automation#

Enterprise deployments require GitOps: infrastructure managed via Git, with CI/CD pipelines for testing, validation, and deployment. VCF Automation integrates with Terraform Cloud/Enterprise and version control for at-scale IaC.

GitOps Repository Structure

Organize code for maintainability across teams:

Repository Structure (monorepo pattern):

• ---
• vcf-automation-templates/
• README.md

Cloud templates (YAML)

• cloud-templates/
• dev/
• web-app.yaml
• vdi-pool.yaml
• k8s-cluster.yaml
• staging/
• web-app.yaml
• prod/
• web-app.yaml
• vdi-pool.yaml
• k8s-cluster.yaml

Terraform modules for VCF Automation resources

• terraform/
• modules/
• vcf_org/
• main.tf
• variables.tf
• outputs.tf
• vcf_project/
• vcf_cloud_template/
• vcf_service_item/

• environments/
• dev/
• main.tf # Deploys dev org, projects
• terraform.tfvars
• staging/
• prod/

ABX actions (source code)

• abx-actions/
• provision-vm/
• handler.py
• requirements.txt
• tests/
• test_handler.py
• sync-cmdb/
• handler.js
• package.json

Helm charts for K8s deployments

• helm-charts/
• web-app/
• database/

Tests, linting, policy checks

• tests/
• validate_templates.sh
• test_abx_actions.py

• .github/workflows/
• validate-templates.yaml
• deploy-templates.yaml
• test-abx.yaml

.gitlab-ci.yml # Alternative: GitLab CI
Jenkinsfile # Alternative: Jenkins

Branching Strategy: Trunk-Based + Environment Branches

Two common approaches:

Trunk-based (recommended):

All changes to main, feature branches short-lived (<1 day). Fast iteration, single source of truth.

GitFlow (enterprises with slower release cycle):

develop branch for staging, main branch for production. More ceremony, better for regulated environments.

Trunk-Based Workflow:

---
main (always deployable)

├─ feature/add-k8s-template (short-lived PR, <24h)
├─ feature/abx-vm-quota-check
└─ feature/terraform-module-refactor

Each PR:

1. Must pass all tests (validate-templates, lint, ABX tests)
2. Requires code review (2 approvals)
3. CI/CD validates Terraform plan (no breaking changes)
4. On merge, auto-deploy to dev environment
5. Manual approval to deploy to staging/prod

---

Environment-Based Branching (alternative, slower):

---
main (production templates, stable)

├─ staging    (deploy staging weekly)
└─ dev        (deploy on every commit)

Feature branches pull from dev, PR to dev/staging/main

Promotes: dev → staging → main (1 week cadence)

Testing Approach: tflint, checkov, Integration Tests

Validate templates before deployment:

CI/CD Pipeline (GitHub Actions):

---
name: Validate & Test

on:
  pull_request:
    paths:
      - 'cloud-templates/**'
      - 'terraform/**'
      - 'abx-actions/**'
  push:
    branches: [main]

jobs:
  validate-templates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Validate YAML syntax
        run: |
          find cloud-templates/ -name "*.yaml" -exec \
            python3 -c "import yaml; yaml.safe_load(open('{}')); print('OK: {}')" \;

• name: Check template schema

run: python3 scripts/validate_template_schema.py cloud-templates/

      - name: Lint Terraform
        uses: terraform-linters/setup-tflint@v3
        with:
          tflint_version: latest

• name: Run tflint

run: tflint terraform/

  policy-as-code:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Run checkov (policy validation)
        uses: bridgecrewio/checkov-action@master
        with:
          directory: terraform/
          framework: terraform
          quiet: true  # Only show failures

Custom policies

• name: Run custom cost policy

run: python3 scripts/check_cost_limits.py terraform/

  test-abx:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4

      - name: Install dependencies
        run: |
          cd abx-actions/provision-vm
          pip install -r requirements.txt -r requirements-dev.txt

• name: Run unit tests

run: pytest abx-actions/*/tests/ -v --cov

• name: Lint Python

run: pylint abx-actions//.py

  integration-tests:
    runs-on: ubuntu-latest
    needs: [validate-templates, policy-as-code, test-abx]
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v3

      - name: Deploy to sandbox org (test)
        env:
          VCF_SANDBOX_URL: ${{ secrets.VCF_SANDBOX_URL }}
          VCF_SANDBOX_TOKEN: ${{ secrets.VCF_SANDBOX_TOKEN }}
        run: |
          python3 scripts/deploy_to_env.py \
            --environment sandbox \
            --templates cloud-templates/dev/

• name: Run end-to-end tests

run: python3 tests/e2e_tests.py

      - name: Cleanup test deployments
        if: always()
        r

VCF Automation API Mastery#

The REST API is the programmatic interface to VCF Automation. VCDX-level expertise requires understanding API structure, authentication, pagination, rate limiting, and async operations.

API Gateway & Provider Endpoint Structure

VCF Automation exposes multiple API endpoint groups:

API Gateway Structure (VCF 9.0):

---
https://vcf-automation.example.com/

├─ /iaas/                        # Main IaaS API
│  ├─ /deployments               # Deployment CRUD
│  ├─ /machines                  # VM management
│  ├─ /requests                  # Service requests
│  ├─ /blueprints                # Cloud templates
│  ├─ /catalog-items             # Service catalog
│  ├─ /approvals                 # Approval workflows
│  ├─ /provisioning-requests     # Request tracking
│  └─ /resource-types            # Custom resource types

├─ /orchestration/
│  ├─ /executions                # Workflow runs
│  ├─ /tasks                     # Task status
│  └─ /logs                      # Execution logs

├─ /api-groups/
│  ├─ /infrastructure
│  ├─ /network
│  ├─ /storage
│  └─ /custom

├─ /auth/
│  ├─ /login                     # OAuth2 token endpoint
│  ├─ /logout
│  └─ /tokens

├─ /inventory/
│  ├─ /compute-resources
│  ├─ /networks
│  ├─ /storage-profiles
│  └─ /templates

└─ /content-management/
   ├─ /library-items
   └─ /content-sources

OAuth2 Authentication & Token Management

VCF Automation uses OAuth2 for API authentication (VCF 8.9+ and all 9.x):

OAuth2 Token Flow:

---

1. Authenticate user/service account
2. Receive access token (expires in 1 hour default) + refresh token (30 days)
3. Use access token in API calls
4. On expiry, use refresh token to get new access token
5. Refresh token itself expires; re-authenticate after 30 days

Python Example - Token Management:

---
import requests
import json
from datetime import datetime, timedelta

• class VCFAutomationAPIClient:
• def __init__(self, api_url, username, password):
• self.api_url = api_url
• self.username = username
• self.password = password
• self.access_token = None
• self.refresh_token = None
• self.token_expires_at = None

• def authenticate(self):
• """Get initial access token"""
• auth_url = f"{self.api_url}/iaas/api/login"
• payload = {
• "username": self.username,
• "password": self.password
• }

response = requests.post(auth_url, json=payload, timeout=10)
response.raise_for_status()

• data = response.json()
• self.access_token = data['access_token']
• self.refresh_token = data['refresh_token']

Token expires in 1 hour

self.token_expires_at = datetime.utcnow() + timedelta(hours=1)

• def refresh_access_token(self):
• """Use refresh token to get new access token"""
• refresh_url = f"{self.api_url}/iaas/api/tokens"
• headers = {
• "Authorization": f"Bearer {self.refresh_token}",
• "Content-Type": "application/json"
• }

response = requests.post(refresh_url, headers=headers, timeout=10)
response.raise_for_status()

• data = response.json()
• self.access_token = data['access_token']
• self.token_expires_at = datetime.utcnow() + timedelta(hours=1)

    def get_headers(self):
        """Get authorization headers, refresh if needed"""
        if datetime.utcnow() >= self.token_expires_at:

self.refresh_access_token()

• return {
• "Authorization": f"Bearer {self.access_token}",
• "Content-Type": "application/json"
• }

• def api_request(self, method, endpoint, **kwargs):
• """Make API request with auto token refresh"""
• url = f"{self.api_url}{endpoint}"
• headers = self.get_headers()

response = requests.request(method, url, headers=headers, timeout=30, **kwargs)
response.raise_for_status()

return response.json() if response.text else None

Usage

client = VCFAutomationAPIClient("https://vcf-automation.example.com", "user@example.com", "***")
client.authenticate()

deployments = client.api_request("GET", "/iaas/api/deployments")
print(deployments)

API Versioning & Backward Compatibility

VCF Automation API versions endpoints. Always specify version to ensure stability:

API Version Header:

• ---
• GET /iaas/api/deployments HTTP/1.1
• Authorization: Bearer $TOKEN
• Content-Type: application/json
• Accept: application/json;version=6.0.0

Version history:

• 6.0 (VCF 8.9)

- 7.0 (VCF 9.0) - Breaking: blueprint → cloud_template renaming

• 7.1 (VCF 9.0.2) - Additive: vSAN Max support

Best practice: Pin API version in client code, test before upgrading

Pagination & Rate Limiting

Large result sets use cursor-based pagination. Respect rate limits:

Pagination Example - Fetch All Deployments:

---
GET /iaas/api/deployments?limit=50&offset=0

Response:

{
  "content": [
    { "id": "dep-001", "name": "web-app-prod" },

• ... (50 items)
• ],
• "totalElements": 523, # Total records acro

Integration Patterns#

VCF Automation integrates with enterprise systems (ITSM, CMDB, configuration management, secrets, monitoring). Understanding integration patterns and failure modes is essential for production deployments.

ServiceNow Integration: Webhook vs MID Server

Two integration patterns for ITSM workflows:

Webhook pattern (direct):

VCF Automation calls ServiceNow REST API directly over HTTPS. Requires firewall rule. Credentials stored in VCF Automation.

MID Server pattern:

Lightweight agent in DMZ bridges network gap. VCF Automation and ServiceNow communicate via agent. More secure (no direct API exposure).

Webhook Pattern:

---

VCF Automation ABX Action → (HTTPS) → ServiceNow REST API

Pros: Simple, no infrastructure, low latency

Cons: Firewall rule needed, credentials in config

Implementation:

def handler(context, inputs):
    import requests
    import json
    from datetime import datetime

• deployment = inputs['deployment']
• sn_url = os.environ['SERVICENOW_URL']
• sn_token = os.environ['SERVICENOW_API_TOKEN'] # From vault

    change_request = {
        "short_description": f"VCF Deployment: {deployment['name']}",
        "description": json.dumps({

• "deployment_id": deployment['id'],
• "requested_by": deployment['requestedBy'],
• "resources": len(deployment['resources']),
• "estimated_cost": deployment['estimatedCost']
• }, indent=2),
• "assignment_group": "Infrastructure Change Management",
• "priority": "3",
• "type": "Standard",
• "cmdb_ci": "VCF_Automation_Platform"
• }

    response = requests.post(
        f"{sn_url}/api/now/table/change_request",
        json=change_request,
        headers={
            "Authorization": f"Bearer {sn_token}",
            "Content-Type": "application/json"
        },
        timeout=30

)

cr_number = response.json()['result']['number']

• return {
• "status": "success",
• "change_request": cr_number
• }

---

MID Server Pattern:

---

VCF Automation → (HTTPS) → MID Server Agent → (REST) → ServiceNow

Pros: Secure, no direct API, better for segregated networks

Cons: Extra infrastructure, agent management

MID Server Config:

• Deployed as VM/container between VCF and ServiceNow
• Handles credential encryption
• VCF Automation stores only MID Server endpoint URL
• All authentication via MID Server

Ansible Tower / Ansible Automation Platform Integration

Post-deployment configuration management via Ansible:

Workflow: Provision → Ansible Config → Monitoring

---

1. VCF Automation provisions VM
2. Day-2 event triggers ABX action
3. ABX action calls Ansible Tower API to launch job template
4. Job template configures guest OS (install packages, configure services)
5. Ansible registers deployment in monitoring (Datadog, New Relic tags)

ABX Action - Trigger Ansible Playbook:

---
import requests
import time

• def handler(context, inputs):
• deployment = inputs['deployment']
• machines = deployment['resources'] # List of provisioned VMs

tower_url = os.environ['ANSIBLE_TOWER_URL']
tower_token = os.environ['ANSIBLE_TOWER_TOKEN']

Prepare inventory for Ansible

• host_list = [
• {
• "name": vm['name'],
• "ip": vm['networks'][0]['address'],
• "environment": deployment['inputs']['environment']
• }
• for vm in machines
• ]

Launch Ansible job template

job_template_id = 12 # "Configure Web App" template

    job_launch = requests.post(
        f"{tower_url}/api/v2/job_templates/{job_template_id}/launch/",
        json={
            "extra_vars": {
                "hosts": host_list,
                "app_version": deployment['inputs']['app_version'],
                "environment": deployment['inputs']['environment']
            }
        },
        headers={"Authorization": f"Bearer {tower_token}"},
        timeout=30

)

job_id = job_launch.json()['job']

Poll job completion

    while True:
        job_status = requests.get(
            f"{tower_url}/api/v2/jobs/{job_id}/",
            headers={"Authorization": f"Bearer {tower_token}"}
        ).json()

        if job_status['status'] in ['successful', 'failed', 'cancelled']:
            return {
                "status": "success",

• "ansible_job_id": job_id,
• "ansible_status": job_status['status']
• }

time.sleep(5)

Jenkins / GitLab CI / Bamboo Pipeline Integration

Trigger VCF Automation deployments from CI/CD pipelines:

Use Case: App team pushes code → CI builds artifact → Deploys to staging via VCF

Jenkins Pipeline Example:

• ---
• pipeline {
• agent any

  stages {
    stage('Build') {
      steps {
        sh 'docker build -t myapp:${BUILD_NUMBER} .'
        sh 'docker push gcr.io/myorg/myapp:${BUILD_NUMBER}'
      }
    }

    stage('Deploy to Staging') {
      steps {
        script {
          def deployment = httpRequest(
            url: "${VCF_AUTOMATION_URL}/iaas/api/deployments/requ