What Changed​

The azure-function-app module now accepts an entra_id_authentication variable. When set, the Function App enforces token-based authentication: every inbound request must carry a valid JWT issued by Entra ID. Unauthenticated calls receive HTTP 401 — no key required, no secret to rotate.

APIM obtains the token automatically using its system-assigned Managed Identity, so the integration remains seamless from the caller's perspective.

How It Works​

The module wires up Entra ID authentication (EasyAuth) under the hood: every inbound request is validated against the tenant endpoint, and only APIM identities explicitly listed in allowed_callers_client_ids can authenticate.

Getting Started​

module "function_app" {
  source  = "pagopa-dx/azure-function-app/azurerm"
  version = "~> 4.1"

  # ... other required parameters ...

  entra_id_authentication = {
    audience_client_id         = data.azuread_application.my_app.client_id
    allowed_callers_client_ids = [data.azuread_service_principal.apim.client_id]
    tenant_id                  = data.azurerm_subscription.current.tenant_id
  }
}

On the APIM side, replace the x-functions-key header with the authentication-managed-identity policy element — that's the only policy change needed.

Key Benefits​

Learn More​

For step-by-step configuration, APIM policy examples, and a recommended blue-green migration strategy for existing Function Apps, read the full guide:

👉 Authenticating APIM to Function Apps with Managed Identity

SaveMoney is read-only, metric-driven, and intelligent. It doesn't delete or modify anything—instead, it generates detailed reports to support informed FinOps decisions.

The Problem: Hidden Cloud Waste​

In large cloud environments, it's easy to lose track of resources:

• A developer spins up a VM for testing and forgets to deallocate it
• An App Service Plan remains active after all apps are removed
• Disks persist after VMs are deleted
• Public IPs sit idle with no associated resources

These scenarios are common across organizations, and they add up to significant unnecessary costs. Traditional cloud cost management tools often rely on simple heuristics or require manual tagging—neither of which provides the full picture.

The Solution: Metric-Based Analysis​

SaveMoney takes a data-driven approach to cost optimization by analyzing actual Azure Monitor metrics over configurable time periods. Instead of guessing whether a resource is unused, it examines:

• CPU utilization for Virtual Machines and App Service Plans
• Network traffic for VMs, Public IPs, and Static Web Apps
• Memory consumption for App Service Plans and Container Apps
• Transaction counts for Storage Accounts
• Deployment state for Container Apps and App Service Plans

This metric-based detection provides high confidence in the results, reducing false positives and helping teams focus on truly wasteful resources.

Architecture & Multi-Cloud Vision​

SaveMoney follows a modular architecture designed for multi-cloud support:

Currently, Azure is the primary supported cloud provider, with comprehensive analysis for common resource types. AWS support is planned for future releases. The architecture separates cloud-specific analyzers from the core logic, making it straightforward to add new providers and expand coverage.

What SaveMoney Detects​

SaveMoney currently analyzes 9 common Azure resource types, focusing on the ones that typically drive the most cloud waste:

• High Cost Risk: Virtual Machines (deallocated or idle), App Service Plans (empty or oversized)
• Medium Cost Risk: Orphaned Disks, unused Public IPs, idle Network Interfaces, disconnected Private Endpoints, low-traffic Storage Accounts and Container Apps
• Low Cost Risk: Static Web Apps with minimal traffic

Each resource is evaluated using a combination of state checks (is it attached? running?) and Azure Monitor metrics (CPU usage, network traffic, transactions) to minimize false positives.

Getting Started​

SaveMoney is available as part of the @pagopa/dx-cli toolchain. Here's how to use it:

Prerequisites​

• Azure subscription(s) with appropriate permissions
• Azure CLI authentication
• Node.js 22 or higher

Installation​

You can invoke the CLI directly via npx without installing globally, and if you're already using the DX CLI, SaveMoney is included:

npx @pagopa/dx-cli savemoney --help

When installed locally in a monorepo you can also run:

pnpm dx savemoney --help

Configuration Options​

SaveMoney supports multiple configuration methods to fit different workflows:

{
  "tenantId": "your-tenant-id",
  "subscriptionIds": ["subscription-id-1", "subscription-id-2"],
  "preferredLocation": "italynorth",
  "timespanDays": 30
}

npx @pagopa/dx-cli savemoney --config config.json

export ARM_TENANT_ID="your-tenant-id"
export ARM_SUBSCRIPTION_ID="subscription-id-1,subscription-id-2"

npx @pagopa/dx-cli savemoney

npx @pagopa/dx-cli savemoney \
  --config config.json \
  --format table \
  --location italynorth \
  --days 30 \
  --verbose

npx @pagopa/dx-cli savemoney
# Enter Tenant ID: your-tenant-id
# Enter Subscription IDs (comma-separated): sub-1,sub-2

Output Formats​

SaveMoney offers three output formats to suit different use cases:

npx @pagopa/dx-cli savemoney --format json

[
  {
    "costRisk": "high",
    "location": "italynorth",
    "name": "vm-test-01",
    "reason": "Low CPU usage (avg 0.57%). Low network traffic (0.06 MB/day avg).",
    "resourceGroup": "rg-test",
    "subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "suspectedUnused": true,
    "type": "Microsoft.Compute/virtualMachines"
  }
]

npx @pagopa/dx-cli savemoney --format table

┌────────────────────────┬─────────────────────────────────────┬────────────────┬────────┬──────────────────────────────────────────┐
│ Name                   │ Type                                │ Resource Group │ Risk   │ Reason                                   │
├────────────────────────┼─────────────────────────────────────┼────────────────┼────────┼──────────────────────────────────────────┤
│ vm-test-01             │ Microsoft.Compute/virtualMachines   │ rg-test        │ high   │ VM is deallocated.                       │
│ asp-unused-premium     │ Microsoft.Web/serverfarms           │ rg-prod        │ high   │ No apps deployed. Premium tier.          │
│ disk-unattached        │ Microsoft.Compute/disks             │ rg-dev         │ medium │ Disk is not attached to any VM.          │
│ pip-idle-static        │ Microsoft.Network/publicIPAddresses │ rg-network     │ medium │ Not associated. Static IP not in use.    │
└────────────────────────┴─────────────────────────────────────┴────────────────┴────────┴──────────────────────────────────────────┘

npx @pagopa/dx-cli savemoney --format detailed-json

The Road Ahead​

We're continuously evolving SaveMoney with exciting features on the horizon:

• Multi-Cloud Expansion – AWS support is on the roadmap, extending the same metric-driven approach to Amazon Web Services and beyond. Our modular architecture makes it straightforward to add analyzers for additional cloud providers.

• Automated Governance – Automated reports showing unused resources detected in your subscriptions, and CI/CD pipeline integration to flag potential waste before it accumulates.

• Smarter Analysis – Enhanced detection logic for additional Azure resource types, more sophisticated metric analysis with configurable thresholds, trend analysis to identify usage patterns, and integration with cloud pricing APIs for precise cost impact estimates.

• Additional Output Formats – Reports based on community feedback for different use cases and workflow integrations.

• Workflow Integration – Automated PR comments with cost impact analysis, policy-based governance for automatic resource tagging, and custom dashboards showing cost trends and optimization opportunities.

Get Started Today​

Ready to uncover hidden waste in your Azure environment? SaveMoney is open-source and available now as part of the DX toolchain.

Start your first analysis in under a minute:

npx @pagopa/dx-cli savemoney --help

Explore the documentation:

• Full CLI documentation – Configuration options, output formats, and usage examples
• GitHub repository – Source code and architecture details

Cloud cost optimization doesn't have to be a monthly fire drill. With the right tools and automation, it becomes part of your daily workflow—invisible, effortless, and continuous.

Happy cost optimization! 💰

We're officially moving from Yarn to PNPM for our dependency management. This change is a significant step forward for the PagoPA DX Toolchain, bringing improvements in speed, consistency and security to our development workflow.

Why PNPM?​

You might be asking why we're making this switch. Here are a few key reasons:

• Faster and More Efficient: PNPM's unique approach to dependency management, using a content-addressable store, avoids the need for duplicated packages. This means faster installations and less disk space consumption.

• A Safer Dependency Tree: PNPM creates a strict, non-flat node_modules structure, which helps prevent accidental access to dependencies that are not explicitly declared in your package.json. This makes our dependency trees more predictable and secure. This is particularly beneficial for our monorepo setups, as it will prevent ghost dependencies issues before the deployment.

Getting Started​

We've already updated the PagoPA DX Toolchain to include PNPM support. To make the migration as smooth as possible, we've created a migration script accessible via our DX CLI.

To run the migration script, use the following command in your project's root directory:

npx @pagopa/dx-cli codemod apply use-pnpm

This script will import your existing lockfile and it will automatically replace Yarn and NPM commands in your CI/CD workflows with their PNPM equivalents.

Prerequisites​

• Node.js 20.19.5 or higher with corepack enabled.

  note

  Corepack is included by default with Node.js versions 16.9.0 and above, but you'll need to enable it manually for version 20 and above. You can do this by running:

  corepack enable
  

• Your project should use the PagoPA DX workflows for code review and deployment. The migration script will update only the following workflows:

  • js_code_review.yaml
  • web_app_deploy.yaml
  • function_app_deploy.yaml

After the migration​

After running the migration script, test your project locally to ensure all scripts are working as expected, and all dependencies are correctly resolved.

Quick Guide: Using PNPM​

To help you get started, here's a quick reference guide on how to perform common tasks with PNPM, compared to their Yarn equivalents.

Install dependencies​

  # Install dependencies
  yarn install

  # Don't alter the lockfile
  yarn install --immutable

  # Add a new dependency to root package
  yarn add express

  # Add a development dependency to root package
  yarn add -D typescript

  # Install dependencies
  npm install

  # Don't alter the lockfile
  npm ci

  # Add a new dependency to root package
  npm install express

  # Add a development dependency to root package
  npm install --save-dev typescript

  # Install dependencies
  pnpm install

  # Don't alter the lockfile.
  # In CI is the default behavior, even without this flag.
  pnpm install --frozen-lockfile

  # Add a new dependency to root package
  # By default, it does not allow adding dependencies to root packages
  # unless the -w (workspace) flag is used
  pnpm add -w express

  # Add a development dependency to root package
  pnpm add -w -D typescript

Multi-projects support​

Both Yarn and PNPM support managing multiple projects within a single repository. Yarn requires that packages are defined in the workspaces field of the root package.json, while PNPM uses a dedicated pnpm-workspace.yaml file.

  {
    "name": "my-monorepo",
    "private": true,
    "workspaces": [
      "apps/*"
    ]
  }

  # Add a new dependency to my-app package
  yarn workspace my-app add express

  {
    "name": "my-monorepo",
    "private": true,
    "workspaces": [
      "apps/*"
    ]
  }

  # Add a new dependency to my-app package
  npm -w my-app add express

  packages:
    - 'apps/*'

  # Add a new dependency to my-app package
  pnpm --filter my-app add express

  # Note that filters, supports wildcards and complex expressions
  # Example: Add a new dependency to all packages starting with "my-"
  pnpm --filter "my-*" add express

  # Examples: Install dependencies only in packages that have changed
  # relative to the main branch
  pnpm --filter "...[origin/main]" install

  # The --filter syntax is the same used by Turbo

Run scripts​

Both Yarn and PNPM allow you to run scripts defined in your package.json files.

  # Run the build script in the current package
  yarn build

  # Run the build script in the my-app package
  yarn workspace my-app build

  # Run the build script in all packages
  yarn workspaces foreach --all run build

  # Run the build script in the current package
  npm run build

  # Run the build script in the my-app package
  npm run build -w my-app

  # Run the build script in all packages
  npm run build --workspaces

  # Run the build script in the current package
  pnpm build

  # Run the build script in the my-app package
  pnpm --filter my-app build

  # Run the build script in all packages
  pnpm -r build

Other features​

PNPM offers unique features that help us manage dependencies at scale. Here are some examples.

Catalogs​

Catalogs are a workspaces feature for defining dependency version ranges as reusable constants.

Imagine you have multiple packages that depend on react and react-dom. Instead of specifying the version in each package, you can define a catalog in the root pnpm-workspace.yaml file.

packages:
  - "apps/*"
  - "packages/*"

catalogs:
  react: ^18.0.0
  react-dom: ^18.0.0

Then, in each package's package.json, you can reference the catalog:

{
  "name": "my-app",
  "version": "1.0.0",
  "dependencies": {
    "react": "catalog:",
    "react-dom": "catalog:"
  }
}

{
  "name": "my-library",
  "version": "1.0.0",
  "dependencies": {
    "react": "catalog:",
    "react-dom": "catalog:"
  }
}

When you run pnpm install, PNPM will resolve the catalog: references to the versions defined in the pnpm-workspace.yaml file. This ensures that all packages use the same version of react and react-dom, making it easier to manage and update dependencies across the monorepo.

Security​

PNPM has built-in security features that help protect your projects from vulnerabilities.

• Post Install Scripts Control: PNPM allows you to control the execution of post-install scripts, which can be a vector for supply chain attacks. You can disable these scripts globally or on a per-package basis.
• minimumReleaseAge: This setting allows you to specify a minimum age for package releases. This can help avoid using newly published packages that might not have been thoroughly used yet.

For a complete list of PNPM features and commands, check out the official PNPM documentation.

As our projects grew in number and complexity, we faced a common challenge: how to effectively share and reuse Terraform code while maintaining quality and consistency. This led us to explore publishing our modules and provider to the Terraform Registry, all while preserving our internal development process based on a monorepo structure.

In this post, we'll share our journey as the Developer Experience team, the challenges we faced, and the technical solutions we implemented to bridge the gap between developing Terraform modules in a single repository and meeting the Terraform Registry requirements.

The Challenges We Faced​

Before diving into our solution, let's outline the main challenges we were trying to overcome:

1. Encouraging Module Reuse: Publishing modules to the registry provides a central, versioned source, encouraging teams to reuse the canonical module instead of duplicating code or reinventing the wheel.

2. Inefficient Module Downloads: Referencing modules directly from GitHub meant Terraform downloaded the entire repository for each module, causing slow terraform init times and inefficient storage space management.

3. Version Management: Without proper versioning, tracking changes and ensuring backward compatibility became increasingly difficult.

4. Documentation: Without a centralized place to document our modules, teams struggled to understand how to use them correctly.

5. Registry Requirements: The Terraform Registry has specific requirements for modules and providers, particularly around repository structure (one module per repository).

6. Development Workflow: We enjoy the benefits of a monorepo approach for code reviews, issue tracking, and shared CI/CD pipelines.

Our Solution: Bridging Monorepo and Terraform Registry​

We developed a system that allows us to:

1. Develop and maintain modules in our monorepo
2. Automatically publish them to dedicated repositories
3. Release them to the Terraform Registry with proper versioning

Here's a visual representation of our approach:

Technical Implementation for Modules​

The technical implementation involves several components working together to bridge our monorepo workflow with the Terraform Registry requirements.

1. Structured Module Organization​

Within our monorepo, we organize modules in the infra/modules directory following a consistent pattern. Each module (such as azure_api_management and azure_container_app) contains its own code, tests, examples, and documentation, making it easy for developers to navigate and contribute.

2. Automated Push to Sub-repositories​

The heart of our solution is an automated GitHub Action workflow that handles synchronization between our monorepo and individual repositories. When changes are made to a module in our monorepo, the workflow detects these changes and pushes them to a dedicated sub-repository following the Terraform Registry naming convention. Crucially, before pushing, automated tests are run against the modules to ensure quality and prevent regressions.

For instance, our Azure API Management module in the monorepo is pushed to a repository named terraform-azurerm-azure-api-management, making it ready for Registry distribution while preserving our centralized development model.

3. Version Management with Changesets​

We use Changesets to manage versions and generate changelogs:

1. Developers create a changeset describing their changes using the tool's command-line interface.

   # Example command for creating a changeset
   yarn changeset
   

2. This process prompts for the change type (patch/minor/major), a description of the changes, and the affected modules.

3. When merged, a release action automated by the tool bumps versions, updates changelogs, tags the repository, and triggers the sub-repository push.

This gives us proper semantic versioning and clear documentation of changes.

4. Module Security with Lock Files​

To ensure security and prevent supply chain attacks, we implemented a pre-commit system that generates lock files for all Terraform configurations using a custom script, similar to provider locks.

This is critical when using Registry modules, as it ensures that the exact module versions and their content hashes are verified before deployment.

{
  "modules": {
    "registry.terraform.io/pagopa-dx/azure-role-assignments/azurerm": {
      "version": "0.0.1",
      "hash": "a123bc4567890def1234567890abcdef1234567890"
    }
  }
}

repos:
  - repo: https://github.com/pagopa/dx
    rev: pre_commit_scripts@0.0.1
    hooks:
      - id: lock_modules
        exclude: ^.*/(_modules|modules|\.terraform)(/.*)?$
        files: src/(prod|dev) # Example paths

Bonus: Publishing Our Custom Provider​

Beyond modules, we also developed a custom Terraform provider, DX Azure provider, to encapsulate specific logic like our resource naming conventions. We applied a similar automated approach:

1. Development happens in the monorepo (infra/provider).
2. Automated Workflow (_release-bash-provider-to-subrepo.yaml) pushes code to a dedicated terraform-provider-azure repository.
3. Registry Publication: Tagged releases trigger a CI/CD pipeline that builds, signs with GPG, and publishes the provider to the Terraform Registry under the pagopa-dx/azure namespace.

This allows teams to use our custom provider easily:

terraform {
  required_providers {
    dx = {
      source  = "pagopa-dx/azure"
      version = "~> 0.0" // Specify desired version
    }
  }
}

provider "dx" {
  # Configuration options
}

Benefits and Results​

Our approach has yielded significant benefits:

1. Better Developer Experience​

Teams can now simply reference our modules with clear versioning:

module "roles" {
  source = "github.com/pagopa/dx//infra/modules/azure_role_assignments?ref=main"
  // parameters...
}

module "roles" {
  source  = "pagopa-dx/azure-role-assignments/azurerm"
  version = "~> 0.0"
  // parameters...
}

2. Semantic Versioning Benefits​

With our approach to semantic versioning, teams now enjoy a clear contract for module updates. They can confidently upgrade modules within the same minor version knowing that breaking changes won't occur. This system helps developers easily identify when major changes require careful migration planning, reducing refactoring time during updates and making dependency management more straightforward.

3. Enhanced Documentation Access​

The Registry serves as a professional, centralized documentation interface for all our infrastructure components. Teams can now browse comprehensive module documentation, review input and output variable specifications, find practical usage examples, and access version histories all in one place. This centralization eliminates the need to navigate multiple GitHub repositories or documentation sources, making it faster and easier to implement modules correctly.

4. Centralized Development with Decentralized Distribution​

Our solution preserves the benefits of monorepo development while gaining the advantages of Registry distribution. We maintain centralized code reviews and issue tracking, consistent CI/CD pipelines, and a unified development workflow. Simultaneously, we benefit from an official publication channel, standardized documentation formats, semantic versioning, and simplified consumption for end users.

Lessons Learned​

Throughout this journey, we've learned several important lessons that shaped our approach and thinking about infrastructure code.

Automation proved absolutely essential. Without automated workflows, maintaining synchronization between our monorepo and sub-repositories would be practically impossible at scale. We've also discovered that versioning discipline matters deeply; semantic versioning provides a clear contract for users of our modules and provider.

Documentation has become a first-class citizen in our process. Good documentation dramatically reduces onboarding time and support requests, making teams more productive. We've embedded security into our workflow by using module lock files, ensuring supply chain security without compromising convenience.

Perhaps most interestingly, we found that the Registry's requirements for structure and documentation actually drove better practices across our entire infrastructure codebase, improving our overall IaC quality.

Future Directions​

As we continue to evolve our approach, we're now exploring several exciting directions. We plan to expand our module catalog to cover more infrastructure patterns common across our organization. Our team is also enhancing the provider with additional functions to address more use cases.

Conclusion​

Our journey from monorepo to Terraform Registry demonstrates that you don't have to choose between a centralized development workflow and standardized distribution. With the right automation and tools, you can have the best of both worlds.

By publishing our modules and provider to the Terraform Registry, we've improved code reuse, standardization, and documentation while maintaining our preferred development workflow. The investment in automation has paid dividends in terms of development velocity and code quality.

We hope our experience provides valuable insights for teams facing similar challenges in their Infrastructure as Code journey.

Have questions about our approach or want to see our modules? Check out our Terraform Registry namespace or feel free to contact us.

This module is useful for anybody that has just created a new repository and wants to focus quickly on their goals rather than spending hours in setting up everything around the new repository. The module focuses on projects which leverage Azure, GitHub, and a single environment (production). After applying this module, the repository will have:

• completed the setup needed to launch GitHub Actions workflows
• a dedicated private GitHub Runner to connect to private Azure resources from GitHub pipelines
• an Azure resource group to deploy resources generally contained in infra/resources
• the required permissions to operate on domain resources
• a secure and smooth configuration

To accomplish the setup, the module provisions:

• an Azure Container App Job to run workflows in a private GitHub Runner integrated with an Azure VNet
• a federation between the GitHub repository and Azure user-assigned managed identities to allow workflows to connect with Azure tenant
• the creation of a project-specific Azure resource group which will contain the infrastructure of the entire repository as it will share the same lifecycle and IAM setup
• the IAM setup of both team and Azure user-assigned managed identities following the latest DevEx framework
• the GitHub repository settings according to the best practices found by the DX team

As the Azure GitHub Environment Bootstrap module is designed to support teams in new project development, it is particularly indicated for mono repositories which traditionally require more time to be properly prepared: in just a few minutes, the repository will be ready to be used.

On the other hand, the new module may be a bit overwhelming for small repositories, where you may have only one application. For this reason, the module azure_federated_identity_with_github is still maintained and available, and is suggested for those cases. However, please note that this design is not advisable, and instead, the use of mono repositories is recommended.

Getting Started​

We recommend to reference the module via the Terraform Registry, and pin the version to the latest stable release:

module "repo" {
  source  = "pagopa-dx/azure-github-environment-bootstrap/azurerm"
  version = "~>1.0"
}

Despite the Azure GitHub Environment Bootstrap module requiring a lot of inputs, its usage remains easy. In fact, a lot of values are the same for different projects under the same product's umbrella, and moreover are well-known by our user base. The following data is requested:

• the Azure subscription and tenant IDs
• the IDs of team-specific Entra ID groups as expected by DevEx IAM framework
• the ID of an existing Azure Container App Environment
• the details of the Storage Account holding the Terraform state file
• the details of the current repository (name, description, topics, optionally reviewers, etc.)
• the IDs of the product-shared cloud resources, if any (API Management, Private Endpoints, VNet, etc.)

More details about the usage can be found in the documentation, which is available in the module entry in the Terraform Registry.

Examples​

Explore these repositories, which have successfully adopted the Azure GitHub Environment Bootstrap module from its early stages:

• IO Messages
• IO Wallet
• IO Services CMS

This module abstracts inner complexities such as networking, authentication, and scaling. By choosing the l tier (which matches the Premium SKU), the module provides private communication via private endpoints and also includes an autoscaler resource to dynamically change the number of instances depending on CPU and memory metrics.

The module also supports the m tier (Standard SKU), but it is recommended to use this tier only in test environments as the resource cannot be isolated in terms of networking and remains on the public internet.

Getting Started​

We recommend referencing the module via the Terraform Registry and pinning the version to the latest stable release:

module "sbns" {
  source  = "pagopa-dx/azure-service-bus-namespace/azurerm"
  version = "~>0.0"
}

Roles can be managed through the traditional role assignment module, which you should already be familiar with. By passing names of queues, topics, and subscriptions, the IAM complexity is taken away.

module "roles" {
  source  = "pagopa-dx/azure-role-assignments/azurerm"
  version = "~>1.0"

  principal_id    = <principal_id>
  subscription_id = data.azurerm_subscription.current.subscription_id

  service_bus = [
    {
      namespace_name      = <value>
      resource_group_name = <value>
      role                = "reader|writer|owner"
      description         = <value>
      queue_names         = (optional) [<list of queues>]
      topic_names         = (optional) [<list of topics>]
      subscriptions = {
        (optional) <list of key-value pairs for subscriptions, e.g. topic = [<list of subscriptions>]>
      }
    }
  ]
}

Additionally, the Azure Bootstrap module has been updated to add the needed roles to apply Terraform code to a centralized Service Bus instance, if you have one.

More details can be found in the documentation, which is available in the module entry in the Terraform Registry.

Moreover, if you want to learn more about best practices and patterns, check out DX guidelines.

Examples​

Explore module examples to create the Service Bus namespace. Other examples are also available to show queue, topic, and subscription definitions.

This module allows teams to configure alerts for both active and dead-lettered messages across queues and topics, providing flexibility through customizable thresholds, severities, and metric windows.

It is designed to integrate easily with your existing Service Bus namespaces and can be customized through simple input variables. Multiple entities can be monitored at once, and alert actions can be routed to one or more Azure Monitor Action Groups.

Getting Started​

We recommend referencing the module via the Terraform Registry and pinning the version to the latest stable release:

module "alerts" {
  source  = "pagopa-dx/azure-service-bus-dlq-alert/azurerm"
  version = "~> 0.0"

  environment = {
    prefix          = "example"
    env_short       = "d"
    location        = "westeurope"
    app_name        = "myapp"
    instance_number = "01"
  }

  service_bus_namespace_id = azurerm_servicebus_namespace.example.id

  alerts_on_dlq_messages = {
    description   = "DLQ messages alert"
    entity_names  = ["queue1", "topic1"]
    threshold     = 5
    severity      = "Error"
    check_every   = "PT5M"
    lookback_period = "PT10M"
  }

  alerts_on_active_messages = {
    description   = "Active messages alert"
    entity_names  = ["queue1", "topic1"]
    threshold     = 20
    severity      = "Warning"
    check_every   = "PT15M"
    lookback_period = "PT30M"
  }

  action_group_ids = [azurerm_monitor_action_group.example.id]
  tags             = local.tags
}

Examples​

Explore a simple example to get started quickly.

More information about metrics and setup can be found in the module documentation.

Using API clients requires the code that handles HTTP requests and responses to strictly adhere to the OpenAPI specifications. This is crucial to ensure that the exchanged data is valid and consistent with the defined schemas. Similarly, when providing APIs through NodeJS services, such as Express or similar frameworks, it is important to follow the same standards. This ensures that the APIs are reliable, scalable, and easy to maintain.

The purpose of this document is to evaluate and compare various TypeScript OpenAPI generators for both frontend and backend development. The evaluation will focus on critical features and help determine whether to invest in a custom tool (make) or adopt an existing tool.

Guiding Principles​

Main Requirements​

1. Runtime Payload Control The generators must provide runtime controls on the payloads to ensure data integrity and adherence to defined schemas. We believe runtime control is necessary to guarantee that the Typescript type matches the actual structure of the payload. Without this, there's a higher chance of runtime errors due to discrepancies in the data format.

2. Generation of Isomorphic Types The ability to generate types usable both on the client and server sides (e.g., in Express route handlers).

3. Support for String Patterns (Regex) Generators must support string patterns using regular expressions to enforce constraints on string values.

4. Support for JSON Schema Constraints For example, the handling of constraints such as minimum, maximum, exclusiveMinimum, and exclusiveMaximum is essential for validating numeric data.

5. Community and Maintainability A strong community and/or easily maintainable codebase are crucial for the long-term viability and support of the generator.

Additional Features (Nice to Have)​

1. Support for Security Headers Generators should support security headers to enhance API security.

2. Support for File Uploads The ability to handle file uploads seamlessly within the API.

3. Support for File Downloads (Octet Stream) There must be support for file downloads, specifically as an octet stream.

4. Minimal Footprint Generators should aim for minimal footprint, e.g., generating one type per file with tree-shaking capabilities to optimize performance.

5. Developer Experience High performance during code generation is important. The generator should be fast and not slow down the IDE, even during type inference.

Comparison​

• typed-openapi: Converts to Typescript before converting to runtime types (typebox-codegen → zod), losing information on constraints.
• openapi-generator: Runtime checks are limited to verifying if the value is defined.
• swagger-codegen: An older version of openapi-generator.
• autorest: Seems promising on paper, but couldn't get it to start (freezes on startup).
• openapi-io-ts: Inexistent community, doesn't work with our specs, not maintained.
• orval: Primarily targets React; generates code that works on our specs.

Exclusion Criteria​

Some tools are excluded from this benchmark due to lack of runtime controls (at least, as of the writing of this document):

• heyapi
• oazapfts
• openapi-backend
• openapi-typescript
• swagger-typescript-api

Although it is possible to translate the generated Typescript types into structures (e.g., zod/typebox) that allow runtime controls, this kind of conversion results in a loss of information on constraints (e.g., maximum, minimum, maxLength, pattern, etc.).

Generated Code Repository​

A repository is available to compare the code of different generators: GitHub - gunzip/openapi-generator-benchmark

Issues with openapi-zod-client​

1. Inaccurate type generation.
2. Poor performance in the IDE due to type inference from the single object passed to the zodios makeApi method.
3. Dependency on @zodios/core for HTTP calls (via Axios), parameter validation, and query string handling.
4. Does not support multiple successful values (2xx codes).
5. Both the client and types are contained in a single file, making it hard to optimize the bundle (important for frontend clients).
6. The maintainer of zodios has been inactive for a while.

Example of Inaccurate Type Generation​

const MessageSubject = z.string();
const MessageBodyMarkdown = z.string();
const MessageContent = z
  .object({
    subject: MessageSubject.min(10).max(120).optional(),
    markdown: MessageBodyMarkdown.min(80).max(10000),
  })
  .passthrough();

Generated from the foloowing OpenAPI schema:

MessageSubject:
      type: string
      minLength: 10
      maxLength: 120
    MessageBodyMarkdown:
      type: string
      minLength: 80
      maxLength: 10000
          MessageContent:
      type: object
    MessageContent:
      properties:
        subject:
          $ref: "#/components/schemas/MessageSubject"
        markdown:
          $ref: "#/components/schemas/MessageBodyMarkdown"
      required:
        - markdown

Issues with swagger-typescript-api + ts-to-zod​

Unlike other Typescript type generators, swagger-typescript-api retains information on constraints (minimum, maximum, maxLength, pattern, etc.) in JSDoc comments associated with type definitions. This allows for the reconstruction of runtime schemas, for example, using tools like ts-to-zod. However:

1. The generated runtime schemas are separate from the client code, so the “plumbing” to connect the runtime validation (ts-to-zod) to the generated types (swagger-typescript-api) must be implemented manually.
2. Types are not always accurate due to discrepancies between the annotations generated by swagger-typescript-api and those expected by ts-to-zod (e.g., @min vs @minimum).
3. Some inaccuracies in type generation produce invalid Typescript code.

Nevertheless, the zod schemas are more faithful than those generated by openapi-zod-client.

Example of Accurate Type Generation​

export const messageSubjectSchema = z.string().min(10).max(120);
export const messageBodyMarkdownSchema = z.string().min(80).max(10000);
export const messageContentSchema = z.object({
  subject: messageSubjectSchema.optional(),
  markdown: messageBodyMarkdownSchema,
});

Conclusion​

As of Jun 2024, no OpenAPI generator fully meets our guiding principles. While openapi-zod-client seems to cover most of the desired features, it still appears to be an immature project, not always accurate in type generation.

It is appropriate to invest in the development of a Typescript code generator from OpenAPI that performs correct and complete runtime checks.

At the heart of the Engineering Area, a group of Senior, Cloud, and Staff Engineers has decided to tackle the daily challenges that slow down our work. We're here to break down barriers, simplify processes, and make software development smoother and more rewarding for everyone.

Why DevEx​

How much time is lost before even writing a line of code? How many decisions need to be made? And how long does it take for a new engineer to become truly productive?

With DevEx, we want to answer these questions and solve the problems that hinder our daily work:

• Reduce downtime: Every second spent configuring an environment or interpreting old code is a second lost to innovation.
• Lower cognitive load: Less complexity means more room to create value, reducing errors, and speeding up time-to-market
• Speed up onboarding: Every new developer should be able to contribute from day one.

What We Do​

As DevEx team we doesn't just identify problems and suggest solutions: we tackle them head-on, getting our hands dirty.

• Common patterns and golden paths: We provide golden paths so you don't have to reinvent the wheel every time.
• Ready-to-use abstractions and tools: We reduce boilerplate code so you can focus on the code that matters.
• Optimized development environments: We share pre-configured and ready to use templates because infrastructure should never be an obstacle but a resource.
• Centralized documentation: You'll no longer have to search for an answer; it will always be at your fingertips.

How We're Organized​

Currently, the DevEx initiative is driven by engineers working asynchronously, contributing alongside their primary responsibilities within their main projects. It may not always be this way, but for now, it seems to be working!

Every two weeks, we hold a review & demo session on Google Meet. This is a key moment to review our goals (OKRs) and share our progress. Each member autonomously selects a task from the Jira DevEx Board and carries it forward, collaborating with the rest of the team through Code Reviews and RFCs.

Staff Engineers, in particular, are expected to independently select priority tasks beyond the scope of the project (domain) they are already engaged in. This approach requires collaboration from Product Managers to ensure that activities are aligned with strategic business objectives.

Innovation doesn't stop at theory: we bring solutions directly into projects, proactively supporting stream-aligned teams, and promoting the adoption of DX tools.

Where We Stand​

We started with an idea and a brainstorming session, then we got to work.

Between April and July 2024, we have...

• Identified shared technologies and patterns to focus our initial efforts on, currently: Typescript, Terraform, Azure, and GitHub Actions.

• Established some new practices such as:

  • naming conventions for Azure resources
  • structure for npm tasks for Typescript projects
  • HCL code structure for Terraform modules

• Implemented new Terraform modules for:

  • creating a GitHub Actions runner
  • setting up GitHub repository identity on Azure
  • easily creating roles and permissions for Azure services
  • creating an Azure Functions App
  • creating an Azure App Service
  • setting up autoscaling for an App Service or Function App

• Developed GitHub Actions for:

  • conducting code reviews on Typescript codebases
  • building Docker images
  • deploying Azure services via Terraform
  • planning infrastructure changes
  • notifying of drifts between IaC code and infrastructure
  • deploying to production Azure Function or App Service
  • linting Terraform code

• Set up basic configurations for Typescript tooling: eslint, yarn, turbo, changeset.

• Shared the results of a benchmark on tools for generating clients from OpenAPI specifications

• Analyzed the state of the art for distributed monitoring and log correlation on Azure

While it may seem like a lot of material, we recognize that without proper documentation and effective communication, our efforts might be overlooked. That's why we've launched a dedicated website to be populated with content and continuously updated as a reference for all engineers in the organization.

And that's not all! We're also working to provide tools (scaffolding) that can automate repetitive and redundant tasks.

Pilot Projects​

Although still in its early stages, much of the tooling we’ve developed has already been successfully used in several real projects.

Here are some examples!

Trial System​

The team that implemented the Trial System used DevEx tooling to create a platform that allows any digital service to segment users and test new features on a selected group (feature flags):

https://github.com/pagopa/trial-system

IO FIMS​

IO FIMS is a project that manages Single Sign-On for IO users. The team used DevEx tooling to create a new service:

https://github.com/pagopa/io-fims

IO communication​

The team that implemented the messaging service for IO used DevEx tooling to create a new Typescript monorepo and manage the deployment of Azure Functions:

https://github.com/pagopa/io-messages

IO authentication​

The team that implemented the user authentication service for IO used DevEx tooling to create a new Typescript monorepo and manage the deployment of Azure Functions:

https://github.com/pagopa/io-auth-n-identity-domain

IO services​

The team that implemented the backoffice for organizations in IO used DevEx tooling to manage the deployment of infrastructure and Azure Functions:

https://github.com/search?q=repo%3Apagopa%2Fio-services-cms+pagopa%2Fdx&type=code

Current Benefits of DevEx​

Let's share some brief code snippets to show how DevEx tooling can make engineers' work easier and faster.

Setting Up GitHub Repository Permissions on Azure​

> cd io-messages/infra/identity
> find . -type f
./prod/outputs.tf
./prod/locals.tf
./prod/main.tf
./prod/README.md
./prod/.terraform.lock.hcl
> find . -type f | xargs wc -l | tail -n 1 | awk '{print $1}'
160 # LoC

> cd io-services-cms/.identity
> find . -type f
.
./github_repository.tf
./99_locals.tf
./main.tf
./03_github_environment_ci.tf
./01_data.tf
./env/prod/backend.ini
./env/prod/terraform.tfvars
./env/prod/backend.tfvars
./99_variables.tf
./04_github_identity.tf
./terraform.sh
./03_github_environment_infra.tf
./99_outputs.tf
./03_github_repo_secrets.tf
./.terraform.lock.hcl
./03_github_environment_cd.tf
./03_github_environment_opex.tf
> find . -type f | xargs wc -l | tail -n 1 | awk '{print $1}'
731 # LoC

Setting Up Autoscaling for an App Service or Function App​

module "function_app_user_autoscaler" {
  source = "github.com/pagopa/dx//infra/modules/azure_app_service_plan_autoscaler?ref=main"

  resource_group_name = var.resource_group_name

  target_service = {
    function_app_name = module.function_app_user.function_app.function_app.name
  }

  scheduler = {
    maximum = 30
    normal_load = {
      default = 5
      minimum = 3
    }
  }

  scale_metrics = {
    cpu = {
      upper_threshold = 50
      increase_by = 2
    }
  }
}

resource "azurerm_monitor_autoscale_setting" "cms_fn" {
  name                = "${var.prefix}-${var.env_short}-${var.location_short}-${var.domain}-cms-func-as-01"
  resource_group_name = module.cms_fn.function_app.resource_group_name
  location            = var.location
  target_resource_id  = module.cms_fn.function_app.plan.id

  profile {
    name = "default"

    capacity {
      default = local.cms.autoscale_settings.default
      minimum = local.cms.autoscale_settings.min
      maximum = local.cms.autoscale_settings.max
    }

    rule {
      metric_trigger {
        metric_name              = "Requests"
        metric_resource_id       = module.cms_fn.function_app.function_app.id
        metric_namespace         = "microsoft.web/sites"
        time_grain               = "PT1M"
        statistic                = "Average"
        time_window              = "PT1M"
        time_aggregation         = "Average"
        operator                 = "GreaterThan"
        threshold                = 3000
        divide_by_instance_count = false
      }

      scale_action {
        direction = "Increase"
        type      = "ChangeCount"
        value     = "2"
        cooldown  = "PT1M"
      }
    }

    rule {
      metric_trigger {
        metric_name              = "CpuPercentage"
        metric_resource_id       = module.cms_fn.function_app.plan.id
        metric_namespace         = "microsoft.web/serverfarms"
        time_grain               = "PT1M"
        statistic                = "Average"
        time_window              = "PT5M"
        time_aggregation         = "Average"
        operator                 = "GreaterThan"
        threshold                = 60
        divide_by_instance_count = false
      }

      scale_action {
        direction = "Increase"
        type      = "ChangeCount"
        value     = "2"
        cooldown  = "PT5M"
      }
    }

    rule {
      metric_trigger {
        metric_name        = "MemoryPercentage"
        metric_resource_id = module.cms_fn.function_app.plan.id
        metric_namespace   = "microsoft.web/serverfarms"
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT5M"
        time_aggregation   = "Average"
        operator           = "GreaterThan"
        threshold          = 80
      }
      scale_action {
        direction = "Increase"
        type      = "ChangeCount"
        value     = "2"
        cooldown  = "PT5M"
      }
    }

    rule {
      metric_trigger {
        metric_name              = "Requests"
        metric_resource_id       = module.cms_fn.function_app.function_app.id
        metric_namespace         = "microsoft.web/sites"
        time_grain               = "PT1M"
        statistic                = "Average"
        time_window              = "PT7M"
        time_aggregation         = "Average"
        operator                 = "LessThan"
        threshold                = 2000
        divide_by_instance_count = false
      }

      scale_action {
        direction = "Decrease"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT5M"
      }
    }

    rule {
      metric_trigger {
        metric_name              = "CpuPercentage"
        metric_resource_id       = module.cms_fn.function_app.plan.id
        metric_namespace         = "microsoft.web/serverfarms"
        time_grain               = "PT1M"
        statistic                = "Average"
        time_window              = "PT7M"
        time_aggregation         = "Average"
        operator                 = "LessThan"
        threshold                30
        divide_by instance_count = false
      }

      scale_action {
        direction = "Decrease"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT5M"
      }
    }

    rule {
      metric_trigger {
        metric_name        = "MemoryPercentage"
        metric_resource_id = module.cms_fn.function_app.plan.id
        metric_namespace   = "microsoft.web/serverfarms"
        time_grain         = "PT1M"
        statistic          = "Average"
        time_window        = "PT7M"
        time_aggregation   = "Average"
        operator           = "LessThan"
        threshold          30
      }
      scale_action {
        direction = "Decrease"
        type      = "ChangeCount"
        value     = "1"
        cooldown  = "PT5M"
      }
    }
  }
}

Deploying an Azure Function App​

name: Deploy (op-func)

on:
  workflow_dispatch:

jobs:
  op_func_deploy:
    uses: pagopa/dx/.github/workflows/web_app_deploy.yaml@add-web-app-deploy-workflow
    name: Deploy
    secrets: inherit
    with:
      workspace_name: op-func
      environment: app-prod
      resource_group_name: io-p-weu-fims-rg-01
      web_app_name: io-p-weu-fims-op-func-01
      use_staging_slot: false
      use_private_agent: true

# Azure DevOps pipeline to release a new version and deploy to production.

variables:
  HEALTHCHECK_PATH: "api/info"

parameters:
  - name: "RELEASE_SEMVER"
    displayName: "When packing a release, define the version bump to apply"
    type: string
    values:
      - major
      - minor
      - patch
    default: minor
  # Map of production apps to deploy to, in the form
  #   {logicName}:
  #     appname: {name of the resource}
  #     rg: {name of the resource group}
  # Although it's a parameter, it's not intended to be edited at runtime.
  # It's here because variables only handle scalar values
  - name: "PRODUCTION_APPS"
    displayName: ""
    type: object
    default:
      servicesfn1:
        appname: io-p-services-fn-1
        rg: io-p-services-rg-1
      servicesfn2:
        appname: io-p-services-fn-2
        rg: io-p-services-rg-2

# Only manual activations are intended
trigger: none
pr: none

# This pipeline has been implemented to be run on hosted agent

 pools based both
# on 'windows' and 'ubuntu' virtual machine images and using the scripts defined
# in the package.json file. Since we are deploying on Azure functions on Windows
# runtime, the pipeline is currently configured to use a Windows hosted image for
# the build and deploy.
pool:
  vmImage: "ubuntu-latest"

resources:
  repositories:
    - repository: pagopaCommons
      type: github
      name: pagopa/azure-pipeline-templates
      ref: refs/tags/v18
      endpoint: "io-azure-devops-github-ro"

stages:
  # Create a release
  # Activated when ONE OF these are met:
  # - is on branch master
  # - is a tag in the form v{version}-RELEASE
  - stage: Release
    condition:
      and( succeeded(), or( eq(variables['Build.SourceBranch'],
      'refs/heads/master'), and( startsWith(variables['Build.SourceBranch'],
      'refs/tags'), endsWith(variables['Build.SourceBranch'], '-RELEASE') ) ) )
    pool:
      vmImage: "ubuntu-latest"
    jobs:
      - job: make_release
        steps:
          - ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/master') }}:
              - template: templates/node-job-setup/template.yaml@pagopaCommons
                parameters:
                  persistCredentials: true
              - template: templates/node-github-release/template.yaml@pagopaCommons
                parameters:
                  semver: "${{ parameters.RELEASE_SEMVER }}"
                  gitEmail: $(GIT_EMAIL)
                  gitUsername: $(GIT_USERNAME)
                  gitHubConnection: $(GITHUB_CONNECTION)

          - ${{ if ne(variables['Build.SourceBranch'], 'refs/heads/master') }}:
              - script: |
                  echo "We assume this reference to be a valid release: $(Build.SourceBranch). Therefore, there is no need to bundle a new release."
                displayName: "Skip release bundle"

  # Prepare Artifact
  - stage: Prepare_artifact
    dependsOn:
      - Release
    jobs:
      - job: "prepare_artifact"
        steps:
          # Build application
          - template: templates/node-job-setup/template.yaml@pagopaCommons
            parameters:
              # On the assumption that this stage is executed only when Release stage is,
              #  with this parameter we set the reference the deploy script must pull changes from.
              # The branch/tag name is calculated from the source branch
              #  ex: Build.SourceBranch=refs/heads/master --> master
              #  ex: Build.SourceBranch=refs/tags/v1.2.3-RELEASE --> v1.2.3-RELEASE
              gitReference:
                ${{ replace(replace(variables['Build.SourceBranch'],
                'refs/tags/', ''), 'refs/heads/', '') }}
          - script: |
              yarn predeploy
            displayName: "Build"

          # Install functions extensions
          - task: DotNetCoreCLI@2
            inputs:
              command: "build"
              arguments: "-o bin"
          # Copy application to
          - task: CopyFiles@2
            inputs:
              SourceFolder: "$(System.DefaultWorkingDirectory)"
              TargetFolder: "$(System.DefaultWorkingDirectory)/bundle"
              Contents: |
                **/*
                !.git/**/*
                !**/*.js.map
                !**/*.ts
                !.vscode/**/*
                !.devops/**/*
                !.prettierrc
                !.gitignore
                !README.md
                !jest.config.js
                !local.settings.json
                !test
                !tsconfig.json
                !tslint.json
                !yarn.lock
                !Dangerfile.js
                !CODEOWNERS
                !__*/**/*
            displayName: "Copy deploy files"

          - publish: $(System.DefaultWorkingDirectory)/bundle
            artifact: Bundle

  # Deploy on staging slot
  - ${{ each app in parameters.PRODUCTION_APPS }}:
      - stage: Deploy_${{ app.Key }}_to_staging
        dependsOn:
          - Prepare_artifact
        jobs:
          - job: "do_deploy_${{ app.Key }}"
            steps:
              - checkout: none
              - download: current
                artifact: Bundle

              - task: AzureFunctionApp@1
                inputs:
                  azureSubscription: "$(PRODUCTION_AZURE_SUBSCRIPTION)"
                  resourceGroupName: "${{ app.Value.rg }}"
                  appType: "functionApp"
                  appName: "${{ app.Value.appname }}"
                  package: "$(Pipeline.Workspace)/Bundle"
                  deploymentMethod: "auto"
                  deployToSlotOrASE: true
                  slotName: "staging"
                displayName: Deploy to staging slot

  # Check that the staging instance is healthy
  - ${{ each app in parameters.PRODUCTION_APPS }}:
      - stage: Healthcheck_${{ app.Key }}
        dependsOn:
          - Deploy_${{ app.Key }}_to_staging
        pool:
          name: $(AGENT_POOL)
        jobs:
          - job: "do_healthcheck_${{ app.Key }}"
            steps:
              - checkout: none
              - script: |
                  # fails if response status is not 2xx
                  curl -f 'https://${{ app.Value.appname }}-staging.azurewebsites.net/$(HEALTHCHECK_PATH)'
                displayName: "Healthcheck"

  # Promote the staging instance to production
  - ${{ each app in parameters.PRODUCTION_APPS }}:
      - stage: Swap_${{ app.Key }}_to_production
        dependsOn:
          - Deploy_${{ app.Key }}_to_staging
          # Wait for every healthcheck to succeed
          # This implied that no app is swapped to prod if at least one healthcheck fails
          - ${{ each appInner in parameters.PRODUCTION_APPS }}:
              - Healthcheck_${{ appInner.Key }}
        jobs:
          - job: "do_deploy_${{ app.Key }}"
            steps:
              - checkout: none
              - task: AzureAppServiceManage@0
                inputs:
                  azureSubscription: "$(PRODUCTION_AZURE_SUBSCRIPTION)"
                  resourceGroupName: "${{ app.Value.rg }}"
                  webAppName: "${{ app.Value.appname }}"
                  sourceSlot: staging
                  swapWithProduction: true
                displayName: Swap with production slot

  # Publish client SDK to NPM
  - stage: PublishClientSDKtoNPM
    dependsOn: Release
    pool:
      vmImage: "ubuntu-latest"
    jobs:
      - job: publish_SDK
        steps:
          # Template for generating and deploying client SDK to NPM
          - template: templates/client-sdk-publish/template.yaml@pagopaCommons
            parameters:
              openapiSpecPath: "openapi/index.yaml"

Applying Terraform Changes to Cloud Resources​

name: Release Infrastructure

on:
  workflow_dispatch:
  push:
    branches:
      - main
    paths:
      - "infra/resources/**"
      # Terraform version edit
      - ".terraform-version"

jobs:
  release_prod:
    uses: pagopa/dx/.github/workflows/infra_apply.yaml@3585afcf2d306da91d68cabfd0344a82e1ee10f3
    name: Release Infrastructure
    secrets: inherit
    with:
      environment: prod
      base_path: infra/resources
      override_github_environment: "infra-prod"
      use_private_agent: true

name: Continuous Delivery on prod citizen-auth

on:
  workflow_dispatch:
  # Trigger the workflow on push on the main branch
  push:
    branches:
      - main
    paths:
      - "src/domains/citizen-auth**"
      - ".github/workflows/prod**"

permissions:
  id-token: write
  contents: read

env:
  DIR: "src/domains/citizen-auth"
  AZURE_ENVIRONMENT: prod

jobs:
  create_runner:
    name: Create Runner
    runs-on: ubuntu-22.04
    environment: prod-runner
    outputs:
      runner_name: ${{ steps.create_github_runner.outputs.runner_name }}
    steps:
      - name: Create GitHub Runner
        id: create_github_runner
        # from https://github.com/pagopa/github-self-hosted-runner-azure-create-action/commits/main
        uses: pagopa/github-self-hosted-runner-azure-create-action@b4590e069e753daee6bc9809d484523cc7026035
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          container_app_environment_name:
            ${{ secrets.AZURE_CONTAINER_APP_ENVIRONMENT_NAME }}
          resource_group_name: ${{ secrets.AZURE_RESOURCE_GROUP_NAME }}
          pat_token: ${{ secrets.BOT_TOKEN }}

  terraform_preapply_job:
    name: Terraform Pre Apply
    runs-on: [self-hosted, "${{ needs.create_runner.outputs.runner_name }}"]
    environment: prod-ci
    needs: create_runner
    steps:
      - name: Checkout
        id: checkout
        # from https://github.com/actions/checkout/commits/main
        uses: actions/checkout@1f9a0c22da41e6ebfa534300ef656657ea2c6707
        with:
          persist-credentials: false
          fetch-depth: 0

      - name: Setup terraform
        id: setup-version
        # https://github.com/pagopa/terraform-install-action/commits/main
        uses: pagopa/terraform-install-action@1f76f593176e58c423b88d72273a612ba7ba430b

      - name: Terraform pre apply common
        # from https://github.com/pagopa/terraform-preapply-azure-action/commits/main
        uses: pagopa/terraform-preapply-azure-action@70c6db53dea31f0ddc65ed43e9b7f013859be64e
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-common
          azure_environment: prod

      - name: Terraform pre apply weu-beta
        # from https://github.com/pagopa/terraform-preapply-azure-action/commits/main
        uses: pagopa/terraform-preapply-azure-action@70c6db53dea31f0ddc65ed43e9b7f013859be64e
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-app
          azure_environment: weu-beta

      - name: Terraform pre apply weu-prod01
        # from https://github.com/pagopa/terraform-preapply-azure-action/commits/main
        uses: pagopa/terraform-preapply-azure-action@70c6db53dea31f0ddc65ed43e9b7f013859be64e
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-app
          azure_environment: weu-prod01

  terraform_apply_job:
    name: Terraform Apply
    runs-on: [self-hosted, "${{ needs.create_runner.outputs.runner_name }}"]
    environment: prod-cd
    needs: [create_runner, terraform_preapply_job]
    steps:
      - name: Terraform apply common
        # from https://github.com/pagopa/terraform-apply-azure-action/commits/main
        uses: pagopa/terraform-apply-azure-action@845c8b98c03f6c364e11b755935b4d29a1a46e0d
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-common

      - name: Terraform apply weu-beta
        # from https://github.com/pagopa/terraform-apply-azure-action/commits/main
        uses: pagopa/terraform-apply-azure-action@845c8b98c03f6c364e11b755935b4d29a1a46e0d
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-app

      - name: Terraform apply weu-prod01
        # from https://github.com/pagopa/terraform-apply-azure-action/commits/main
        uses: pagopa/terraform-apply-azure-action@845c8b98c03f6c364e11b755935b4d29a1a46e0d
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          dir: ${{ env.DIR }}-app

  cleanup_runner:
    name: Cleanup Runner
    if: always()
    runs-on: ubuntu-22.04
    environment: prod-runner
    needs: [create_runner, terraform_preapply_job, terraform_apply_job]
    steps:
      - name: Cleanup GitHub Runner
        id: cleanup_github_runner
        # from https://github.com/pagopa/github-self-hosted-runner-azure-cleanup-action/commits/main
        uses: pagopa/github-self-hosted-runner-azure-cleanup-action@97731a35e6ffc79b66c4dfd2aae5e4fd04e3ebb5
        with:
          client_id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant_id: ${{ secrets.AZURE_TENANT_ID }}
          subscription_id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          resource_group_name: ${{ secrets.AZURE_RESOURCE_GROUP_NAME }}
          runner_name: ${{ needs.create_runner.outputs.runner_name }}
          pat_token: ${{ secrets.BOT_TOKEN }}

Creating a Production Ready API Management Instance​

module "azure_apim" {
  source = "../../"

  environment         = local.environment
  resource_group_name = azurerm_resource_group.example.name
  use_case            = "high_load"

  # Change this values
  publisher_email = "example@pagopa.it"
  publisher_name  = "Example Publisher"

  virtual_network = {
    name                = local.virtual_network.name
    resource_group_name = local.virtual_network.resource_group_name
  }
  subnet_id                     = azurerm_subnet.example.id
  virtual_network_type_internal = true

  tags = local.tags
}

module "apim" {
  source = "git::https://github.com/pagopa/terraform-azurerm-v3.git//api_management?ref=v4.1.15"

  subnet_id                 = module.apim_snet.id
  location                  = azurerm_resource_group.rg_internal.location
  name                      = format("%s-apim-api", local.project)
  resource_group_name       = azurerm_resource_group.rg_internal.name
  publisher_name            = var.apim_publisher_name
  publisher_email           = data.azurerm_key_vault_secret.apim_publisher_email.value
  notification_sender_email = data.azurerm_key_vault_secret.apim_publisher_email.value
  sku_name                  = var.apim_sku
  virtual_network_type      = "Internal"

  # not used at the moment
  redis_connection_string = null # module.redis_apim.primary_connection_string
  redis_cache_id          = null # module.redis_apim.id

  # This enables the Username and Password Identity Provider
  sign_up_enabled = false

  hostname_configuration = {
    proxy = [
      {
        # io-p-apim-api.azure-api.net
        default_ssl_binding = false
        host_name           = "io-p-apim-api.azure-api.net"
        key_vault_id        = null
      },
      {
        # api-internal.io.italia.it
        default_ssl_binding = true
        host_name           = local.apim_hostname_api_internal
        key_vault_id = replace(
          data.azurerm_key_vault_certificate.api_internal_io_italia_it.secret_id,
          "/${data.azurerm_key_vault_certificate.api_internal_io_italia_it.version}",
          ""
        )
      },
      {
        # api-app.internal.io.pagopa.it
        default_ssl_binding = false
        host_name           = local.apim_hostname_api_app_internal
        key_vault_id = replace(
          data.azurerm_key_vault_certificate.api_app_internal_io_pagopa_it.secret_id,
          "/${data.azurerm_key_vault_certificate.api_app_internal_io_pagopa_it.version}",
          ""
        )
      },
    ]
    developer_portal = null
    management       = null
    portal           = null
  }

  application_insights_instrumentation_key = data.azurerm_application_insights.application_insights.instrumentation_key

  lock_enable = false # no lock

  autoscale = var.apim_autoscale

  alerts_enabled = var.apim_alerts_enabled

  action = [
    {
      action_group_id    = azurerm_monitor_action_group.slack.id
      webhook_properties = null
    },
    {
      action_group_id    = azurerm_monitor_action_group.email.id
      webhook_properties = null
    }
  ]

  # metrics docs
  # https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported#microsoftapimanagementservice
  metric_alerts = {
    capacity = {
      description   = "Apim used capacity is too high"
      frequency     = "PT5M"
      window_size   = "PT5M"
      severity      = 1
      auto_mitigate = true

      criteria = [{
        metric_namespace       = "Microsoft.ApiManagement/service"
        metric_name            = "Capacity"
        aggregation            = "Average"
        operator               = "GreaterThan"
        threshold              = 40
        skip_metric_validation = false
        dimension              = []
      }]
      dynamic_criteria = []
    }

    duration = {
      description   = "Apim abnormal response time"
      frequency     = "PT5M"
      window_size   = "PT5M"
      severity      = 2
      auto_mitigate = true

      criteria = []

      dynamic_criteria = [{
        metric_namespace         = "Microsoft.ApiManagement/service"
        metric_name              = "Duration"
        aggregation              = "Average"
        operator                 = "GreaterThan"
        alert_sensitivity        = "High"
        evaluation_total_count   = 2
        evaluation_failure_count = 2
        skip_metric_validation   = false
        ignore_data_before       = "2021-01-01T00:00:00Z" # sample data
        dimension                = []
      }]
    }

    requests_failed = {
      description   = "Apim abnormal failed requests"
      frequency     = "PT5M"
      window_size   = "PT5M"
      severity      = 2
      auto_mitigate = true

      criteria = []

      dynamic_criteria = [{
        metric_namespace         = "Microsoft.ApiManagement/service"
        metric_name              = "Requests"
        aggregation              = "Total"
        operator                 = "GreaterThan"
        alert_sensitivity        = "High"
        evaluation_total_count   = 2
        evaluation_failure_count = 2
        skip_metric_validation   = false
        ignore_data_before       = "2021-01-01T00:00:00Z" # sample data
        dimension = [{
          name     = "BackendResponseCode"
          operator = "Include"
          values   = ["5xx"]
        }]
      }]
    }
  }

  tags = var.tags
}