Workflow Profile
Last updated
Last updated
In SEI, users can create Workflow Profiles to measure and assess the entire software development lifecycle. These profiles allow you to define the stages, events, and measurement criteria that are relevant to your development process.
By utilizing these workflow profiles, you can measure key metrics such as DORA (DevOps Research and Assessment) and Lead Time, which provide valuable insights into the time taken to ship changes or deploy bug fixes to production.
You can configure the profile depending on the factors you want to include in your calculations. For example, you can:
Define the Start of Change to start calculating lead time, which can be ticket creation in an issue management system, commits to source code, pull requests, or deployments to dev, staging, or prod environments.
Track time spent in various issue statuses, if your profile includes monitoring for an issue management system, such as Jira.
Track time spent in source code lifecycle stages, such as commit to merge and activities related to pull requests.
Track lead time and DORA metrics for your engineering teams through your CI/CD tools.
Workflow profile settings include:
When creating a new workflow profile, you will need to choose the profile type. The available profile types include DORA profile & Velocity lead time profile.
DORA Profile: Supports the four key DORA metrics - Lead time for changes, Deployment frequency, Mean time to restore, and Change failure rate.
Velocity Lead Time Profile: Supports all the pre-defined workflow profile-based widgets.
DORA metrics are vital for evaluating and enhancing engineering team performance. The available DORA metrics are Lead Time for Changes, Deployment Frequency, Mean Time to Restore, and Change Failure Rate.
Lead Time for Changes measures the time it takes for a task to progress from development to production. To configure:
Choose the tool used for tracking tasks (e.g., new features, stories, epics) in your team.
Configure the stages involved in your workflow based on the requirements.
Note that for lead time metrics you can define stages based on either of the following events:
Ticket Created: This event ensures that lead time tracking starts in issue management.
Commit Created: This event ensures that lead time tracking starts when the first commit is committed.
API Event: This This event triggers the lead time calculation based on a custom API event.
To learn more about how lead time metrics are calculated, Go to Lead Time for Changes calculation.
Deployment Frequency measures how frequently a team successfully deploys code to production. To configure:
Specify the tool used for measuring deployments in your team.
Select any existing integrations you wish to use for calculating deployment frequency.
Defind the settings for how you want to calculate deployment frequency. The additional filters being used to define the deployments will be applicable to all the integrations that you selected.
Mean Time to Restore represents the duration it takes a team to recover from a production failure. To configure:
Choose the tool used for tracking tasks similar to the Lead Time for Changes metric.
Configure the stages involved in your workflow based on the requirements.
Change Failure Rate is computed by dividing the total number of deployments causing failure by the overall number of deployments. To configure:
Specify the tool used for measuring deployments in your team.
Choose any existing integrations you want to utilize for calculating the change failure rate.
Select how to configure your integration between SCM (Source Code Management) and CI/CD.
Add attributes and filters to identify and define deployments causing failure and total deployments.
Total deployments represent all deployments that have occurred within a specified time range, regardless of whether they resulted in success or failure.
Note: The additional filters being used to define the deployments will be applicable to all the integrations that you have selected.
Lead time is based on time spent in stages defined in a Workflow profile.
For example, the default configuration for a PR-based Workflow profile has four stages:
PR creation time.
Time to Comment.
Approval time.
Merge time.
Similarly, the default configuration for a Ticket-based Workflow profile has five stages:
Lead time to First Commit.
PR Creation time.
Time to Comment.
Approval time.
Merge time.
When calculating lead time, the time spent in each stage depends on the stages that a PR or issue actually goes through. For example, if your Workflow profile includes a time to first comment stage, but there are no comments on the PR or ticket, then the time to first comment is zero.
You can configure grading thresholds (good, acceptable, and slow) for each stage. These thresholds determine grades that appear on your lead time widgets. Grades are reported for each stage as well as a cumulative grade for all stages combined.
You can modify Workflow profile stages and grades according to your team's SDLC process. If your Workflow profile includes stages across issue management, SCM, and CI/CD, make sure the same event is not tracked in multiple tools, such as Deploy to Production in Jira and a CI/CD Deploy stage.
To add or edit Workflow profiles:
Go to the Account Settings on the SEI application.
Select Workflow under Profiles.
To create a profile, select +Add Profile. To edit a profile, select the profile's name in the profiles list.
You can create profiles by copying existing profiles. Make sure to edit copies accordingly and that your Lead Time widgets reference the correct profile.
DORA Profile is a type of Workflow Profile that allows you to define the thresholds for measuring the DORA metrics for your engineering team. Follow the steps below to configure the DORA profile:
In the Workflow tab under Profiles select +Add Profile
Select DORA profile as the type for the Workflow profile.
Enter a Name for the profile.
Add a profile description. (Optional)
Define the settings for calculating the DORA Lead Time for Changes report.
Select the Issue Management Platform that you want to use to track tasks in your team.
Choose the Start Event and configure the stages involved in the workflow based on the selected event.
For tracking Lead Time only using the SCM tool, select the start event as Commit Created
For measuring Lead Time using Issue Management Platform and SCM both or only using Issue Management Platform, select the start event as Ticket Created.
When configuring a workflow profile in Jira, you have the option to add a release stage to measure Lead Time. This allows you to track the time it takes for features/issues to be rolled out to customers i.e. released.
Configure the stages involved in the workflow based on the selected event. To learn more, go to Development Stages in Lead Time for Changes.
To add custom stages, Click on the plus button within the workflow.
Add the stage name and description.
Define the Stage Definition by selecting the trigger event (options include Issue Management, Other CI/CD tools, Harness CI) and setting the event parameters.
SEI currently supports only HarnessNG integration as the CD tool for configuring stages in the Lead Time workflow.
Set acceptable time limits and target times (e.g., IDEAL TIME, ACCEPTABLE TIME) for the custom stage and save it.
When configuring a workflow profile in Jira, you have the option to add a release stage to measure Lead Time. This allows you to track the time it takes for features to be rolled out to customers by considering the Jira releases.
Modify this settings to define how you want to calculate Deployment Frequency for your engineering team.
Choose the tool that you want to use to measure deployments in your team.
Select the existing integrations that you would like to use to calculate the Deployment Frequency.
Configure the additional Filters to define the criteria for deployments that are to be considered in the Deployment Frequency calculation.
Define the settings for measuring the time it takes a team to recover from a failure in production. The configuration is similar to the settings for Lead Time.
Define the configuration for measuring the Change Failure Rate for your team.
Choose the tool that you want to use to measure deployments in your team.
Select the existing integrations that you would like to use for the calculation. Configuration details vary by SEI integration type. Default values are pre-populated, and you can change them if desired.
Select factors to use to calculate failed deployments and total deployments.
Ensure to select the checkbox in case you want to calculate the Change Failure Rate using only deployments that cause failure.
Associate the DORA profile with the Collection and Project under which you have created the DORA Insight.
Once you have finished configuring the profile setting click on Save to save the profile.
Note that you can also edit existing Collections and associate them with the DORA profile if required.
Note that for calculating DORA metrics, each profile can be associated with only one Collection in a one-to-one mapping
You can also associate Collections to existing DORA profiles from the Collections Tab. Collection categories are shown as tabs on the Collection Settings page.
To associate a DORA profile with the existing Collections, click on the Associate Workflow Profile option under the Associated Profiles column.
Select the DORA profile from the available options. The pop-up dialog box will display the list of all the profiles that can be associated with the current collection.
Select the DORA profile and click on the Associate Profile button.
To create a Velocity Lead Time profile, you need to follow these key steps:
In the Workflow tab under Profiles select +Add Profile
Select Velocity lead time profile as the type for the Workflow profile.
Enter a Name for the profile.
Add a profile description. (Optional)
Choose either JIRA or Azure as your issue management system.
In this section, you'll define the settings for Stages, New Features, Deployment, Hotfix, and Defects.
Choose the Start Event and configure the stages involved in the workflow based on the selected event.
For tracking Lead Time only using the SCM tool, select the start event as Commit Created
For measuring Lead Time using Issue Management Platform and SCM both or only using Issue Management Platform, select the start event as Ticket Created.
Define the stages involved in the workflow based on the selected event. To learn more, go to Development Stages in Lead Time calculation.
To add custom stages, Click on the plus button within the workflow.
Add the stage name and description.
Define the Stage Definition by selecting the trigger event (options include Issue Management, Other CI/CD tools, Harness CI) and set the event parameters.
Define the acceptable time limits and target times (e.g., IDEAL TIME, ACCEPTABLE TIME) for the custom stage and save it.
SEI currently supports only HarnessNG integration as the CD tool for configuring stages in the Lead Time workflow.
The following configuration settings apply to all the categories (New Features, Deployment, Hotfix, and Defects):
Pull Requests to Branches: Specify values that the branches for pull requests should start with or contain.
Pull Requests from Branches: Define values that the source branches for pull requests should start with or contain.
Direct Merges to Branches: Set criteria for branches that should be considered for direct merges, based on starting or containing specified values.
Tags on Commits of Merged PRs: Specify values that tags on commits of merged pull requests should start with or contain.
Labels on Pull Requests: Define values for labels on pull requests that should start with or contain specific values.
Configure the settings to define the criteria for New Features.
Configure the settings to define the criteria for Deployment.
Configure the settings to define the criteria for Hotfix.
Configure the settings to define the criteria for Defects.
Separate multiple values with a comma.
Depending on your chosen Start Event, the stage configuration can vary:
You can change the start event that initiates the first stage, and you can add, edit, and remove stages. When editing stages you can customize the fields, define ideal and acceptable time ranges, grades, and more. This refines how you track KPIs.
For Ticket Created Start Event by default the Development stages are enabled which includes Lead time to first commit, PR creation time, Time to Comment, Approval time, and Merge time (requires destination branch confirmation). For Approval Time you have the option to choose between choosing the First Approval or the Final Approval depending on your workflow's needs.
PR review stages cannot be rearranged. Default values for stages are customizable to meet specific requirements. We can add new custom stages at the beginning of the workflow or after the completion of the Development stages.
For Commit Created Start Event Workflow starts with Commit Created followed by PR Creation Time, Time to Comment, Approval Time, and Merge Time stages. Default values are customizable based on specific requirements. Custom stages can only be added after completing the default stages.
For API Event Start Event Development stages are the same as for Ticket Created and the PR review stages cannot be rearranged. The default values are customizable and custom stages can be added at the beginning or after the development stages.
To add custom stages, follow these steps:
Click on the plus button within the workflow.
Add a stage name and description.
Define the Stage Definition by selecting the trigger event (options include Issue Management, Other CI/CD tools, Harness CI) and set event parameters.
Set acceptable time limits and target times (e.g., IDEAL TIME, ACCEPTABLE TIME) for the custom stage and save it.
You can choose to measure lead time exclusively by JIRA statuses. This feature is especially useful for teams using JIRA as their primary issue management tool. It is a mandatory configuration for the JIRA Releases report. Since lead time is being measured by JIRA statuses only, the start event will be Ticket Created by default.
You can add custom stages to your workflow, each with a name and description. Additionally, you can define acceptable time limits for each stage to measure the lead time.
These custom stages, along with the JIRA release stage, can be used to replicate your software delivery process and measure the overall lead time.
At least one stage, apart from the release stage, is mandatory when measuring lead time by JIRA statuses. This ensures that the lead time metric captures the entire workflow process with all the steps involved in bringing an issue to its final state.
Intermediate stages allow for a more detailed analysis of the time spent at each phase of the workflow, helping teams identify potential bottlenecks and areas for optimization. These stages can only be added before the release stage.
For newly created profiles, the release stage is disabled by default in the configuration. Selecting the checkbox to measure lead time using JIRA statuses enables the release stage settings.
The Issue Management System and Start Event is disabled and cannot be customized. Since JIRA is selected as the Issue Management Platform, Ticket Created is the only start event supported
If you want to switch to Azure as the Issue Management System, you’ll have to disable the release stage from the profile configuration.
You cannot choose a start event other than Ticket Created when the release stage is configured as it will lead to incorrect settings.
For existing profiles already in use, the release stage remains disabled by default.
The Jira Release Stage allows you to track the time it takes for features to be rolled out to customers. This helps derive valuable insights into the end-to-end delivery process, beyond just the internal development lifecycle.
When you enable the JIRA Release Stage, it measures the lead time from the point where a ticket is resolved (i.e., the Last Stage before the Jira Release Stage) to the actual date when the product or feature is released to the users.
By default, the Jira release stage is disabled, so you'll need to enable it to use it. Once enabled, the starting event for the workflow is automatically set to Ticket Created and cannot be modified as it will result in incorrect configuration.Similarly, the issue management system is automatically set to Jira by default.
If you want to customize the starting event and Issue Management fields, you'll have to disable the release stage first. Once disabled, you'll be able to customize your starting event and Issue Management configuration for your profile.
In cases where a single ticket is associated with multiple versions, the user can choose between two methods for the calculation:
Considering the Earliest Released Version: This option measures lead time to the first released version linked to the ticket. This approach prioritizes the initial delivery of value to stakeholders, emphasizing the initial completion of the issue and its subsequent release.
Considering the Latest Released Version: This option measures lead time to the most recently released version associated with the ticket. This approach focuses on the final iteration of the issue, capturing the cumulative development effort and ensuring that the lead time reflects the issue's final state as it reaches users.
The following examples describe Workflow profile configurations to track Lead Time.
