Create a DORA Insight

Enhancing the efficiency of software delivery is paramount for the success of your engineering team. However, achieving this without a clear understanding of the key performance indicators (KPIs) associated with the software delivery process is not easy. Without the right data, it becomes challenging to make well-informed decisions on optimizing workflows and communicating progress and achievements to stakeholders becomes equally difficult.

That's where DORA metrics come in handy. By using Harness SEI, you can streamline the process of identifying bottlenecks, get teams working together better, accelerate delivery speed, and articulate your team's accomplishments effectively.

This tutorial explains how to set up a DORA metrics Insight on Harness SEI, ensuring a smoother and simpler software delivery process for your engineering team.

Prerequisites

• Complete setting up your Projects and Collection

• Install and map the integrations for your Issue Management Platform, Source Code Manager and Deployment Manager.

  • To create a new Jira integration, go to Jira integration.

  • To create a new GitHub integration, go to GitHub integration.

  • To create a new Harness NG integration, go to Harness NG integration.

• DORA Profile (DORA type Workflow Profile)

DORA Profile

DORA metrics are vital for evaluating and enhancing engineering team performance. The available DORA metrics are Lead Time for Changes, Deployment Frequency, Mean Time to Restore, and Change Failure Rate. You can adjust the threshold and definition for each of these metrics in the associated DORA Profile which acts as the single source of truth for your DORA Metrics calculations.

To create or edit a DORA type Workflow profile:

• Go to the Account Settings of your SEI account

• Select Workflow under Profiles.

• To create a profile, select +New Workflow Profile and select the DORA profile. To edit an existing DORA-type workflow profile, select the profile's name in the profiles list.

To calculate the DORA Metrics, this profile must be associated with the collection under which your DORA Insight is configured.

To learn more about customizing an existing DORA profile or creating a new profile, go to DORA type Workflow Profile.

Create the Insight

1. Log in to your SEI Account and go to the Project where you have your integration installed.

2. If you don't have any existing Insight, then click on the Create Insight button on the landing page to create a new Insight.

3. If you already have existing Insights in your selected Project, then go to any Insight. For instructions, go to View Insights.

4. In the header, select All Insights, and then select Manage Insights.

5. Select Create Insight.

Insight settings

1. Enter a Name for the Insight.

2. Select at least one Collection category to associate with this Insight.

3. Select Create to save the Insight metadata. From here, you can add reports to this Insight.

Add DORA reports

You can use a DORA Metrics Insight to examine your organization's DORA metrics. This helps you understand how your organization or team is performing and helps you get an overview of daily, weekly, and monthly trends.

Deployment Frequency report

Include DORA metrics reports to understand how well your team is doing. The Deployment Frequency report tells you how often your team successfully releases software.

To keep track of Deployment Frequency, set up a Workflow profile. This helps you pick what to monitor like merged pull requests or CI/CD jobs for the associated Collections. You can adjust Workflow profiles to fit your team's way of working, focusing on specific steps like Source Code Management (SCM) or a combination of issue management, SCM, and CI/CD. For more information, go to Workflow profile.

To add the Deployment Frequency widget to Insights:

1. Select Settings, and then select Add Widget.

2. Select the Deployment Frequency widget.

3. Adjust the widget settings as needed.

Modifying the collection filters in the widget settings will direct you to the Edit Collection tab. Changes made here will impact the entire collection, not just the widget.

1. Select Next: Place Widget, place the widget on the Insight, and then select Save Layout.

The widget automatically detects the relevant Workflow profile based on the Collections associated with the Insight.

Lead Time For Changes report

DORA calculation for Lead Time is similar to how lead time, in general, is calculated, with the difference being the ability to associate a collection while defining the profile, i.e., at the profile level.

This report represents the amount of time it takes for a commit to get into production.

To add the Lead Time for Changes widget to Insights:

1. Select Settings, and then select Add Widget.

2. Select the Lead Time for Changes widget.

3. Configure filters to refine conditions (e.g., Issue Resolved In for the last quarter) impacting overall lead time calculations.

4. Select Average Time in the Stage as the metric under the metrics tab.

5. Under the Settings tab, specify the relevant Workflow profile.

6. Select Next: Place Widget, place the widget on the Insight and then select Save Layout.

For information, go to Lead time reports.

Change Failure Rate report

Change Failure Rate represents the percentage of deployments that cause a failure in production. To monitor Change Failure Rate in SEI, you will again need to associate the widget with an existing workflow profile.

To add the Change Failure Rate Report widget to Insights:

1. Select Settings, and then select Add Widget.

2. Select the Change Failure Rate widget.

3. Configure the widget settings. Similar to the Deployment Frequency report customizing the Collection-level Filters redirects you to the Edit Collection tab.

4. Select Next: Place Widget, select where you want to place the widget on the Insight, and then select Save Layout.

The Change Failure Rate widget is now part of your Insight.

Mean Time To Restore (MTTR) report

The Mean Time To Restore (MTTR), also known as Time to Recover, represents the duration it takes for an organization to recover from a production failure. This metric serves as a valuable measure for evaluating the efficiency of the recovery process.

The overall time can be analyzed stage by stage over the organization's failure recovery workflow.

To add the DORA Mean Time To Restore report to the Insight:

1. Select Settings, and then select Add Widget.

2. Select the Mean Time To Restore widget.

3. Customize the widget by configuring Filters. This step allows you to specify conditions (such as Issue Created In/Resolved In) that contribute to the overall calculations. In this case, we will add the Issue Resolved In filter to restrict our calculation only for the last quarter.

4. Select the metric as Average Time in Stage under the Metrics tab

5. On the Settings tab, select the relevant Workflow profile.

6. Select Next: Place Widget, place the widget on the Insight and then select Save Layout.

Best Practices

• Ensure that the selected DORA metrics align with your organization's overarching business goals and objectives.

• Standardize workflow profiles across teams to maintain consistency in metric tracking.

• Define workflow profile at a granular level to capture the entire software delivery lifecycle. This includes specifying events such as the exclusion of PRs and commits, events defining what constitutes a deployment or a failure for CI/CD jobs, and other crucial items to provide a comprehensive view of the deployment process.

• Customize collection filters within widgets carefully. These filters impact not only the widget but the collection as a whole. Optimize filters to focus on the specific aspects of your software development process that you want to analyze.