跳到主要内容
版本:v0.11

Engineering Metrics

CategoryMetric NameDefinitionData RequiredUse Scenarios and Recommended PracticesValue            
Delivery VelocityRequirement CountNumber of issues in type "Requirement"Issue/Task Management entities: Jira issues, GitHub issues, etc1. Analyze the number of requirements and delivery rate of different time cycles to find the stability and trend of the development process.
2. Analyze and compare the number of requirements delivered and delivery rate of each project/team, and compare the scale of requirements of different projects.
3. Based on historical data, establish a baseline of the delivery capacity of a single iteration (optimistic, probable and pessimistic values) to provide a reference for iteration estimation.
4. Drill down to analyze the number and percentage of requirements in different phases of SDLC. Analyze rationality and identify the requirements stuck in the backlog.
1. Based on historical data, establish a baseline of the delivery capacity of a single iteration to improve the organization and planning of R&D resources.
2. Evaluate whether the delivery capacity matches the business phase and demand scale. Identify key bottlenecks and reasonably allocate resources.
Requirement Delivery RateRatio of delivered requirements to all requirementsIssue/Task Management entities: Jira issues, GitHub issues, etc
Requirement Lead TimeLead time of issues with type "Requirement"Issue/Task Management entities: Jira issues, GitHub issues, etc1. Analyze the trend of requirement lead time to observe if it has improved over time.
2. Analyze and compare the requirement lead time of each project/team to identify key projects with abnormal lead time.
3. Drill down to analyze a requirement's staying time in different phases of SDLC. Analyze the bottleneck of delivery velocity and improve the workflow.
1. Analyze key projects and critical points, identify good/to-be-improved practices that affect requirement lead time, and reduce the risk of delays
2. Focus on the end-to-end velocity of value delivery process; coordinate different parts of R&D to avoid efficiency shafts; make targeted improvements to bottlenecks.
Requirement GranularityNumber of story points associated with an issueIssue/Task Management entities: Jira issues, GitHub issues, etc1. Analyze the story points/requirement lead time of requirements to evaluate whether the ticket size, ie. requirement complexity is optimal.
2. Compare the estimated requirement granularity with the actual situation and evaluate whether the difference is reasonable by combining more microscopic workload metrics (e.g. lines of code/code equivalents)
1. Promote product teams to split requirements carefully, improve requirements quality, help developers understand requirements clearly, deliver efficiently and with high quality, and improve the project management capability of the team.
2. Establish a data-supported workload estimation model to help R&D teams calibrate their estimation methods and more accurately assess the granularity of requirements, which is useful to achieve better issue planning in project management.
Commit CountNumber of CommitsSource Code Management entities: Git/GitHub/GitLab commits1. Identify the main reasons for the unusual number of commits and the possible impact on the number of commits through comparison
2. Evaluate whether the number of commits is reasonable in conjunction with more microscopic workload metrics (e.g. lines of code/code equivalents)
1. Identify potential bottlenecks that may affect output
2. Encourage R&D practices of small step submissions and develop excellent coding habits
Added Lines of CodeAccumulated number of added lines of codeSource Code Management entities: Git/GitHub/GitLab commits1. From the project/team dimension, observe the accumulated change in Added lines to assess the team activity and code growth rate
2. From version cycle dimension, observe the active time distribution of code changes, and evaluate the effectiveness of project development model.
3. From the member dimension, observe the trend and stability of code output of each member, and identify the key points that affect code output by comparison.
1. identify potential bottlenecks that may affect the output
2. Encourage the team to implement a development model that matches the business requirements; develop excellent coding habits
Deleted Lines of CodeAccumulated number of deleted lines of codeSource Code Management entities: Git/GitHub/GitLab commits
Pull Request Review TimeTime from Pull/Merge created time until mergedSource Code Management entities: GitHub PRs, GitLab MRs, etc1. Observe the mean and distribution of code review time from the project/team/individual dimension to assess the rationality of the review time1. Take inventory of project/team code review resources to avoid lack of resources and backlog of review sessions, resulting in long waiting time
2. Encourage teams to implement an efficient and responsive code review mechanism
Bug AgeLead time of issues in type "Bug"Issue/Task Management entities: Jira issues, GitHub issues, etc1. Observe the trend of bug age and locate the key reasons.
2. According to the severity level, type (business, functional classification), affected module, source of bugs, count and observe the length of bug and incident age.
1. Help the team to establish an effective hierarchical response mechanism for bugs and incidents. Focus on the resolution of important problems in the backlog.
2. Improve team's and individual's bug/incident fixing efficiency. Identify good/to-be-improved practices that affect bug age or incident age
Incident AgeLead time of issues in type "Incident"Issue/Task Management entities: Jira issues, GitHub issues, etc
Delivery QualityPull Request CountNumber of Pull/Merge RequestsSource Code Management entities: GitHub PRs, GitLab MRs, etc1. From the developer dimension, we evaluate the code quality of developers by combining the task complexity with the metrics related to the number of review passes and review rounds.
2. From the reviewer dimension, we observe the reviewer's review style by taking into account the task complexity, the number of passes and the number of review rounds.
3. From the project/team dimension, we combine the project phase and team task complexity to aggregate the metrics related to the number of review passes and review rounds, and identify the modules with abnormal code review process and possible quality risks.
1. Code review metrics are process indicators to provide quick feedback on developers' code quality
2. Promote the team to establish a unified coding specification and standardize the code review criteria
3. Identify modules with low-quality risks in advance, optimize practices, and precipitate into reusable knowledge and tools to avoid technical debt accumulation
Pull Request Pass RateRatio of Pull/Merge Review requests to mergedSource Code Management entities: GitHub PRs, GitLab MRs, etc
Pull Request Review RoundsNumber of cycles of commits followed by comments/final mergeSource Code Management entities: GitHub PRs, GitLab MRs, etc
Pull Request Review CountNumber of Pull/Merge ReviewersSource Code Management entities: GitHub PRs, GitLab MRs, etc1. As a secondary indicator, assess the cost of labor invested in the code review process1. Take inventory of project/team code review resources to avoid long waits for review sessions due to insufficient resource input
Bug CountNumber of bugs found during testingIssue/Task Management entities: Jira issues, GitHub issues, etc1. From the project or team dimension, observe the statistics on the total number of defects, the distribution of the number of defects in each severity level/type/owner, the cumulative trend of defects, and the change trend of the defect rate in thousands of lines, etc.
2. From version cycle dimension, observe the statistics on the cumulative trend of the number of defects/defect rate, which can be used to determine whether the growth rate of defects is slowing down, showing a flat convergence trend, and is an important reference for judging the stability of software version quality
3. From the time dimension, analyze the trend of the number of test defects, defect rate to locate the key items/key points
4. Evaluate whether the software quality and test plan are reasonable by referring to CMMI standard values
1. Defect drill-down analysis to inform the development of design and code review strategies and to improve the internal QA process
2. Assist teams to locate projects/modules with higher defect severity and density, and clean up technical debts
3. Analyze critical points, identify good/to-be-improved practices that affect defect count or defect rate, to reduce the amount of future defects
Incident CountNumber of Incidents found after shippingSource Code Management entities: GitHub PRs, GitLab MRs, etc
Bugs Count per 1k Lines of CodeAmount of bugs per 1,000 lines of codeSource Code Management entities: GitHub PRs, GitLab MRs, etc
Incidents Count per 1k Lines of CodeAmount of incidents per 1,000 lines of codeSource Code Management entities: GitHub PRs, GitLab MRs, etc
Delivery CostCommit Author CountNumber of Contributors who have committed codeSource Code Management entities: Git/GitHub/GitLab commits1. As a secondary indicator, this helps assess the labor cost of participating in coding1. Take inventory of project/team R&D resource inputs, assess input-output ratio, and rationalize resource deployment
Delivery CapabilityBuild CountThe number of builds startedCI/CD entities: Jenkins PRs, GitLabCI MRs, etc1. From the project dimension, compare the number of builds and success rate by combining the project phase and the complexity of tasks
2. From the time dimension, analyze the trend of the number of builds and success rate to see if it has improved over time
1. As a process indicator, it reflects the value flow efficiency of upstream production and research links
2. Identify excellent/to-be-improved practices that impact the build, and drive the team to precipitate reusable tools and mechanisms to build infrastructure for fast and high-frequency delivery
Build DurationThe duration of successful buildsCI/CD entities: Jenkins PRs, GitLabCI MRs, etc
Build Success RateThe percentage of successful buildsCI/CD entities: Jenkins PRs, GitLabCI MRs, etc