Skip to main content
Version: v0.20

Supported Data Sources

Data Sources and Data Plugins

Apache DevLake(incubating) supports the following data sources. The data from each data source is collected with one or more plugins. Detailed plugin docs can be found here.

Data SourceDomain(s)Supported VersionsConfig UI AvailabilityTriggered PluginsCollection Mode
GitHubSource Code Management, Code Review, Issue Tracking, CI/CD (GitHub Actions)CloudAvailablegithub, gitextractorIncremental Sync
GitLabSource Code Management, Code Review, Issue Tracking, CI/CD (GitLab CI)Cloud, Community Edition 11+Availablegitlab, gitextractorFull Refresh, Incremental Sync(for issues,MRs)
JiraIssue TrackingCloud, Server/Data Center 7.x, 8.xAvailablejiraFull Refresh, Incremental Sync(for issues and related)
JenkinsCI/CD2.263.x+AvailablejenkinsIncremental Sync
BitBucket (Beta)Source Code Management, Code ReviewCloudAdvanced Mode Availablebitbucket, gitextractorFull Refresh
TAPD (Beta)Issue TrackingCloudAdvanced Mode AvailabletapdFull Refresh, Incremental Sync(for stories, bugs, tasks)
Teambition (Beta)Issue TrackingCloudAdvanced Mode AvailableteambitionFull Refresh
Zentao (Beta)Issue Trackingv17.x, v18.xAdvanced Mode AvailablezentaoFull Refresh
Gitee (WIP)Source Code Management, Code Review, Issue TrackingCloudNot Availablegitee, gitextractorFull Refresh, Incremental Sync(for issues,MRs)
PagerDutyIssue TrackingCloudAvailablepagerdutyIncremental Sync
Feishu (WIP)CalendarCloudNot AvailablefeishuFull Refresh
AESource Code ManagementOn-premNot AvailableaeFull Refresh
SonarqubeCODE QUALITYSonarQube v8.x, v9.xAvailablesonarqubeFull Refresh
Bamboo CI(WIP)CI/CDv6.8.1Not AvailablebambooFull Refresh
Azure Devops (Beta)CI/CD, Source Code Management, Code ReviewCloudAvailableazuredevops, gitextractorFull Refresh

Data Collection Scope By Each Plugin

This table shows the entities collected by each plugin. Domain layer entities in this table are consistent with the entities here. ✅ : Collect by default. 💪 : Collect not by default. You need to add the corresponding subtasks to collect these entities in the advanced mode.

Domain Layer Entitiesaedoragitextractorincoming webhookGitHubGitLabJenkinsJirarefdiffTAPDSonarQubeBambooAzure DevOpsBitBucket
accounts
board_issues
board_repos
board_sprints
boards
cicd_deployment_commits
cicd_deployments
cicd_pipeline_commits
cicd_pipelines
cicd_scopes
cicd_tasks💪
commit_file_components
commit_files
commit_line_change
commit_parents
commits💪💪
commits_diffs
components
finished_commits_diffs
issue_changelogs
issue_comments
issue_commits
issue_labels
issue_repo_commits
issue_worklogs
issues
project_issue_metrics
project_mapping
project_metrics
project_pr_metrics
project
pull_request_comments
pull_request_commits
pull_request_issues
pull_request_labels
pull_requests
ref_commits
refs
refs_issues_diffs
ref_pr_cherry_picks
repo_commits💪💪
repo_snapshot
repos
sprint_issues
sprints
team_users
teams
user_account
users
cq_projects
cq_issues
cq_issue_code_blocks
cq_file_metrics

Data Sync Policy

bold: means it may collect slowly.

*bold*: means it may collect very slowly.

Jira

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectStatusMeta1--
CollectProjectsMeta<10-
CollectIssueTypesMeta<10-
CollectIssuesMeta<10^4
CollectIssueChangelogsMeta1000~10^5
CollectAccountsMeta<10^3
CollectWorklogsMeta1000~10^5
CollectRemotelinksMeta1000~10^5
CollectSprintsMeta<100
CollectEpicsMeta<100

Jenkins

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiBuildsMeta≈100
CollectApiStagesMeta≈10^4

GitLab

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiIssuesMeta<10^4
CollectApiMergeRequestsMeta<10^3
CollectApiMrNotesMeta<10^5
CollectApiMrCommitsMeta<10^5
CollectApiPipelinesMeta<10^4
CollectApiJobsMeta<10^5

Github

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
---------------------------------Common-----------------------
CollectMilestonesMeta≈10
CollectRunsMeta<10^4
CollectApiCommentsMeta400 (max page that GitHub supports)
CollectApiEventsMeta400 (max page that GitHub supports)
CollectApiPullRequestReviewsMeta<10^5
---------------------------------Graphql Only (Default)-----------------------
CollectIssueMeta≈10^4
CollectPrMeta≈10^3
CollectCheckRunMeta<10^4
CollectAccountMeta≈10^2-
---------------------------------Restful Only (Not by Default)-----------------------
CollectApiIssuesMeta≈10^4
CollectApiPullRequestsMeta≈10^2
CollectApiPullRequestCommitsMeta≈10^4
CollectApiPrReviewCommentsMeta≈10^4
CollectAccountsMeta≈10^4
CollectAccountOrgMeta≈10^4
CollectJobsMeta<10^6
CollectApiCommitsMetaNot enabled--
CollectApiCommitStatsMetaNot enabled--

Feishu

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectMeetingTopUserItemMeta≈10^3

Bitbucket

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiRepoMeta1
CollectApiPullRequestsMeta≈10^3
CollectApiIssuesMeta≈10^4
CollectApiPrCommentsMeta≈10^5
*CollectApiIssueCommentsMeta*≈10^6
CollectApiPipelinesMeta<10^4
CollectApiDeploymentsMeta<10^2

Gitee

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiRepoMeta1
CollectApiPullRequestsMeta≈10^3
CollectApiIssuesMeta≈10^4
CollectCommitsMeta?≈10^4
CollectApiPrCommentsMeta≈10^5
*CollectApiIssueCommentsMeta*≈10^6
CollectApiPullRequestCommitsMeta≈10^5
CollectApiPullRequestReviewsMeta≈10^5
*CollectApiCommitStatsMeta*≈10^6 (Not enable)

SonarQube

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectAccounts<10^4
CollectIssues<10^4
CollectHotspots<10^4
CollectFilemetrics<10^4
CollectAdditionalFilemetrics<10^4

Bamboo

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectPlanMeta<10^4
CollectJobMeta<10^5
CollectPlanBuildMeta<10^6
CollectJobBuildMeta<10^6
CollectDeployMeta1

Zentao

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectExecutionMeta<10^3
CollectStoryMeta<10^4
CollectBugMeta<10^4
CollectTaskMeta<10^4
CollectAccountMeta≈10^2
CollectDepartmentMeta≈10^2

Tapd

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectWorkitemTypesMeta≈10--
CollectStoryCustomFieldsMeta≈10--
CollectTaskCustomFieldsMeta≈10--
CollectBugCustomFieldsMeta≈10--
CollectStoryCategoriesMeta≈10--
CollectStoryStatusMeta≈10--
CollectStoryStatusLastStepMeta≈10--
CollectBugStatusMeta≈10--
CollectBugStatusLastStepMeta≈10--
CollectAccountsMeta≈10^3
CollectIterationMeta≈10^4
CollectStoryMeta≈10^4
CollectBugMeta≈10^4
CollectTaskMeta≈10^4
CollectBugChangelogMeta≈10^6
CollectStoryChangelogMeta≈10^6
CollectTaskChangelogMeta≈10^6
CollectWorklogMeta≈10^6
CollectBugCommitMeta≈10^6
CollectStoryCommitMeta≈10^6
CollectTaskCommitMeta≈10^6
CollectStoryBugMeta≈10^6

Azure Devops

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectBuilds<10^3
CollectJobs<10^4
CollectPullRequests<10^3
CollectPullRequestCommits<10^4