Skip to main content
Version: v0.12

Developer Setup

Requirements​

How to setup dev environment​

  1. Navigate to where you would like to install this project and clone the repository:

    git clone https://github.com/apache/incubator-devlake
    cd incubator-devlake

  2. Install dependencies for plugins:

  3. Install Go packages

    go get

  4. Copy the sample config file to new local file:

    cp .env.example .env

  5. Update the following variables in the file .env:

    • DB_URL: Replace mysql:3306 with 127.0.0.1:3306

  6. Start the MySQL and Grafana containers:

    Make sure the Docker daemon is running before this step.

    docker-compose up -d mysql grafana

  7. Run lake and config UI in dev mode in two separate terminals:

    # install mockery
    go install github.com/vektra/mockery/v2@latest
    # generate mocking stubs
    make mock
    # run lake
    make dev
    # run config UI
    make configure-dev

    Q: I got an error saying: libgit2.so.1.3: cannot open share object file: No such file or directory

    A: Make sure your program can find libgit2.so.1.3. LD_LIBRARY_PATH can be assigned like this if your libgit2.so.1.3 is located at /usr/local/lib:

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

  8. Visit config UI at localhost:4000 to configure data connections.

    • Please follow the tutorial
    • Submit the form to update the values by clicking on the Save Connection button on each form page

  9. Visit localhost:4000/pipelines/create to RUN a Pipeline and trigger data collection.

Pipelines Runs can be initiated by the new "Create Run" Interface. Simply enable the Data Connection Providers you wish to run collection for, and specify the data you want to collect, for instance, Project ID for Gitlab and Repository Name for GitHub.

Once a valid pipeline configuration has been created, press Create Run to start/run the pipeline. After the pipeline starts, you will be automatically redirected to the Pipeline Activity screen to monitor collection activity.

Pipelines is accessible from the main menu of the config-ui for easy access.

  • Manage All Pipelines: http://localhost:4000/pipelines

  • Create Pipeline RUN: http://localhost:4000/pipelines/create

  • Track Pipeline Activity: http://localhost:4000/pipelines/activity/[RUN_ID]

    For advanced use cases and complex pipelines, please use the Raw JSON API to manually initiate a run using cURL or graphical API tool such as Postman. POST the following request to the DevLake API Endpoint.

    [
        [
            {
                "plugin": "github",
                "options": {
                    "repo": "lake",
                    "owner": "merico-dev"
                }
            }
        ]
    ]

    Please refer to Pipeline Advanced Mode for in-depth explanation.

  1. Click View Dashboards button in the top left when done, or visit localhost:3002 (username: admin, password: admin).

    We use Grafana as a visualization tool to build charts for the data stored in our database. Using SQL queries, we can add panels to build, save, and edit customized dashboards.

    All the details on provisioning and customizing a dashboard can be found in the Grafana Doc.

  2. (Optional) To run the tests:

    make test

  3. For DB migrations, please refer to Migration Doc.