Overview

The GitLab Integration project expanded the tool's capabilities beyond GitHub to include GitLab, responding to the demands of enterprise customers. One of our largest clients played a pivotal role in driving the expansion. This integration allowed for repository management, webhook events, and authentication, supporting our strategic move to serve more customers.

Project Highlights

• Enabled users to connect GitLab accounts.

• Provided repository access, event management, and secure webhook subscriptions for GitLab.

• Improved the architecture for supporting multiple Git providers, paving the way for future integrations.

Problem Statement

Initially, the product only supported GitHub integration, assuming it would be the most widely used platform for customers. However, significant demand arose from clients who used GitLab, making it crucial to expand integration support and enable broader adoption.

Solution Overview

The integration involved enabling GitLab users to connect their accounts, access repositories, and manage workflows through webhook events, similar to GitHub. The backend was refactored to ensure that supporting multiple git providers was efficient, and the user interface was updated to maintain simplicity across providers.

Flowchart

Table of contents

System Architecture

Backend

• Refactored the existing AccountService to introduce a common interface (GitAccountService). This made authentication and operations provider-agnostic, facilitating GitLab support and future scalability.

• Used Go programming language with the GitLab client library (xanzy/go-gitlab) for API interactions.

Frontend

• Built using React and TypeScript.

• Updated the onboarding UI to allow users to select either GitHub or GitLab during account setup.

• Provided a consistent experience in the Settings page for managing Git connections.

Deployment

• Dockerized the service for ease of deployment.

• Integrated with Kubernetes for container orchestration, ensuring high availability and scalability.

• PostgreSQL used for database management.

API Documentation

Endpoints

• Connect GitLab: POST /users/{identity-uuid}/accounts

• Fetch Repositories: GET /users/{identity-uuid}/accounts/{account-uuid}/repos

• Disconnect GitLab: DELETE /users/{identity-uuid}/accounts/{account-uuid}

Authentication

• Used personal access tokens for GitLab, as GitLab does not support an app-based integration similar to GitHub.

• Extended the authentication logic to allow tokens from both GitHub and GitLab.

Error Responses

• 401 Unauthorized: Invalid or expired token.

• 404 Not Found: Repository or account not found.

User Documentation

Dashboard

• Added a dropdown menu to select Git provider for creating new workflows and pipelines.

• Instructions for connecting/disconnecting GitLab or GitHub accounts from the settings page.

Troubleshooting

• GitLab Connection Issues: Guide to regenerating personal access tokens and reconnecting.

• Webhook Failures: Checklist for verifying webhook configuration and validating secrets.

Maintenance Guide

Database Integration

• Managed user account information in a relational database, with GitLab account details linked to user identity UUIDs.

Logging

• Implemented centralized logging for all GitLab integration activities using ELK Stack for monitoring and debugging.

Version Control

• Used Git for version control with branches dedicated to GitLab features.

Routine Maintenance Tasks

• Token Expiry Check: Routine validation of GitLab tokens to ensure ongoing API access.

• Webhook Review: Monthly review of active webhooks to remove obsolete entries.

Future Enhancements

• Bitbucket Integration: Extend support to Bitbucket to capture a broader segment of the market.

• OAuth Integration for GitLab: Replace personal access tokens with OAuth to improve security and usability.

Lessons Learned

• Refactoring for scalability early in the integration process made subsequent changes easier and prevented major rewrites.

• Maintaining UI consistency was crucial for a positive user experience when introducing a new git provider.

Impact

• Customer Adoption: The client onboarded successfully and began providing feedback, aiding product improvements.

• Increased Versatility: Supporting multiple git providers increased the product's attractiveness to potential enterprise customers.

Conclusion

The GitLab Integration project was a key initiative aimed at meeting enterprise customer demands and expanding the product's capabilities. By implementing a provider-agnostic backend and ensuring a seamless UI experience, the product is now better positioned to serve a wider range of customers and accommodate additional git providers in the future.