Managing downstream packages

For a few internal applications, we manage downstream forks with some customizations tailored to our needs. We try to contribute as many changes to upstream as possible, but for some cases, this does not make sense, e.g. when we customize user synchronization processes.

First, we create and clone an internal repository. After that, we start cloning the upstream repository and push the master or main branch into our repository:

git clone git@internal.loc:bitExpert/application.git
git remote add upstream git@remote:vendor/application.git
git fetch upstream
git pull upstream main
git push origin main

Next, we push the latest release tag into our repository as well. This will act as the starting point for any feature branch that we will create. Let's assume the latest tag is 3.5.0:

git push origin 3.5.0

With this tag, we start creating a release branch which at a later point will contain all changes that we made and will be used to deploy the application in our infrastructure:

git checkout 3.5.0
git checkout -b release/r3.5.0
git push origin release/r3.5.0

For every release branch, we do it in the same way. Let's say we want to add the Nomad deployment configuration:

git checkout 3.5.0
git checkout -b feature/nomad
git push origin feature/nomad

Adding the feature to the respective release branch is done by merging the feature/nomad into the latest release branch release/r3.5.0:

git checkout release/r3.5.0
git merge feature/nomad

Since our release strategy is based on Git tags, we need to tag the latest version and push the tag to our GitLab instance to kick off the deployment pipeline:

git tag 3.5.0.1
git push origin 3.5.0.1

Since we have to keep the release tags from upstream and our "local" tags in the GitLab repository, and since we want to keep the original application version number visible, we decided to introduce a deployment count in the version tag which is the last part of the version number, .1 in this case. Since we deploy docker images, this helps to keep track of the most current images and it is relatively easy to clean up older images based on the version information.

What happens when version 3.6.0 of the upstream application is released?

First, we update the master or main branch:

git fetch upstream
git pull upstream main
git push origin main

We fetch the latest tag of the upstream repository and push the tag into our Git repository:

git push origin 3.6.0

We create a new release branch out of the tag and push it also:

git checkout 3.6.0
git checkout -b release/r3.6.0
git push origin release/r3.6.0

Next, we need to update the feature branches. For this, we don't merge the master or main branch into the feature branch but we rebase the feature branch. Rebasing means, applying our local changes on of the branch we rebase from. If we would merge the master or main branch it would mix our history with the upstream history. This will make it harder to see our own changes and it could result in some more merge conflicts:

git checkout feature/nomad
git rebase main

Since rebasing rewrites the Git history, we need to force push the changes to our main Git repository.

git push -f

As a final step we merge the features branches in the new release branch, tag, and push it to kick off the deployment pipeline:

git checkout release/r3.6.0
git merge feature/nomad
git push
git tag 3.6.0.1
git push origin 3.6.0.1