Commit 68e3b47c authored by Mario Lassnig's avatar Mario Lassnig Committed by Gerrit Code Review
Browse files

[RUCIO-1573] gitlab: branching scripts ; renamed hotfix to patch ; added docs ; hotfix

Change-Id: Ifcd824b117232f377c0fb8dbfdd75ba0038e1d73
parent 3115c241
......@@ -2,90 +2,74 @@
Developing Rucio
================
--------------------------------------
Code Review and Submitting a patch set
--------------------------------------
----------------------------------
Contributing patches & code review
----------------------------------
The code review tool is Gerrit and can be found at::
Rucio follows a next/master development scheme: two protected branches called "next" and "master" track all features and patches. Contributors create their own private development branches to do their work, and once finished and code reviewed, these branches are merged into next and/or master.
https://atlas-gerrit.cern.ch:8443/
The hosting is at GitLab, and the upstream is at:
In principle, you should always work on a local feature branch until a feature is complete, and then submit the feature as a patch set for review::
https://gitlab.cern.ch/rucio01/rucio
$ git checkout -b new_feature # create new local branch and switch to it
$ git branch -a # list all branches
$ git checkout master # switch to master branch
$ git branch -d new_feature # delete the local feature branch
-----------
First setup
-----------
Never push a feature branch directly to origin, it is your local development environment! Origin only keeps the code reviewed master.
To submit a new patch set for review::
Go to the GitLab web-interface mentioned above, and login with your CERN account. It will give you the option to fork the rucio01/rucio repository into your private account (upper left corner). Do this.
$ git commit -m "new feature added"
$ tools/submit-review
Afterwards, switch to your private account and clone it, for example:
Assuming that the review was not okay, and you have to make some changes, DO NOT COMMIT AGAIN as this will create a new review request! Instead amend the current bad patch set with::
git clone https://gitlab.cern.ch/<cern_username>/rucio
$ git checkout new_feature # make sure we're on the right branch
$ emacs # as needed
$ git add # as needed
$ git rm # as needed
$ tools/submit-review -a "now it is fixed"
This will be your private copy of Rucio that has no knowledge of upstream. So first, add upstream as a remote:
In case you need to fix an older commit, stash away your current changes, rebase to the old commit, fix the code, amend for review, re-stash your original changes::
git remote add upstream https://gitlab.cern.ch/rucio01/rucio
$ git stash # make sure we don't lose our current changes
$ git rebase -i HEAD~5 # go back 5 commits interactively
$ emacs # as needed
$ git add # as needed
$ git rm # as needed
$ tools/submit-review -a "finally it is fixed" # amend the change
$ git apply # get our changes back
Verify that everything is alright with:
Of course, this is potentionally dangerous if someone has already changed files from any of these commits and pushed them to the official master, so some synchronisation with colleagues might be needed.
git remote -v
If the patch set was reviewed and approved, don't forget to fetch the repository metadata, and, optionally, pull the changes from the origin master again::
You should see both push/pull remotes for origin (your private account) and upstream (official rucio repository).
$ git fetch
$ git pull
--------------------
Developing a feature
--------------------
Should you get confused in any way, don't forget that you can always clone the official master branch afresh, pull the necessary commits, and copy the new files over.
Features are scheduled for the next Rucio release and are collected from the protected "next" branch. Create a new feature branch with:
TL;DR If something is weird, ask Mario.
tools/create-feature-branch <ticketnumber> <branch description>
--------------------------------------------------------------------
I have a conflict in my patch set and I need to merge. What do I do?
--------------------------------------------------------------------
and do your development there. When done, push the branch into origin (your private account) for code review:
If you get an error message from gerrit like "Please merge (or rebase) the change locally and upload the resolution for review.", then that means that someone got a change approved for a file while you were working on the same file. This means that you need to fix your commit:
git push origin
1. Make sure you're on your master branch::
------------------
Developing a patch
------------------
git checkout master
A patch works exactly the same, but is branched off the "master". Create a new patch branch with
2. Get the newest changesets from origin/master::
tools/create-patch-branch <ticketnumber> <branch description>
git fetch; git pull
and do your development there. When done, push the branch into origin (your private account) for code review:
3. Switch to your feature branch and merge in the changes::
git push origin
git checkout my_feature
git rebase master
-------------------------------
Code review and merging a patch
-------------------------------
4. This will break at some point at the problematic file(s). Edit them and mark them as resolved::
Two rules must be obeyed:
emacs file1
emacs file2
git add file1
git add file2
1. Feature branches must be merged into "next"
2. Patch branches must be merged into "master" and "next"
5. Finish the merge::
(For now, step 2 is manual, we will automate it in the future.) Click the "Merge request" button in the web-interface and select the (potentially two) appropriate destination branches, e.g., from youraccount/featurebranch to rucio/next. Don't forget that patch branches need two merge requests, both into "next" and "master". (In the future, this will be automated. It is also possible to do this via CLI only, no web interface is actually needed.)
git rebase --continue
The merge request will enable the code review. After successful code review, the responsible can merge the patch on the web interface.
6. Submit for review::
tools/submit-review -a "merged conflicts"
TL;DR: If something is weird, ask for help on rucio-dev@cern.ch :-D
----------------
Ticketing system
......@@ -95,7 +79,7 @@ For Rucio we are using Jira to manage the development of the project:
https://its.cern.ch/jira/browse/RUCIO
Tickets for new features should be submitted with a functional granularity, that is according to the API call being introduced and at which level it belongs. For example, "register_dataset API (CORE)", "register_dataset (REST)", and "register_dataset API (CLIENT)", instead of big and vague new feature definitions like "new dataset functionality". This level of granularity allows better tracking of the progress of the RUCIO project, informs developers when new interfaces become available, and leads to more meaningful changelogs when a release is made.
Tickets for new features should be submitted with a functional granularity, that is according to the API call being introduced and at which level it belongs. For example, "register_dataset API (CORE)", "register_dataset (REST)", and "register_dataset API (CLIENT)", instead of big and vague new feature definitions like "new dataset functionality". This level of granularity allows better tracking of the progress of the RUCIO project, informs developers when new interfaces become available, and leads to more meaningful changelogs when a release is made.
In order to avoid generating too many tickets and insuring the documentation of relevant work is placed in a single description, all minor schema changes and corresponding test cases should be included as part of the new feature ticket and seperate tickets should not be made. The exception to this is if additional functionality, a bug fix or a new test case is added to the task in a newer release of Rucio, this then should be documented as a new ticket, rather than modifying the existing ticket (as it is assigned to the previous Rucio release).
......@@ -103,8 +87,8 @@ The ticket workflow in Jira is summarised here:
https://confluence.atlassian.com/download/attachments/284367573/system-workflow.png
When one is finished working on a new feature or bug fix and this has been commited and submitted to Gerrit for approval, the ticket status should be changed to 'resolved'. Once the new code has been approved and commited to the GIT master the ticket status should be changed to 'closed'.
When one is finished working on a new feature or bug fix and this has been commited and submitted to Code Review for approval, the ticket status should be changed to 'resolved'. Once the new code has been approved and commited to the GIT master the ticket status should be changed to 'closed'.
GIT commits should include the relevant JIRA ticket number(s) in the beginning of the commit message. This is because Jira is integrated with GIT and will associate the tickets to the corresponding GIT commits.
GIT commits should include the relevant JIRA ticket number(s) in the beginning of the commit message. This is because Jira is integrated with GIT and will associate the tickets to the corresponding GIT commits.
Jira ticket headers and descriptions will be included on release changelogs. For this reason the titles and descriptions should be meaningful.
......@@ -83,11 +83,4 @@ You can selectively run test cases by giving directories or files as parameters
Executing unit tests the correct way
------------------------------------
1. Open two terminals A and B, and go to ``/opt/rucio``
2. Use terminal A to reset the database and restart Apache
``sudo rm -rf /tmp/rucio.db; python tools/reset_database.py; chmod 777 /tmp/rucio.db; sudo apachectl restart; tail -f /var/log/apache2/*_log /var/log/rucio/httpd_*``
3. Use terminal B to clean the development and client environment and execute the unit tests
``rm -rf /tmp/.rucio_root/; find lib -iname *.pyc | xargs rm; nosetests -d -v --logging-filter=-sqlalchemy,-migrate,-rucio.client.baseclient; nosetests -d -v --logging-filter=-sqlalchemy,-migrate,-rucio.client.baseclient``
1. Run ``tools/run_tests -1qa``
#!/bin/bash
if [[ $# -ne 2 ]] ; then
echo 'usage: tools/create-feature-branch <ticketnumber> <branchname>'
echo
echo 'examples: tools/create-feature-branch 1234 fancyfeature'
echo ' tools/create-feature-branch 1234 "my fancy feature"'
exit 1
fi
echo "Switching to next"
git checkout next
echo "Updating next"
git pull --all --prune --progress
echo "Creating feature branch"
git checkout -b feature-$1-${2//[^a-zA-Z0-9]/_} next
echo "Done"
#!/bin/bash
if [[ $# -ne 2 ]] ; then
echo 'usage: tools/create-patch-branch <ticketnumber> <branchname>'
echo
echo 'examples: tools/create-patch-branch 1234 fancypatch'
echo ' tools/create-patch-branch 1234 "my fancy patch"'
exit 1
fi
echo "Switching to master"
git checkout master
echo "Updating master"
git pull --all --prune --progress
echo "Creating patch branch"
git checkout -b patch-$1-${2//[^a-zA-Z0-9]/_} master
echo "Done"
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment