Changes between Initial Version and Version 1 of Developer/5aReleaseProcess


Ignore:
Timestamp:
Jul 15, 2020, 4:06:36 PM (15 months ago)
Author:
seskar
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Developer/5aReleaseProcess

    v1 v1  
     1= Release Process =
     2
     3This document describes the procedure for making a new OML release. It is intended for developers and maintainers of OML. It also applies to making releases of the [wiki:Applications OML-instrumented applications. ]
     4
     5We distinguish two types of releases: patch releases that fix bugs, and feature releases that implement new features. OML releases are numbered as <major>.<minor>.<patch>, for instance 2.3.4 means minor release 3, patch 4 for OML2. The minor version number increments by one for each feature release. Patch level releases only fix bugs and introduce no new features, and each release increments the patch number by one.
     6
     7The procedure for both types of OML releases is the same, except that the non-patch release involves creating a new release branch.
     8
     9== Test everything ==
     10This step before the release tag is pushed to the public repository. All of the available tests should pass. If they don't, fix the problems and re-test, then commit the changes. The testing is now done automatically using Jenkins which works with both the master and staging branches.
     11
     12{{{
     13$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
     14$ ./autogen.sh
     15$ ./configure
     16$ make CFLAGS="-Wall -Werror"
     17$ make distcheck
     18}}}
     19
     20This should also create a tarball named 'oml2-2.x.x.tar.gz', and run all the
     21installation/deinstallation/clean tests.
     22
     23Make sure that the install put the right things in the right places (/usr/local/bin, /usr/local/include, /usr/local/lib, /usr/local/share/man); tree(1) is a good helper here.
     24
     25{{{
     26$ make DESTDIR=$PWD/_inst install
     27$ tree _inst
     28}}}
     29
     30If any of the above steps fails, it most likely means there are missing files in the distribution tarball. You will need to go back and patch the Makefile.am's to include any missing files. Note that all source files including header files (.h) have to appear in a 'SOURCES' target for Automake to automatically include them in the distribution. For files that aren't found this way, an EXTRA_DIST Automake target might be required (see for instance the doc/Makefile.am on the master branch).
     31
     32If you do have to make changes at this point, make sure to test them and commit them to the repository.
     33
     34== Define environment ==
     35A lot of the following commands rely on the version number for the release to be. We declare it once and for all in the environment.
     36{{{
     37$ export REPO=origin \
     38  OML_MAJOR=2 \
     39  OML_MINOR=10 \
     40  OML_REV=0
     41$ export OML_VER=${OML_MAJOR}.${OML_MINOR}.${OML_REV} \
     42  VERSION=${OML_MAJOR}.${OML_MINOR} # Useful for branches
     43}}}
     44
     45== Update ChangeLog ==
     46
     47The ChangeLog should be updated to record the significant changes in this release such as what bugs it fixed (for a bug fix release), what features it introduces (for a major feature release), etc.
     48
     49A good startup point is to use the Git log. It probably will need to be trimmed to the most relevant changes, and certainly reformatted.
     50{{{
     51$ (echo "`date +%Y-%m-%d`  `git config --get user.name` <`git config --get user.email`>"; \
     52   git log v2.$((OML_MINOR - 1)).0..HEAD  --pretty="^v<Tab>* %s"; \
     53   echo; \
     54   cat ChangeLog) > ChangeLog.new; mv ChangeLog.new ChangeLog # "^v<Tab>" means Ctrl+V then Tab, to insert a literal tabulation character
     55$ vim ChangeLog
     56[trim and reformat]
     57$ git add ChangeLog
     58$ git commit -sm "Update ChangeLog for ${OML_VER}"
     59}}}
     60
     61== Tag the release ==
     62All releases must be made from a tag. This is enforced for Debian, RPM and Arch packages (see below). The tag should be called "v<major>.<minor>.<patch>", for instance, "v2.3.4". For the first release of a new feature release, the <patch> number should be 0. So, for instance, it might be 2.4.0.
     63
     64Here's how to create the tag:
     65{{{
     66$ git tag -a v${OML_VER} -m "OML v${OML_VER}"  # It's also a good idea to sign the tag: -su PGPID
     67}}}
     68
     69The tag must be annotated and the tag message should follow the format shown. The commit message should also indicate the update of the version number. It is important to make sure that the tag is on the right commit, which is why the testing steps come before this one. It is possible to re-tag, but it is not preferred. MAKE SURE that you don't push the tag upstream before you are confident that this commit is definitely the one you are going to release from.
     70
     71For the applications, Iperf will need to be tagged separately.
     72{{{
     73$ git tag -a v2.0.5+oml${OML_VER} -m "Iperf instrumentation for OML ${OML_VER}" # It's also a good idea to sign the tag: -su PGPID
     74}}}
     75
     76== Make a new release branch (new feature release only) ==
     77
     78Releases are always made from a release branch. Release branches should be called 'release/<major>.<minor>', for example 'release/2.3', 'release/2.4'. A new feature release should normally be made from the tip of the 'master' branch:
     79{{{
     80$ git checkout master
     81$ git checkout -b release/${VERSION}
     82}}}
     83
     84This would create a new release branch for the v2.9.x release series. All subsequent changes made on the 'release/${VERSION}' branch should be bug fixes only. No new features should be implemented on a release branch.
     85
     86== Create a release tarball ==
     87{{{
     88$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
     89$ ./autogen.sh
     90$ ./configure
     91$ make distcheck DISTCHECK_CONFIGURE_FLAGS=--with-pgsql-inc=/usr/include/postgresql # Argument needed when building on Debian, to get the
     92}}}
     93
     94== Upload the release tarball ==
     95Release tarballs are stored on the Files tab (or here for the applications). Generally only the most recent tarball in a minor release series needs to be kept around, so you can delete the previous patch release if any
     96
     97== Update online documentation ==
     98Documentation for the dev version is pulled once a day from the Jenkins oml-doc artifacts by a cron job (omf-mytestbed:/etc/cron.daily/oml-doc).
     99
     100However, documentation for releases needs more curation. They are installed on omf-mytestbed in /var/www/redmine_mytestbed.net/public/doc/oml/MAJOR.MINOR/. From a tree of built sources, with the documentation properly generated, the files can be pushed as follows:
     101
     102{{{
     103$ ./autogen.sh
     104$ ./configure
     105$ make doc-publish
     106$ ssh ${USER}@mytestbed.net ln -sf ${VERSION} /var/www/redmine_mytestbed.net/public/doc/oml/latest
     107}}}
     108
     109== Build packages ==
     110Debian, RPM and ArchLinux packages can be built straight from the Git clone,
     111provided you have the relevant packaging tools set up and configured correctly.
     112
     113In a nutshell, all possible packages are built as follows at any time.
     114{{{
     115$ ./autogen.sh
     116$ ./configure --enable-packaging
     117$ make pkg-all
     118}}}
     119
     120However, if they are releases, some more things must be adjusted first.
     121
     122=== Debian ===
     123
     124The main release channel for OML is the Debian packages.
     125
     126Debian packaging rules are taken from the 'debian/*' branches, depending on the current version number of the tree (based on git describe), and defaulting to 'debian/master'.
     127
     128If this is a new feature release, a topic branch for the packaging rules must be created. For example:
     129{{{
     130$ git checkout -b debian/release/${VERSION} debian/master
     131}}}
     132
     133If you look at the contents of this branch and its history, you will see that it shares no history with the main source tree; they are different root commits. The 'debian/xyz' branches contain only the 'debian' directory, which contains all of the configuration required to build the packages. Here is how to build the packages: first, update the debian/changelog using dch:
     134
     135{{{
     136$ dch -v ${OML_VER}-mytestbed1 # RC and PRE packages should be versioned as 2.x.x~[rc|pre]; using the tilde for proper ordering
     137}}}
     138
     139Put a brief description of the package after the asterisk. Normally a simple "New upstream release" is sufficient:
     140{{{
     141oml (2.9.0-mytestbed1) karmic; urgency=low
     142
     143  * New upstream release 2.9.0.
     144}}}
     145
     146Save the changelog, quit your editor. Commit and tag the release branch. Checkout the
     147release branch, then build the package from there.
     148{{{
     149$ git commit -as -m "OML Debian package v${OML_VER}-mytestbed1" # Be careful when using commit -a that you *really* want all unstaged changes in
     150$ git tag -a debian/v${OML_VER}-mytestbed1 -m "OML Debian package v${OML_VER}-mytestbed1" # It's also a good idea to sign the tag: -su PGPID
     151$ git checkout release/${VERSION}
     152$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
     153$ ./autogen.sh
     154$ ./configure --enable-packaging
     155$ make pkg-deb # On a Debian system only, to properly set dependences on shared libraries!
     156}}}
     157
     158Lots of log messages should fly past, and then you should end up with the .deb files in the p-debian directory. There may be couple of lintian warnings, but they are basically harmless and can be ignored.
     159
     160=== RedHat and other RPM-based distros ===
     161The RedHat-ish packaging system is conceptually similar to Debian's. Package-building rules are located in the 'rpm/*' branches.
     162
     163$ git checkout -b rpm/release/${VERSION} rpm/master
     164$ vim SPECS/oml2.spec # or oml2-apps.spec
     165[ update specfile,
     166change the redmineid and version (NOTE: RC and PRE packages should be versioned as 2.x.x~[rc|pre]; using the tilde for proper ordering) variables at the top,
     167be mindful of the Release counter too]
     168{{{
     169$ git commit -as -m "OML RPM package v${OML_VER}-1"
     170$ git tag -a rpm/v${OML_VER}-1 -m "OML RPM package v${OML_VER}-1" # It's also a good idea to sign the tag: -su PGPID
     171$ git checkout release/${VERSION}
     172$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
     173$ ./autogen.sh
     174$ ./configure --enable-packaging
     175$ make pkg-rpm # On an RPM system only, to properly set dependences on shared libraries for binary packages!
     176}}}
     177
     178Once this completes, packages will be found in the p-rpm directory.
     179
     180For old Fedora releases that OBS no longer support (but that platform such as PlanetLab still use), packages can be built and published locally using the build_old_fedora_pkg.sh script (see #1354). The syntax for the script is as follows from a directory containing both of an oml and oml-apps Git clones.
     181{{{
     182./build_old_fedora_pkg.sh [OML_TAG [OMLAPPS_TAG=OML_TAG]]
     183}}}
     184
     185If unspecified, OMLAPPS_TAG will default to OML_TAG, which itself will default to whatever is hardcoded in the script (i.e., don't rely on the default behaviour, and specify at least the OML_TAG).
     186
     187=== ArchLinux ===
     188The Arch packaging of OML follows a similar logic, with a slightly different approach, as Arch packaging depends on upstream source archives.
     189
     190{{{
     191$ git checkout -b archlinux/release/${VERSION} archlinux/master
     192$ vim PKGBUILD.proto
     193[update PKGBUILD.proto, update _redmineid, pkgver and pkgrel]
     194$ git commit -as -m "OML Arch package v${OML_VER}-1"
     195$ git tag -a archlinux/v${OML_VER}-1 -m "OML Arch package v${OML_VER}-1" # It's also a good idea to sign the tag: -su PGPID
     196$ git checkout release/${VERSION}
     197$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
     198$ ./autogen.sh
     199$ ./configure --enable-packaging
     200$ make pkg-arch
     201}}}
     202Packages (binary and source) will be located in directory p-arch.
     203
     204== Test the packages ==
     205Once the packages are built, check that they install correctly over the old versions using dpkg -i, and that they install correctly on a system where there are no older versions installed. Make sure that necessary dependencies are correctly installed.
     206
     207== Upload distribution packages ==
     208Since OML compiles platform-specific binaries, you may want to repeat these steps on a machine with a different CPU architecture (e.g. i386, amd64) in order to provide packages for it.
     209
     210=== Debian and RPM ===
     211The easiest is to make source files oml2_${OML_VER}-mytestbed1.diff.gz, oml2_${OML_VER}-mytestbed1.dsc, oml2_${OML_VER}.orig.tar.gz and oml2.spec, generated in p-debian/ and p-rpm by make pkg-all (they can be obtained from Jenkins for OML and the apps), available to OpenSuSE Build System (here for OML). Usually, this involves creating a tarball containing these files, and uploading them to OBS. The "special tarball" obs.tar.gz from Jenkins (oml, oml-apps) can be uploaded for oml or oml-apps.
     212
     213{{{
     214$ mkdir obs; \
     215  cd obs; \
     216  cp ../p-debian/oml2_${OML_VER}{-mytestbed1.diff.gz,-mytestbed1.dsc,.orig.tar.gz} .; \
     217  find ../p-rpm -type f -exec cp {} $PWD \;
     218  tar czvhf obs.tar.gz *; \
     219  cd ..
     220}}}
     221
     222The applications also need the collectd source to be uploaded to their OBS project alongside with the rest (or be shipped as part of the obs.tar.gz file, which is what Jenkins does).
     223
     224{{{
     225$ mkdir obs; \
     226  cd obs; \
     227  cp ../p-debian/oml2-apps_${OML_VER}{-mytestbed1.diff.gz,-mytestbed1.dsc,.orig.tar.gz} .; \
     228  bunzip2 -c /tmp/collectd-4*.tar.bz2 | gzip > collectd.tar.gz; \
     229  find ../p-rpm -type f -exec cp {} $PWD \;
     230  tar czvhf obs.tar.gz *; \
     231  cd ..
     232}}}
     233
     234The build process should start automatically once the special tarball is uploaded.
     235
     236Packages should also be built for Fedora 14 (PLE) and Fedora 15 (InstaGENI). Christoph provides a script to be run on norbit, which logs on to two KVMs, fetches and compiles OML, OML apps and libtrace, creates RPM packages and uploads them to mytestbed.net.
     237
     238=== ArchLinux ===
     239Submit the Arch package source file oml2-${OML_VER}-x.src.tar.gz, built at the same time as the binary package, to AUR (category lib).
     240
     241== Push to the upstream repository ==
     242Any commits and tags created on the release branch and pakcaging branches should be pushed to the main mytestbed.net repository (not GitHub, this will be synced automatically).
     243
     244{{{
     245$ git push ${REPO} release/${VERSION} \
     246  archlinux/master archlinux/release/${VERSION} \
     247  debian/master debian/release/${VERSION} \
     248  rpm/master rpm/release/${VERSION} \
     249  tag v${OML_VER} \
     250  tag archlinux/v${OML_VER}-1 \
     251  tag debian/v${OML_VER}-mytestbed1 \
     252  tag rpm/v${OML_VER}-1 \
     253  master
     254}}}
     255
     256== Merge bug fixes to the 'master' branch ==
     257Any patch level release, which fixes bugs, should be merged back to the 'master' branch, so that the bugs are fixed there as well:
     258{{{
     259$ git checkout master
     260$ git merge release/${VERSION}
     261}}}
     262There will usually be a conflict in configure.ac due to the different version numbers, but it should be easy to resolve.
     263
     264== Prepare the 'master' branches for new work ==
     265The Autoconf package version on the master branch should always be set to "2.<next>.pre0", where "2.<next>.0" would be the next planned feature release (yet to occur), and the "pre0" makes it glaringly obvious that the binary was built from the development branch, and is not an official release binary. For instance, if the most recent feature release was 2.4.x, then the master branch should report version 2.5.pre0. Once this change is commited, it should be tagged with "v2.<next>.pre", which will allow git-version-gen to properly generate development version number.
     266{{{
     267$ export NEXT_VERSION=${OML_MAJOR}.$((OML_MINOR+1)).0
     268$ git checkout master
     269$ git tag -a v${NEXT_VERSION}pre -m "OML v${NEXT_VERSION} development version" # On the first commit on master not on release/${VERSION}; as usual, it's also a good idea to sign the tag: -su PGPID
     270$ git push master
     271$ git push --tags
     272$ git checkout archlinux/master
     273$ git merge archlinux/release/${VERSION}
     274$ git checkout debian/master
     275$ git merge debian/release/${VERSION}
     276$ git checkout rpm/master
     277$ git merge rpm/release/${VERSION}
     278}}}
     279
     280== Tell the world ==
     281
     282 1. Finalize the release notes for that version
     283 1. Add a news item in the OML news
     284 1. Send an email to the oml-user mailing list (containing the ChangeLog and a link to the source)