Changes between Initial Version and Version 1 of Developer/5aReleaseProcess

Timestamp:

Jul 15, 2020, 4:06:36 PM (4 years ago)

Author:

seskar

Comment:

--

Developer/5aReleaseProcess

@@ +1 @@
+= Release Process =
+
+This document describes the procedure for making a new OML release. It is intended for developers and maintainers of OML. It also applies to making releases of the [wiki:Applications OML-instrumented applications. ]
+
+We distinguish two types of releases: patch releases that fix bugs, and feature releases that implement new features. OML releases are numbered as <major>.<minor>.<patch>, for instance 2.3.4 means minor release 3, patch 4 for OML2. The minor version number increments by one for each feature release. Patch level releases only fix bugs and introduce no new features, and each release increments the patch number by one.
+
+The procedure for both types of OML releases is the same, except that the non-patch release involves creating a new release branch.
+
+== Test everything ==
+This step before the release tag is pushed to the public repository. All of the available tests should pass. If they don't, fix the problems and re-test, then commit the changes. The testing is now done automatically using Jenkins which works with both the master and staging branches.
+
+{{{
+$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
+$ ./autogen.sh
+$ ./configure
+$ make CFLAGS="-Wall -Werror" 
+$ make distcheck
+}}}
+
+This should also create a tarball named 'oml2-2.x.x.tar.gz', and run all the
+installation/deinstallation/clean tests.
+
+Make sure that the install put the right things in the right places (/usr/local/bin, /usr/local/include, /usr/local/lib, /usr/local/share/man); tree(1) is a good helper here.
+
+{{{
+$ make DESTDIR=$PWD/_inst install
+$ tree _inst
+}}}
+
+If any of the above steps fails, it most likely means there are missing files in the distribution tarball. You will need to go back and patch the Makefile.am's to include any missing files. Note that all source files including header files (.h) have to appear in a 'SOURCES' target for Automake to automatically include them in the distribution. For files that aren't found this way, an EXTRA_DIST Automake target might be required (see for instance the doc/Makefile.am on the master branch).
+
+If you do have to make changes at this point, make sure to test them and commit them to the repository.
+
+== Define environment ==
+A lot of the following commands rely on the version number for the release to be. We declare it once and for all in the environment.
+{{{
+$ export REPO=origin \
+  OML_MAJOR=2 \
+  OML_MINOR=10 \
+  OML_REV=0
+$ export OML_VER=${OML_MAJOR}.${OML_MINOR}.${OML_REV} \
+  VERSION=${OML_MAJOR}.${OML_MINOR} # Useful for branches
+}}}
+
+== Update ChangeLog ==
+
+The ChangeLog should be updated to record the significant changes in this release such as what bugs it fixed (for a bug fix release), what features it introduces (for a major feature release), etc.
+
+A good startup point is to use the Git log. It probably will need to be trimmed to the most relevant changes, and certainly reformatted.
+{{{
+$ (echo "`date +%Y-%m-%d`  `git config --get user.name` <`git config --get user.email`>"; \
+   git log v2.$((OML_MINOR - 1)).0..HEAD  --pretty="^v<Tab>* %s"; \
+   echo; \
+   cat ChangeLog) > ChangeLog.new; mv ChangeLog.new ChangeLog # "^v<Tab>" means Ctrl+V then Tab, to insert a literal tabulation character
+$ vim ChangeLog
+[trim and reformat]
+$ git add ChangeLog
+$ git commit -sm "Update ChangeLog for ${OML_VER}" 
+}}}
+
+== Tag the release ==
+All releases must be made from a tag. This is enforced for Debian, RPM and Arch packages (see below). The tag should be called "v<major>.<minor>.<patch>", for instance, "v2.3.4". For the first release of a new feature release, the <patch> number should be 0. So, for instance, it might be 2.4.0.
+
+Here's how to create the tag:
+{{{
+$ git tag -a v${OML_VER} -m "OML v${OML_VER}"  # It's also a good idea to sign the tag: -su PGPID
+}}}
+
+The tag must be annotated and the tag message should follow the format shown. The commit message should also indicate the update of the version number. It is important to make sure that the tag is on the right commit, which is why the testing steps come before this one. It is possible to re-tag, but it is not preferred. MAKE SURE that you don't push the tag upstream before you are confident that this commit is definitely the one you are going to release from.
+
+For the applications, Iperf will need to be tagged separately.
+{{{
+$ git tag -a v2.0.5+oml${OML_VER} -m "Iperf instrumentation for OML ${OML_VER}" # It's also a good idea to sign the tag: -su PGPID
+}}}
+
+== Make a new release branch (new feature release only) ==
+
+Releases are always made from a release branch. Release branches should be called 'release/<major>.<minor>', for example 'release/2.3', 'release/2.4'. A new feature release should normally be made from the tip of the 'master' branch:
+{{{
+$ git checkout master
+$ git checkout -b release/${VERSION}
+}}}
+
+This would create a new release branch for the v2.9.x release series. All subsequent changes made on the 'release/${VERSION}' branch should be bug fixes only. No new features should be implemented on a release branch.
+
+== Create a release tarball ==
+{{{
+$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
+$ ./autogen.sh
+$ ./configure
+$ make distcheck DISTCHECK_CONFIGURE_FLAGS=--with-pgsql-inc=/usr/include/postgresql # Argument needed when building on Debian, to get the 
+}}}
+
+== Upload the release tarball ==
+Release tarballs are stored on the Files tab (or here for the applications). Generally only the most recent tarball in a minor release series needs to be kept around, so you can delete the previous patch release if any
+
+== Update online documentation == 
+Documentation for the dev version is pulled once a day from the Jenkins oml-doc artifacts by a cron job (omf-mytestbed:/etc/cron.daily/oml-doc).
+
+However, documentation for releases needs more curation. They are installed on omf-mytestbed in /var/www/redmine_mytestbed.net/public/doc/oml/MAJOR.MINOR/. From a tree of built sources, with the documentation properly generated, the files can be pushed as follows:
+
+{{{
+$ ./autogen.sh
+$ ./configure
+$ make doc-publish
+$ ssh ${USER}@mytestbed.net ln -sf ${VERSION} /var/www/redmine_mytestbed.net/public/doc/oml/latest
+}}}
+
+== Build packages ==
+Debian, RPM and ArchLinux packages can be built straight from the Git clone,
+provided you have the relevant packaging tools set up and configured correctly.
+
+In a nutshell, all possible packages are built as follows at any time.
+{{{
+$ ./autogen.sh
+$ ./configure --enable-packaging
+$ make pkg-all
+}}}
+
+However, if they are releases, some more things must be adjusted first.
+
+=== Debian ===
+
+The main release channel for OML is the Debian packages.
+
+Debian packaging rules are taken from the 'debian/*' branches, depending on the current version number of the tree (based on git describe), and defaulting to 'debian/master'.
+
+If this is a new feature release, a topic branch for the packaging rules must be created. For example:
+{{{
+$ git checkout -b debian/release/${VERSION} debian/master
+}}}
+
+If you look at the contents of this branch and its history, you will see that it shares no history with the main source tree; they are different root commits. The 'debian/xyz' branches contain only the 'debian' directory, which contains all of the configuration required to build the packages. Here is how to build the packages: first, update the debian/changelog using dch:
+
+{{{
+$ dch -v ${OML_VER}-mytestbed1 # RC and PRE packages should be versioned as 2.x.x~[rc|pre]; using the tilde for proper ordering
+}}}
+
+Put a brief description of the package after the asterisk. Normally a simple "New upstream release" is sufficient:
+{{{
+oml (2.9.0-mytestbed1) karmic; urgency=low
+
+  * New upstream release 2.9.0.
+}}}
+
+Save the changelog, quit your editor. Commit and tag the release branch. Checkout the
+release branch, then build the package from there.
+{{{
+$ git commit -as -m "OML Debian package v${OML_VER}-mytestbed1" # Be careful when using commit -a that you *really* want all unstaged changes in
+$ git tag -a debian/v${OML_VER}-mytestbed1 -m "OML Debian package v${OML_VER}-mytestbed1" # It's also a good idea to sign the tag: -su PGPID
+$ git checkout release/${VERSION}
+$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
+$ ./autogen.sh
+$ ./configure --enable-packaging
+$ make pkg-deb # On a Debian system only, to properly set dependences on shared libraries!
+}}}
+
+Lots of log messages should fly past, and then you should end up with the .deb files in the p-debian directory. There may be couple of lintian warnings, but they are basically harmless and can be ignored.
+
+=== RedHat and other RPM-based distros ===
+The RedHat-ish packaging system is conceptually similar to Debian's. Package-building rules are located in the 'rpm/*' branches.
+
+$ git checkout -b rpm/release/${VERSION} rpm/master
+$ vim SPECS/oml2.spec # or oml2-apps.spec
+[ update specfile,
+change the redmineid and version (NOTE: RC and PRE packages should be versioned as 2.x.x~[rc|pre]; using the tilde for proper ordering) variables at the top,
+be mindful of the Release counter too] 
+{{{
+$ git commit -as -m "OML RPM package v${OML_VER}-1" 
+$ git tag -a rpm/v${OML_VER}-1 -m "OML RPM package v${OML_VER}-1" # It's also a good idea to sign the tag: -su PGPID
+$ git checkout release/${VERSION}
+$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
+$ ./autogen.sh
+$ ./configure --enable-packaging
+$ make pkg-rpm # On an RPM system only, to properly set dependences on shared libraries for binary packages!
+}}}
+
+Once this completes, packages will be found in the p-rpm directory.
+
+For old Fedora releases that OBS no longer support (but that platform such as PlanetLab still use), packages can be built and published locally using the build_old_fedora_pkg.sh script (see #1354). The syntax for the script is as follows from a directory containing both of an oml and oml-apps Git clones.
+{{{
+./build_old_fedora_pkg.sh [OML_TAG [OMLAPPS_TAG=OML_TAG]]
+}}}
+
+If unspecified, OMLAPPS_TAG will default to OML_TAG, which itself will default to whatever is hardcoded in the script (i.e., don't rely on the default behaviour, and specify at least the OML_TAG).
+
+=== ArchLinux ===
+The Arch packaging of OML follows a similar logic, with a slightly different approach, as Arch packaging depends on upstream source archives.
+
+{{{
+$ git checkout -b archlinux/release/${VERSION} archlinux/master
+$ vim PKGBUILD.proto
+[update PKGBUILD.proto, update _redmineid, pkgver and pkgrel]
+$ git commit -as -m "OML Arch package v${OML_VER}-1" 
+$ git tag -a archlinux/v${OML_VER}-1 -m "OML Arch package v${OML_VER}-1" # It's also a good idea to sign the tag: -su PGPID
+$ git checkout release/${VERSION}
+$ git clean -fdx # Don't forget to do this in iperf/ too, for the applications
+$ ./autogen.sh
+$ ./configure --enable-packaging
+$ make pkg-arch
+}}}
+Packages (binary and source) will be located in directory p-arch.
+
+== Test the packages ==
+Once the packages are built, check that they install correctly over the old versions using dpkg -i, and that they install correctly on a system where there are no older versions installed. Make sure that necessary dependencies are correctly installed.
+
+== Upload distribution packages ==
+Since OML compiles platform-specific binaries, you may want to repeat these steps on a machine with a different CPU architecture (e.g. i386, amd64) in order to provide packages for it.
+
+=== Debian and RPM ===
+The easiest is to make source files oml2_${OML_VER}-mytestbed1.diff.gz, oml2_${OML_VER}-mytestbed1.dsc, oml2_${OML_VER}.orig.tar.gz and oml2.spec, generated in p-debian/ and p-rpm by make pkg-all (they can be obtained from Jenkins for OML and the apps), available to OpenSuSE Build System (here for OML). Usually, this involves creating a tarball containing these files, and uploading them to OBS. The "special tarball" obs.tar.gz from Jenkins (oml, oml-apps) can be uploaded for oml or oml-apps.
+
+{{{
+$ mkdir obs; \
+  cd obs; \
+  cp ../p-debian/oml2_${OML_VER}{-mytestbed1.diff.gz,-mytestbed1.dsc,.orig.tar.gz} .; \
+  find ../p-rpm -type f -exec cp {} $PWD \;
+  tar czvhf obs.tar.gz *; \
+  cd ..
+}}}
+
+The applications also need the collectd source to be uploaded to their OBS project alongside with the rest (or be shipped as part of the obs.tar.gz file, which is what Jenkins does).
+
+{{{
+$ mkdir obs; \
+  cd obs; \
+  cp ../p-debian/oml2-apps_${OML_VER}{-mytestbed1.diff.gz,-mytestbed1.dsc,.orig.tar.gz} .; \
+  bunzip2 -c /tmp/collectd-4*.tar.bz2 | gzip > collectd.tar.gz; \
+  find ../p-rpm -type f -exec cp {} $PWD \;
+  tar czvhf obs.tar.gz *; \
+  cd ..
+}}}
+
+The build process should start automatically once the special tarball is uploaded.
+
+Packages should also be built for Fedora 14 (PLE) and Fedora 15 (InstaGENI). Christoph provides a script to be run on norbit, which logs on to two KVMs, fetches and compiles OML, OML apps and libtrace, creates RPM packages and uploads them to mytestbed.net.
+
+=== ArchLinux ===
+Submit the Arch package source file oml2-${OML_VER}-x.src.tar.gz, built at the same time as the binary package, to AUR (category lib).
+
+== Push to the upstream repository ==
+Any commits and tags created on the release branch and pakcaging branches should be pushed to the main mytestbed.net repository (not GitHub, this will be synced automatically).
+
+{{{
+$ git push ${REPO} release/${VERSION} \
+  archlinux/master archlinux/release/${VERSION} \
+  debian/master debian/release/${VERSION} \
+  rpm/master rpm/release/${VERSION} \
+  tag v${OML_VER} \
+  tag archlinux/v${OML_VER}-1 \
+  tag debian/v${OML_VER}-mytestbed1 \
+  tag rpm/v${OML_VER}-1 \
+  master
+}}}
+
+== Merge bug fixes to the 'master' branch ==
+Any patch level release, which fixes bugs, should be merged back to the 'master' branch, so that the bugs are fixed there as well:
+{{{
+$ git checkout master
+$ git merge release/${VERSION}
+}}}
+There will usually be a conflict in configure.ac due to the different version numbers, but it should be easy to resolve.
+
+== Prepare the 'master' branches for new work ==
+The Autoconf package version on the master branch should always be set to "2.<next>.pre0", where "2.<next>.0" would be the next planned feature release (yet to occur), and the "pre0" makes it glaringly obvious that the binary was built from the development branch, and is not an official release binary. For instance, if the most recent feature release was 2.4.x, then the master branch should report version 2.5.pre0. Once this change is commited, it should be tagged with "v2.<next>.pre", which will allow git-version-gen to properly generate development version number.
+{{{
+$ export NEXT_VERSION=${OML_MAJOR}.$((OML_MINOR+1)).0
+$ git checkout master
+$ git tag -a v${NEXT_VERSION}pre -m "OML v${NEXT_VERSION} development version" # On the first commit on master not on release/${VERSION}; as usual, it's also a good idea to sign the tag: -su PGPID
+$ git push master
+$ git push --tags
+$ git checkout archlinux/master
+$ git merge archlinux/release/${VERSION}
+$ git checkout debian/master
+$ git merge debian/release/${VERSION}
+$ git checkout rpm/master
+$ git merge rpm/release/${VERSION}
+}}}
+
+== Tell the world ==
+
+ 1. Finalize the release notes for that version
+ 1. Add a news item in the OML news
+ 1. Send an email to the oml-user mailing list (containing the ChangeLog and a link to the source)