Jenkins: yet another list of curated plug-ins

Jenkins is a Continuous Integration software, along the lines of BuildBot and Travis CI.

Continuous Integration software

For starters, here is a small comparison of these:

  • Jenkins is written in Java and released as a single .war binary. It is a full-featured server, and contains lots of features out-of-the-box, such as user accounts / credentials (login with username and password), security, plug-in management, logging...it could be potentially used for both single projects or as a single instance for lots of projects, although the first scenario seems to be the most common among its users [citation needed].
  • BuildBot is written in Python and at the time of this post 0.9.6 is its latest released version. As Jenkins, it is suited for both single projects or as a shared instance for lots of projects. However, if you happen to bother to set it up, it is better to use it for lots of projects, since it has a learning curve which is bigger than Jenkins, and also because it is slightly less automatic and requires manual installation -- you could say Arch Linux is to BuildBot what Ubuntu is to Jenkins.
  • Travis CI is probably more well-known among Github users. It is definitely used for single projects only; each travis project maps to one and only one Github repository. Its elegance lies in the fact all the configuration is written in a single .travis.yml file that lives inside the repository. These days its builds are done inside docker containers, which makes them idempotent (repeatable) and isolated from each other.

All of the 3 projects are good and solid; buildbot and jenkins are the oldest, and jenkins is the most popular (or at least used to be; these days, travis is getting more and more popularity through GitHub; and, well, GitHub is becoming more and more synonym with open source software, the same way 'Googling' became a buzz word for 'searching'). Buildbot recently got a rewrite (from its 0.8 to its 0.9 version) which made it slightly more buggy (its project has lots of issues on GitHub); hopefully this is temporary, however the reader is advised to proceed with caution if they [1] choose buildbot.

Plug-ins

Now here comes the best part. Jenkins is really stable and mature; however, it is like Emacs: it totally sucks without customization through plug-ins / add-ons. Vanilla / plain Jenkins seems to be something from the 2000s. That's why this post is here!

I've been spending the last days testing lots of Jenkins plug-ins that make it suck less. This is a small curated list of plug-ins that I believe to make a Jenkins installation more pleasant to work with and, even more important, more maintainable and reliable.

Don't blindly install these plug-ins just because I told you they are good. Read them, assess the real need for them, see if they are necessary in your use case.

Anyways, I recommend the ones marked as better design and essential.

[1] they as a wrapper for both he and she. These days (and every now and then) the English languague is getting lots of small tractions to modernize it.
[2] Some people will say this won't happen if you are hosting your Jenkins instance in the cloud. You can accept this argument, it is fair enough.

CMake: patterns and antipatterns - Part I

I've been deeply working with CMake during the last few months. It is a piece of software I am really proud and happy to work with; it is very well designed (warning: YMMV), maintained (supported) and runs in a lot of operating systems.

I figured out it would be a waste of time and knowledge not to document my findings and experience with it while my mind is still fresh, therefore I am starting this series to provide some guidelines, {anti,}patterns and -- of course -- opinionated views on how to use it well and/or avoid common annoyances [1].

Disclaimer: you will find lots of opinionated statements in this series. Although I will mostly make them explicit, I advise you to check out its official documentation if unsure.

Starting point: CMake version

CMake 2.8.12.2 is the latest release of the 2.8.x series. It is very solid and mature, and it is the last release included in Ubuntu 14.04 LTS.

CMake 3.0+ has lots of improvements and higher-level functions and modules to make things that used to be too repetitive and annoying in the 2.8.x series. On the other hand, lots of these improvements are not compatible with the 2.8.x series.

The first thing to be decided is which CMake version to use. A safe bet is to stick to 2.8.12.2; however, if there are no hard requirements, I would advise the 3.6.x sub-family. It is another family of solid and stable releases. Note: the latest released version of CMake at the time of this post is 3.8.x.

Another relevant point is that the 3.0x series documentation documentation is released with Sphinx, which makes it waaaaay more convenient to use than the older 2.8.x format.

Hello World project, and baby steps

Create the well-known hello.cpp file:

#include <iostream>

int main() {
    std::cout << "Hello world!" << std::endl;
    return 0;
}

And then a CMakeLists.txt file: 

cmake_minimum_required(VERSION 2.8.12)
project(hello)
add_executable(hello hello.cpp)

Build it as follows: 

mkdir build
cd build
cmake ..
make

You program will be called hello and will be created / placed in this build directory.

KNOL [2]: do out-of-source builds, by creating a directory for the sole purpose to store compilation artifacts, such as Makefiles, libraries and executables. You can always do a 

cmake . && make

However it is not advised (OPP [3]).

KNOL: A good function to keep in your shell is (warning: take care with the rm -rf part): 

t-cmake-clean() {
    local BUILD=$(basename $(pwd))
    cd .. && rm -rf $BUILD
    mkdir $BUILD && cd $BUILD
}

It should be run from within your build directory to regenerate it. Another common approach is to do a rm -rf * from within your build directory, which is also acceptable, however I personally prefer (OPP) the first approach.

KNOL: to introspect your binary, these are some commands you can run: 

file hello
readelf -d hello
ldd hello
objdump -p hello
LD_TRACE_LOADED_OBJECTS=1 ./hello
ldconfig

If you happen to be on SunOS / Solaris, there is also 

/usr/css/bin/dump -Lv ./hello

These are useful for (i) cross-compilation scenarios and (ii) whenever you are linking your binaries to shared libraries. They are also useful to check out the platform your binary has been compiled to.

KNOL: for debug purposes, you might want your build to be more verbose than the usual. There are two common approaches to do it.

The first one is at cmake time. 

cd build && cmake -DCMAKE_VERBOSE_MAKEFILE=ON ..
make

The second one is at compile time. 

cd build && cmake ..
make VERBOSE=1

I personally like the first one (OPP), however the second one is also OK.

KNOL: always keep an open browser tab with the documentation of the cmake version you're using. The Sphinx documentation format and, alas, the CMake documentation itself, are really good and you will be less error-prone if you RTFM. CMake (and build tools, for that matter) is not a tool to be used from the top of your head; it is really not advisable to do it (OPP).

Another resource for CMake documentation is DevDocs.

KNOL: StackOverflow is the best / fastest place where you can obtain documentation from CMake not found (or not easily found) in its Sphinx form. This is not a generic statement. Stack Overflow is good for lots of languages and tools, sure; however it sucks for lots of other resources too. My point here is that, from my prior experience™, it is an excellent place for CMake. Use the CMake tag whenever asking a new question!

The second best place to obtain information about cmake is from its official mailing list, however I don't recommend it for newbies (OPP), as it is more dense. CMake is maintained by the good folks at KitWare Inc. Keep that in mind if you intend to go to its mailing list; although it is open source project, it is backed by a company.

What's next?

There is still a lot to cover: dependencies, Find Modules, libraries, cross-compiling, basic debugging, properties, installations, and so on. This series is not intended to be a complete tutorial, so don't expect me to cover everything; it is only supposed to a priori cover / contain a collection of knols (tips) to make CMake development easier.

Contributing

I hope you found this post useful. If you spot any errors or have any suggestions to improve it, please leave a comment below or just email me privately (tbperrotta aatt gmail ddoott com) instead. This Contributing section will be included in all posts of this series, and will be removed after a few weeks. Thanks for reading, and until the next time.

Metablogging: metablogging is blogging about blogging. You might see a few metablogging snippets in this series. They will likely (though not always) disappear in the future, and it is my way of expressing out-of-topic (OOT) subjects in the middle of a post.

That said, this "will disappear" thing is probably an indirect influence from Snapchat and Instagram Stories. It is a good practice; content will not remain relevant forever, and it tends to lose relevance after a couple of days of weeks; the same thing happens with your money through inflation. When thinking about it, cleaning posts makes a lot of sense if you care about your future readers (and, heck, maybe even the future you).

Also, I really miss blogging, and this series is my way to slowly rollback to this world. Writing is a very good way of expressing thoughts and concerns. Although this thiagowfx.gitlab.io blog is still shining as it's kinda new, my old blog lives at https://thiagoperrotta.wordpress.com. I reactivated it and will now use it to post non-dev stuff.

Footnotes

[1] because most -- if not all -- software has some annoyances which we only learn how to overcome after lots of stumbling upon StackOverflow and SuperUser.
[2] KNOL: an unit of knowledge, aka pro-tip or customized koan, if you will.
[3] OPP: an opinionated statement. Yes, with a double p.

bash with vim editing mode

This post is a brief introduction on how to set-up vim editing mode in bash. It is intended to be a reference and a starting point, not a tutorial.

bashrc

vim $HOME/.bashrc

# Add the following line then:
set -o vi

Side effects

  • You will lose the ability to C-a, C-e, C-d and C-k (to move to the beginning and to the end of the current line, respectively, and to delete a character and kill-line). To "fix" that in insert mode:
# vim $HOME/.bashrc:
bind -m vi-insert "\C-a":beginning-of-line
bind -m vi-insert "\C-d":delete-char
bind -m vi-insert "\C-e":end-of-line
bind -m vi-insert "\C-k":kill-line

You might also want to fix these in normal mode (except C-k, which is already bound):

# vim $HOME/.bashrc:
# vi, vi-command, vi-move
bind -m vi-command "\C-a":beginning-of-line
bind -m vi-command "\C-d":delete-char
bind -m vi-command "\C-e":end-of-line
  • C-u, C-_, C-w, M-b will still work (why??? They are emacs keybindings!!! At least they are handy enough)
  • C-l (clear the screen) will only work in normal mode (<ESC> C-l) [1], but you can force it work on insert mode too:
# vim $HOME/.bashrc:
bind -m vi-insert "\C-l":clear-screen

It is also always a good idea to set your EDITOR and VISUAL variables:

# $HOME/.bashrc
# Set VISUAL if vim is in the system PATH.
command -v vim &>/dev/null && export VISUAL=vim
# Set EDITOR if $VISUAL is in the system PATH.
command -v "$VISUAL" &>/dev/null && export EDITOR="$VISUAL"

To edit the current command-line in your EDITOR [2], use:

  • C-x C-e if you are in emacs mode; or
  • v if you are in vi mode.

List all keybindings

Finally, one of the most useful commands for discoverability purposes:

bind -P

will list all current keybindings.

[1] https://unix.stackexchange.com/questions/104094/is-there-any-way-to-enable-ctrll-to-clear-screen-when-set-o-vi-is-set
[2] https://stackoverflow.com/questions/23230990/bashs-vim-mode-not-vi

virtualenvwrapper setup

virtualenvwrapper is a handy tool, used to manage virtualenvs.

virtualenvs are individual sandboxes that contain an isolated python installation and environment. You should always use a different virtualenv for every different python project that you have. virtualenvwrapper helps exactly with that.

Here's how to set it up

# Make sure you have an up-to-date version of pip.
pip install --upgrade pip

# Download virtualenv and virtualenvwrapper with pip.
pip install virtualenv virtualenvwrapper

# Set this variable to a suitable default.
export WORKON_HOME=$HOME/.virtualenvs

# Source virtualenvwrapper.sh.
# This step depends on your python installation.
# If you installed python with HomeBrew, then use the following path:
source /usr/local/bin/virtualenvwrapper.sh

You will likely want to add the last two commands to your shell init file (for example, .bashrc).

Working with virtualenvwrapper

  • To create a new virtualenv: mkvirtualenv <name>
  • To list all existing virtualenvs: lsvirtualenv
  • To activate it: workon <name>
  • To deactivate it: deactivate
  • To destroy it: rmvirtualenv <name>

Reference: http://exponential.io/blog/2015/02/10/install-virtualenv-and-virtualenvwrapper-on-mac-os-x/

makepkg not war

makepkg is the de-facto standard way to build packages in Arch Linux, called PKGBUILDs. Here are a couple of small recipes for it:

To build a package

makepkg -sir

Rationale:

  • -s: automatically installs missing dependencies
  • -i: installs the package after it is built
  • -r: removes build dependencies once the package is built; good for cleaning

To clean package leftovers

makepkg -code
find . -name '*.pkg.tar.xz' -delete

Rationale:

  • -c, -o, -d, -e: all the switches, used together, can be interpreted as clean all leftovers.
  • find: removes previously built packages in the current directory

tmux: copy and paste with the mouse

tmux is a terminal multiplexer. It works across all major Linux and Unix-like operating systems and provides a multitask experience in the command line.

One of the major frustrations I had over the years using it was that it didn't seem to be capable of mouse selection. This is a very common operation to do within a terminal emulator, especially when googling for errors originated from the output of a program.

This week I learned a very simple trick [1]: you just have to hold the SHIFT key while selecting a text region. Boom, that's it. You're welcome.

Reference: https://awhan.wordpress.com/2012/04/18/tmux-copy-paste-with-mouse/

[1] This has been tested on tmux 2.3.