I have been browsing the internet for blogs or articles to help Chef developers have a way of promoting the code of their cookbooks, a way of vetting code, and avoiding that code goes from Operations guys straight to production.
I have found a lot of theoretical articles on building a CI pipeline for chef cookbooks, but not a lot of practical ones, so I decided to make a proof of concept for the public and my team as well.
when it comes to integration tools, I like Jenkins, it’s open source and the community is very active in adding and updating plugins .
In this example I will use a Java cookbook as a code base, and I will be running 4 test :
- Foodcritic : a helpful lint tool you can use to check your Chef cookbooks for common problems. It comes with 61 built-in rules that identify problems ranging from simple style inconsistencies to difficult to diagnose issues that will hurt in production.
- ChefSpec : a unit testing framework for testing Chef cookbooks. ChefSpec makes it easy to write examples and get fast feedback on cookbook changes without the need for virtual machines or cloud servers.
- Rubocop : a Ruby static code analyzer. Out of the box it will enforce many of the guidelines outlined in the community Ruby Style Guide.
- Test Kitchen : a test harness tool to execute your configured code on one or more platforms in isolation. A driver plugin architecture is used which lets you run your code on various cloud providers and virtualization technologies such as Amazon EC2, Vagrant, Docker, LXC containers, and more. Many testing frameworks are already supported out of the box including Bats, shUnit2, RSpec, Serverspec, with others being created weekly.
of course you can have all these tools in one package…. the famous ChefDK
Code Promotion :
The concept of code promotion helps the CI process distinguish between good and bad builds, I like to define a good build as a build where ALL the tests are successful .
Jenkins helps you implement this concept with a community plugin : Promoted Build Plugin
based of the status of your build ( promoted of not) you can control the code that goes into your repository (github or gitlab), you can use set up hooks to deny merge requests from builds that are not promoted.
Jobs:
let’s setup our jobs, we will too categories of jobs :
- Build Jobs
- Test Jobs
whenever a build job is successful will trigger all the test jobs to start.
Build-java-cookbook : will clone the code repo and create a temporary artifact, this is the config section for this job
Rubocop Test : will copy the temporary artifact decompress it to have all code repo and run rubocop test on the code :
ChefSpec Test :
FoodCritic :
Test Kitchen :
Test Kitchen will spin a vagrant box (ubuntu-14.04) and run the cookbook on it and test the results.
First Run :
with the configurations above we run the build and test jobs :
Result :
Rubocop test failed, by looking at the execution log we can see why :
+ rubocop
Inspecting 44 files
.......CC...CC........C..C..................
Offenses:
providers/alternatives.rb:34:38: C: Use shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path} | grep 'priority #{priority}$'").exitstatus.zero? instead of shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path} | grep 'priority #{priority}$'").exitstatus == 0.
alternative_exists_same_prio = shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path} | grep 'priority #{priority}$'").exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:35:28: C: Use shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path}").exitstatus.zero? instead of shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path}").exitstatus == 0.
alternative_exists = shell_out("#{alternatives_cmd} --display #{cmd} | grep #{alt_path}").exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:43:18: C: Use remove_cmd.exitstatus.zero? instead of remove_cmd.exitstatus == 0.
unless remove_cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:57:18: C: Use install_cmd.exitstatus.zero? instead of install_cmd.exitstatus == 0.
unless install_cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:66:28: C: Use shell_out("#{alternatives_cmd} --display #{cmd} | grep \"link currently points to #{alt_path}\"").exitstatus.zero? instead of shell_out("#{alternatives_cmd} --display #{cmd} | grep \"link currently points to #{alt_path}\"").exitstatus == 0.
alternative_is_set = shell_out("#{alternatives_cmd} --display #{cmd} | grep \"link currently points to #{alt_path}\"").exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:72:16: C: Use set_cmd.exitstatus.zero? instead of set_cmd.exitstatus == 0.
unless set_cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^^^^^
providers/alternatives.rb:87:50: C: Use cmd.exitstatus.zero? instead of cmd.exitstatus == 0.
new_resource.updated_by_last_action(true) if cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^
providers/ark.rb:39:20: C: Omit parentheses for ternary conditions.
package_name = (file_name =~ /^server-jre.*$/) ? 'jdk' : file_name.scan(/[a-z]+/)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
providers/ark.rb:134:14: C: Use 0o for octal literals.
mode 0755
^^^^
providers/ark.rb:149:14: C: Closing method call brace must be on the line after the last argument when opening brace is on a separate line from the first argument.
))
^
providers/ark.rb:150:16: C: Use cmd.exitstatus.zero? instead of cmd.exitstatus == 0.
unless cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^
providers/ark.rb:157:16: C: Use cmd.exitstatus.zero? instead of cmd.exitstatus == 0.
unless cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^
providers/ark.rb:164:16: C: Use cmd.exitstatus.zero? instead of cmd.exitstatus == 0.
unless cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^
providers/ark.rb:172:14: C: Use cmd.exitstatus.zero? instead of cmd.exitstatus == 0.
unless cmd.exitstatus == 0
^^^^^^^^^^^^^^^^^^^
recipes/ibm.rb:44:8: C: Use 0o for octal literals.
mode 00755
^^^^^
recipes/ibm_tar.rb:36:8: C: Use 0o for octal literals.
mode 00755
^^^^^
recipes/ibm_tar.rb:49:8: C: Use 0o for octal literals.
mode 00755
^^^^^
recipes/set_java_home.rb:27:8: C: Use 0o for octal literals.
mode 00755
^^^^^
recipes/set_java_home.rb:32:8: C: Use 0o for octal literals.
mode 00755
^^^^^
resources/ark.rb:39:54: C: Use 0o for octal literals.
attribute :app_home_mode, kind_of: Integer, default: 0755
^^^^
44 files inspected, 20 offenses detected
Build step 'Execute shell' marked build as failure
Finished: FAILURE
let’s go ahead and fix these offenses and commit the code :
we restart the build :
and here everything is green this time :
from this point on you can create do two things :
- save your cookbook in a private supermarket with a corresponding version number
- upload this cookbook to chef server
Promotion status :
After the completion of all tests, this build can now be promoted.
