The Salt community gathered for SaltConf in Salt Lake City this year at the Salt Palace. I’m not sure I’ve ever written the word Salt so many times in one sentence before, but I like the ring of it. This was my first year attending the event and it was a blast. I’ve been around the Python community for years, and Salt definitely gets community and events done right.
Each morning kicked off with a set of keynotes from major partners in the Salt ecosystem, such as Adobe, Suse Linux, Bloomberg and Intuit. A few notable announcements:
Nice, @Intuit has deployed over 30k @SaltStack minions and did it from ink’d deal to done in < 6mo #SaltConf17 pic.twitter.com/58DhU02NQi
— CalvinHP (@calvinhp) November 3, 2017
Gareth Greenaway (SaltStack) covered some great tools for Salt developers and users to streamline their workflow while using Salt. He published an example project that uses GitFS and branches for environments. It is up on GitHub at https://github.com/garethgreenaway/saltconf17 Using the GitHub repo, you can try out the following:
Authors of SaltStack components and states will appreciate the following tools mentioned in his presentation:
C.R. Oldham (SaltStack) went over creating custom Salt modules. The code from his presentation is also on GitHub at https://github.com/cro/saltconf17. Interestingly, when building your modules, __virtual__ is not required, but is recommended, and you can use saltutil.sync_all to bring custom modules into the Salt cache.
C.R. also showed off one of his tools for automating his workflow by watching changes to states etc., and running them in real time by creating a quick shell script called sdc to call the states locally like this (aka the Salt Development Command):
#!/bin/bash
salt-call --local "$@"
You can combine that with the watchdog Python package to monitor the files in a directory and run them as they are changed.
C.R. Oldham (SaltStack) also gave a great talk on using salt-proxy to control devices that can’t run a minion. Here is the code from the presentation: https://github.com/cro/saltconf17/tree/master/SaltStack_Network_Edge_Device There is a sample REST endpoint code here that was used in the presentation: https://github.com/saltstack/salt-contrib As a quick note, one process per controled device each uses about 40MB RAM and each device currently requires its own proxy process.
Stephan Looney and Tyler Jones (Sterling Talent Solutions) showed how they have greatly reduced the time needed to patch their systems. By leveraging Salt’s power with the nice easy UI of SaltStack Enterprise (SSE), they were able to empower other administrators (who had no idea what a state, pillar or orchestration were) to be able to perform common functions, such as patching. Essentially, it’s possible to fire custom events to run orchestrate via the GUI and let SaltStack do all of the heavy lifting. No more weekend patching party conference bridges!
It is no surprise that Tom Hatch (SaltStack) loves plugins: they are, along with async, a core design principle of SaltStack. During the presentation, Tom went over many of the core architecture pieces that make up Salt.
One interesting new component of Salt that I’ve not tried is Thorium. It is a flow-based programming concept to do things that reactors alone can’t. Thorium allow you to watch for trends and react based on thresholds.
This was probably one of my favorite sessions of the conference. Charles Ruffino (SaltStack) demonstrated building a serverless AWS Lambda-like service using Salt’s event bus and Docker. It combines reactors and the docker event engine to spin up and run functions in containers. You can check out the demo from GitHub at https://github.com/SaltyCharles/saltconf/tree/master/17/serverless
Benjamin Allen (BlueTarp Financial) introduced the idea of SecDevOps to the crowd and how they implement it at their organization using the following components:
One interesting fact is that, in 85% of the time, breaches are caused by not patching software. At BlueTarp Financial, they do not patch running systems: they torch the old ones and replace them with fresh ones from their newly built images.
Alexander Thaller (trivago) showcased a scheme that was inspired by Puppet’s “Roles and Profiles” for managing configs across their environments and applications. The main components are states that define:
With these building blocks in place, it’s possible to use Salt grains to assign minions to various environments and roles for applications. With profiles, formulas can be combined together and included in multiple roles as reusable components.
Alexander’s Salt pro-tip: you can move your formulas around and the file path won't break if you include files using this pattern: salt['file.join'](tpldir,"map.jinja")
Alexander has built some cool utilities for use in Salt such as salt-compressor to minimize output noise. He also created a saltstack_helper that looked like it used Graphviz to provide graphic visualization of the salt state dependancies.
Nice quick tips from Megan Wilhite (SaltStack) on debugging Salt:
Install setproctitle for extra awesomeness on the master. It will now change the process title to show what function the Python process is performing.
Use the following flags to get more legible info when debugging master and minion runs:
$ salt-call -ldebug $ salt-run state.event pretty=True
For Python debugging, here are some great options for diving deeper:
log.error()print()Inside Jinja templates, all of the data available can be viewed using a snippet that shows the dictionary of the full context from the viewpoint of the template: {% do salt.log.error(show_full_context()) %}
Daniel Wallace (SaltStack) showcased using Kitchen Salt for automated testing of salt states. Kitchen Salt uses Ruby and Bundler to set up a test runner for building and asserting that your states run correctly. To use Kitchen Salt, you configure a driver, which supports lots of platforms such as FreeBSD, most Linux distributions and Windows. You then set it up to run your suites, and you can pass custom bootstrap options to SaltStack to it matches your current process of installation. From a Pythonista standpoint, you can use pytest to build your tests. Daniel’s pro-tip: use things like the ssh driver against clones of prod machines to refine your states before applying them.
The last session of the conference was a Q&A with Tom Hatch. The crowd was able to ask Tom just about anything they wanted. The questions and stories told ranged from how Salt got its name (from a scene in The Lord of the Rings) to a Star Wars face-off between Leia and Padmé. I asked about Salt’s support of the fileserver environments and if it is really used in production by others: it sounds like using grains might be a better option.
I’m already looking forward to next year’s conference which will be September 10-14th, 2018 in Salt Lake City. Let me know if you’re thinking about going too!
Six Feet Up, Inc.
11208 Windermere Blvd.
Fishers, IN 46037 - USA
Phone: +1 (317) 861-5948
Email: info@sixfeetup.com
