The Ops Dojo

No doubt that the term Devops is probably one of the most worn-out concepts in IT these days, maybe only slightly overtaken by the emerging buzzword of the moment: docker, docker, docker. But from the vast area and topics that Devops can cover, the one that attracts me more lately is how to build the Operations culture in an organization: bringing down the walls, sense of ownership, capability to operate a service/system, etc.

In REA we are quite proud of the Devops culture in the organization and we work really hard towards creating highly autonomous cross-functional teams that can operate efficiently end to end. This includes the ownership and operation of the services and products the team builds and maintains. This vision of Operations as a capability/responsibility in a team, rather than a separate team or static role of the engineers, has been providing really good results since we started championing it a few years ago, therefore the willingness to commit to it. There is probably not a single way to boost this capability. Sometimes we achieve it  by embedding an Ops specialist in the team, whose mission and passion is to enable and train the rest of the team, but which in general means that we are committed to invest in raising the technical capability and awareness of all our engineers and providing them with the level of access to our systems/data/services to do their job in an efficient manner.

In this post I am going to focus in one of the ways that we are trying to build/improve that Operations capability across the company based on running a set of community driven activities that we have called the Ops Dojo. The idea behind the concept is not new for REA, we already had several ad-hoc training sessions, plenty of brownbags and a strong culture of sharing what we learn across the organization. The Ops Dojo is just an initiative to have even more of that and to build a community that follows it up and makes it happen regularly. The Ops Dojo is just one form of the guild initiatives which are on the rise at REA as a mechanism to share knowledge and passion and today covers such diverse topics as Security, Public speaking, Delivery Engineering, Agile, Happiness, etc…

Sure, but what the hell is an Ops Dojo?

It’s a community driven regular event to practice and improve our skills around Web Operations. We all love to attend meetups and brownbags, chat with colleagues from other companies about what they are doing and get inspired but, if you really want to learn something in depth, the best way of doing it is through real practice. Nothing revolutionary here, watching a soccer game is not the same as going out for a kick.

ReaOpsDojo1

How is this different to formal training? First of all it is normally fueled by the community, for each session we have one or multiple senseis leading the session. This is just someone from the company that is passionate about a topic and is happy to lead the session or share their knowledge/discoveries. The rest of the attendees follow his guidance and practice a skill. These activities are not meant to replace formal training but they complement it, as sometimes it is good to have an specialist from outside the company or colleagues from other companies providing the training.

Who comes along to the Dojo sessions? Anyone who is interested in the topic of the session, wants to practice or improve a skill and is keen to have some fun. We still have a majority of attendees from the Site Performance and Reliability roles but we always have people coming from the Developers, QAs and Tech Leads roles. Our objective is to grow the attendance in those groups and that’s why we always try to include topics that can be appealing for them and 101 sessions.

Ok, got it. Tell me more about the Dojo sessions…

Well, they can follow different patterns but the main idea is that they have to be practical and hands on. One of my favorite versions are the sessions where we practice katas , where instead of having a single sensei, multiple people will show different techniques and tricks while the rest get the chance to practice and learn from multiple experiences. So far we have run three kata sessions: an ssh kata and, a bash scripting kata and a splunk kata. It’s quite rewarding to see what a really smart group of people can do with those tools and how they can be used in ways that you probably haven’t even thought.

IMG_1544

We had a good turnout when our colleagues presented on hot topics and got our hands dirty with new tools like Ansible and Docker. It doesn’t take much time to see how new projects and ideas come to life using tools and techniques that may have been practiced in the Ops Dojo for the first time. For example in the network troubleshooting Dojo we had a good dive from the very basics of the troubleshooting techniques to more advanced problems related to networking using tools like tcpdump and wireshark.

When nobody has time to prepare a formal session we run an Ops Cafe session where we chose a series of topics that get voted and we discuss them in batches of 10 minutes. After that time we vote if we want to continue discussing or we want to move to a new topic. Some of the ideas discussed end up in Dojo sessions later on and some or them are really good food for thought.

OpsCafe

Other possible formats can also include Bork Games, hacking sessions, capture the flag, etc..

What are the next steps for the Ops Dojo?

After one year of running the Dojo we hosted recently a session to talk about the future of the Ops Dojo. The feeling in general was that everybody enjoyed it quite a lot during the year and that it was a useful activity across the board. One of the main focuses is going to be to try to attract even more people from other functional areas and for that we came up with some ideas like repeating the most interesting sessions, as we have lots of new engineers that have joined the company that may have not attended the previous one, or running sorter sessions just to see what topics attract more interest from the other communities.

IMG_1543

Some of the challenges we face in the future are opening the dojo to our international teams across the world and maybe giving the chance for people outside of REA to benefit from these sessions. Have I already mentioned that we love sharing at REA? We are thinking about recording the sessions or having some guests that want to join and practice with us.

Let’s discuss!

I hope that you found the concept interesting, but I have a few questions for you to help us with some feedback:

  • Why not running your own Ops Dojo at your current workplace?
  • Would you be interested in attending one of our sessions?
  • Any ideas to make this more efficient?
  • Do you have something similar or other interesting approaches in your organisation?
  • Would you be interested in a recorded version of the sessions?

Please post your ideas ideas in the comment section, I’m looking forward to know your opinion on the topic.

  • epiineg1

    Nice.

  • Nestor Pina

    Sounds great. I think this kind of initiatives really boost employee’s engagement and the general knowledge in the company. We’re starting to implement a Guilds-like groups in our company (as in Spotify or REA as you mentioned) so we have our hands full for the moment, but this idea seems pretty awesome too. Just a couple questions, when do you have the meetings? How long do they take?

    • Casey

      Each guild decides on their frequency. Attendance is optional and guild meetings usually go for an hour.

    • jturegano

      Hey Nestor, thanks for the comment and good luck implementing it in your current workplace!

      As Casey mentioned it depends, but most of the guild/dojo sessions are scheduled during business hours. I think it is a really good way to recognize the need for knowledge sharing as in many cases the elimination of a silo creates a new different kind of silo, and in this case this is a great way of mitigating the new silos by opening “knowledge doors” between teams and areas. In the specific case of the Ops Dojo we have scheduled most of the sessions for at least one to one and half hours and normally at the end of the day just in case people want to stay longer practicing their skills, but we are looking at trying with shorter sessions.

  • Daniel Alonso

    Great post Javier! I hope I can apply some of these techniques at mi current company!

    Thanks for the sharing!

    • jturegano

      No problem, good luck with it!

  • Fazle A

    Recorded versions of the Dojos on your Youtube channel would be nice. Google Hangout on Air is also a possibility.