Next let’s talk about:
- Enabling teams through technology
- Culture and leadership
As previously discussed we’re pretty keen on micro services at REA. Our delivery teams are organised around small, autonomous “squads” that get to choose pretty much any language and technology stack they wish to implement their solutions.
This inevitably leads to a fairly broad church of language use. 🙂
Previously at REA we’d had very special tools for Load and Performance testing that were quite expensive, very richly featured but completely disconnected from our every day development tools. The main outcome of this was that we ended up with a couple of engineers who were quite good at L & P testing with our enterprise tools while the majority of engineers found the barriers too great. We have moved to an approach which is far more inclusive and utilises many of the tools our engineers are working with on a daily basis. I’ll talk about how we did this for the most recent project I worked on.
So, you’re writing microservices! You’re feeling pretty smug, because microservices are all the rage. All the cool kids are doing it. You’re breaking up your sprawling monoliths into small services that Do One Thing Well. You’re even using consumer driven contract testing to ensure that all your services are compatible.
Then… you discover that your consumer’s requirements have changed, and you need to make a change to your provider. You coordinate the consumer and provider codebases so that the contract tests still pass, and then, because you’re Deploying Early and Often, you release the provider. Immediately, your monitoring system goes red – the production consumer still expects the provider’s old interface. You realise that when you make a change, you need to deploy your consumer and your provider together. You think to yourself, screw this microservices thing, if I have to deploy them together anyway, they may as well go in the one codebase.
AWS Custom Resources appeared on the scene around the beginning of the year and appeared to be the perfect remedy for the lack of comms going in and out of a CloudFormation stack. In a nutshell they fit nicely into your template as a snippet of JSON, allow you to pass variables in and out of your stack to 3rd parties and give you the ability to kick off external scripts during your create, update and delete stack operations.
The big win is that all the stuff that you’d otherwise wrap up in your deploy scripts as additional calls becomes a single atomic “create-stack” operation. All that logic for rolling back/updating/deleting is handled by CloudFormation meaning you don’t have to write all that logic yourself in bash or whatever you call the AWS CLI with.
How I learned to stop worrying and love bash scripting
It’s 3am and PagerDuty is waking you up. You just want to re-deploy an app because that’s the quickest way to get things going again and you like sleep. But wait – it turns out this deploy relies on a version of ruby you don’t have, the bundle won’t install because nokogiri is having problems and you wonder if gardening might have been a more rewarding career.
Bash scripting provides a simple way to get things done and avoids many of the above dramas. However, we hear that bash scripts are ok for really short things that can be tested manually and anything else is better left to a “Real Programming Language™” (whether ruby qualifies is left as an exercise for the reader).
bash-spec-2 to the rescue. With this nifty little testing framework you can TDD your way to a set of comprehensible, documented functions. Continue reading