Many teams at REA have adopted Scala, and we were keen to give it a go for our new backend web application. I wanted to talk about our experiences, and ultimately our decision to stick with Ruby (!).
Several months ago my team gained a new roadmap, with a new greenfield project. Our main language was Ruby, but we had been building and maintaining an older product across the full stack of frontend web, backend web, ops and native mobile (both iOS and Android). During the course of the former product, my team worked with the following components:
- A monolithic Ruby on Rails application (written circa 2012)
- Several small Rails microservices
- A mobile API gateway in Kotlin (also known as the mobile proxy)
- Native iOS app in Swift
- Native Android app in Kotlin
I was lucky enough to have worked on all components but the iOS app. So my first experience with a statically typed language was actually with Kotlin. I started learning Kotlin in a domain that I was familiar with; web development! So I first picked a few cards on the mobile proxy to try to get a feel for the language, rather than jumping straight into Android as I had never done mobile development before.
And here’s where it gets interesting, at least for me. Coming from a dynamic language background, I could see some great advantages:
- Intellij’s refactoring – (it’s better than just extract method which is more or less the extent you can do it with Ruby)
- The editor suggests more things, more appropriately
- Null checking (Ruby’s safe access modifier still has some rough edges, in my opinion)
- Importing from Gradle just works, there’s less fluffing around getting debugging and tests running in the IDE compared to Ruby
- Test times! So fast! (of course writing no tests is faster still…)
But I also missed some things from Ruby:
- Ecosystem seems less geared around reuse – I felt really spoilt with Ruby Gems, and very confident in extracting code into Gems as required. This seems murkier in the Kotlin / Scala / Java world, but it’s probably just my own unfamiliarity
- Various testing concepts like spies or mocks seem to require a lot more boiler plate
- Parsing and throwing JSON around (a core requirement of the BFF) required a lot more boiler plate with Moshi
- Those compile times!
Nevertheless, when my team formed we decided that we would spike Scala, as many other teams at REA are adopting it. We were hoping to get the following benefits from Scala compared to Ruby:
- Greater productivity through superior tooling and types
- Better web performance for clients
- Greater maintainability through embracing FP concepts
I want to address each of these in turn, and would love to hear whether you agree or think I’m completely wrong!
1. Theory: Greater Productivity through tools and types
Assessing developer productivity is a hard thing. There’s limits to what can be measured (in my opinion) – how do you concretely put metrics around whether a particular module is well designed? Sure, you can use techniques like static analysis and code review to check if the code quality is good, but these tools cannot tell you if your module is going to meet the business requirements (such as shipping quickly) or if it’s easy for other developers to understand (hello, Monads!).
At the end of the day, I reckon this comes down to tribal knowledge, the agreement within the team. We’ve all worked on projects where some parts of the codebase have been a pleasure to work in (the ones we wrote ourselves, amiright?), and others are complete nightmares of tangled spaghetti and shattered dreams (Rails code from 6 years ago written by a collection of dodgy contractors, Rails code written by me, or just Rails code in general).
One of the biggest advantages of statically typed languages is that they “eliminate a certain class of bugs by having a type system”, or words to that affect. While this is true to an extent, we found that we would end up writing tests to validate that the API gave us a particular response, i.e., Request spec, regardless of which language we used. So while at the unit level we may get away with fewer tests, in practice we didn’t feel like it translated concretely to a better final output. So yes, perhaps fewer unit tests in a Scala project due to the type system, but no greater confidence in delivery.
And regardless of the health of a codebase, having the right tools makes navigating and understanding things easier. For context, I use Intellij with Ruby. This lets me:
- Click to definition
- Open multiple tabs and splits as I need
- Rename things across multiple files, 90% of the time (10% it tries it’s best but gets it hopelessly wrong like my toddler eating spaghetti (protip, a bowl is not a hat, Hugo…))
- Debugging (without typing binding.pry and forgetting to remove it before shipping, ahem)
- Run all the tests
- Run selective tests
- Find by class
- Find by filename
- Refactor by extracting method or renaming variables
When we looked at Scala, we found it could do all of the above, but the only thing it could do better was renames across multiple files, and some additional refactoring options. So from a tooling perspective, we considered adopting Scala to be moot. This is likely to be contentious, call me out in the comments!
Note: if you’re a Ruby or Node developer and you don’t get debugging, method extracting, find by class or good test running in your editor, please, please look into Intellij. It continually surprises me how many devs use (snark alert!) toys like Atom, Vim or Sublime for their professional work.
2. Better web performance for clients
Scala wins. Actually we didn’t test this, but are pretty sure the JVM figuratively dances on Ruby’s grave, doing the Macarena, after successfully completing a Centurion while Ruby is left sobbing on the shitty Ikea sofa it regrets buying one rainy weekend because it is damned uncomfortable and has next to no resell value on Gumtree. Ahem.
3. Greater maintainability through embracing FP concepts
This is an interesting one. Ruby is not a functional language. In fact, as far as Object Oriented ones go, it is probably one of the strongest I’ve used. So naturally one would think that core language vs core language, Scala would dance on Ruby’s grave for a second time be a better experience. Yet, clearly Scala folks think that out-of-the-box, there is room for improvement. Miaow.
As a starting point with FP, we wanted to use the following things:
- Monads (I think I finally figured them out…)
- Immutability (in Ruby? lol)
- Partial functions / currying
So in our Spike, it became apparent that while we could do all these things, the sheer amount of boiler-plate code for a basic Graphql API was significant. At this point, as a team we decided that the benefits we had perceived to gain from Scala would probably not materialise in a way that was great enough to proceed with. After all, it required a substantial learning curve (investment) so there was an initial velocity hit, that we weren’t convinced we would reap benefits from later (payoff). Plus after the effort we had spent to get really fast deployment times by implementing Kubernetes, we thought throwing that away to sit and wait for SBT to (slowly) do it’s thing would be counter-productive.
But the story doesn’t end here folks! We thought we would try FP in Ruby. Whaaat?
Thanks to the excellent Dry RB project, we can have Monads and currying in Ruby too! Here’s some of the FP concepts we are trying to use with our current codebase:
- Partial functions
- Pure functions (wherever possible)
- The ability to say that we FP in Ruby and we like it (as opposed to “fp’ness”, please don’t ever use that made-up word in public again, whoever you are, random guy from Meetup)
Sure, I’m being a bit cheeky with this post, but in all seriousness, we evaluated Scala and decided it wasn’t a good fit for our team and project. We’ve been able to embrace FP concepts we’ve found useful in Ruby, and still deliver quickly, which is a great outcome for all concerned. I’d love to say a huge thank you to my team for giving this a red-hot go, enthusiastically doing awesome work, and patiently explaining FP concepts again and again until I finally demonstrated that I kinda understand what’s going on with this comment. You guys are absolute legends!