The task Here at Redbubble we’ve been running Ruby on Rails since day one. We’re a small development team, so keeping up with even the latest stable release has been a struggle. Earlier this year we had a gap in product development and took the opportunity to move our stack forward. We’d been on Ruby 1.8.7 and Rails 2.3 for a year or two. After some investigation we decided we’d first move to Rails 3.0 which would make Ruby 1.9 an easier option. After significant library updates and code compatibility changes, we were ready to release Rails 3.0 on Ruby 1.8.7.
Over the past few months the team had been getting enthused over the idea of a Hack Day and there had been much discussion on how to make it happen. Very few of us had first hand experience with the concept so taking the philosophy of walk before you run, we decided to do a small-scale run within the Engineering team before moving onto a bigger event. So we nominated a date, started listing some projects and come the day, ended up with three teams of two to get going. The day started with a brief kickoff session whereby each team gave an outline of their project and what they hoped to achieve, taking feedback from the rest of the
Recently we’ve been working on upgrading the version of Ruby the Redbubble site runs on, from 1.8.7 to 1.9.3. We’re doing this for a number of reasons, including improved performance, new language features, and trying to stay relatively current with our tech. Things went fairly smoothly until we hit a problem where we could get one of our rspecs to segfault (i.e. actually crash the ruby interpreter) every time we ran it on our (OSX Lion) development machines:
This is a tale of one of those times where it is actually useful to do code profiling and benchmarking. A bit of background: We’ve been in the process of upgrading our Ruby on Rails stack here at RedBubble, going from Rails 2.3 to 3.0, with an eye to moving forward to 3.2. Through this, to minimise the amount of change happening at once, we’ve stuck with Ruby 1.8.7, instead of jumping right to 1.9.3. When we rolled out Rails 3, we noticed a significant slowdown in the general performance of the site, even though the benchmarks we had run at the time indicated that it should be okay. It has been widely reported that Ruby 1.9.3 has in general,
Recently we noticed that one of our Solr slave processes (a piece of software we use to power the search feature at RedBubble) was taking longer to respond to search queries than usual – enough to trigger a warning alert on our monitoring systems. On first inspection, we saw that the machine was using a lot of memory – enough to go into swap space, but this was sitting around the same level as our other server, which was not experiencing problems. A little more digging, and we noticed, thanks to the JVM monitoring provided by NewRelic, that the Solr process was spending a lot of time (around 15% on average, but up to 50%) doing garbage collection. This means