If you use SBT, which you most probably do if you’re writing Scala repos, you’ve probably used the assembly fat-jar generator plugin for shipping and running your code. If this much is true, you’ve probably encountered merge conflicts and had to wrangle with a verbose build description and struggled...
Being purveyors of 3rd-Party JavaScript, sandboxing is something near and dear to our hearts. We take integrating with our publishers seriously, aiming to ensure a high-quality experience for their engineers and most importantly their users. jQuery has noConflict() and you’re gold; easy enough,...
This past Friday 11/15, Sharethrough Engineering attended the 2013 MLConf here in San Francisco where Netflix, Twitter, Yelp and others presented on large-scale ML trends and challenges.
Read on for thoughts of the day by a few members of the team - Michael Jensen, Michael Ruggiero and Ryan Weald
The big ball of mud is a well understood software anti-pattern, and we’ve all experienced it at some point in our careers (sometimes on the receiving end but all too often as the creator). There is never a specific event where we can look back and say, “this is what caused the problems we’re experiencing...
Testing code that depends on databases has always had twists and turns, but writing tests against large-scale key/value stores has its own set of challenges. If your app writes to Riak, should each developer’s machine have a fully-configured Riak instance? If the code writes to the data store using...
Here at Sharethrough, we’re big consumers of Twitter’s Scalding - a Scala-based DSL providing a functional programming abstraction on top of Cascading/Hadoop. It’s a terrific way to think through workflows and with Amazon’s Elastic MapReduce service, it couldn’t be easier to get started (well it...
This past weekend, we penned a piece over at GigaOM about the importance of integrating your data science folks into your engineering and product process.
For startups, data science should not be seen as a separate scientific initiative but as an integrated part of the product. Speed and efficiency...
We’ve been experimenting with Spark over the past few weeks. Our two main motivations for doing so were Spark’s excellent support for iterative algorithms and the new Spark streaming features for real-time data processing.
While we found the Spark APIs easy to work with, we did run into a few beginner...
It started innocently enough. We had a story to make a few UI changes to a small JavaScript app. There would be no functionality changes, so it was going to be an easy morning. Maybe even an early lunch? When we fired up Vim we saw that lunch was going to be a bit later than expected and that we...
Over the weekend we released the latest version of Elasticity, the gem we use to control our Amazon Elastic MapReduce flows. Two big features and a handful of incremental internal updates went in from 2.5 to 2.6 including job flow completion polling and debug support.
One of...
