Category Archives: developer

It’s good to share…

As you may have gathered I’m on a journey, I’ve worked as a physicist, a data scientist for 20 years and now I’ve fallen amongst software engineers. There are obvious similarities in what we do, we write code to do … Continue reading

Posted in developer | Tagged | Leave a comment

Mastering space and time with jQuery deferreds

Recently Zarino and I were pairing on making improvements to a new scraping tool on ScraperWiki. We were working on some code that allows the person using the tool to pick out parts of some scraped data in order to … Continue reading

Posted in developer, Tools | Tagged , , , | Leave a comment

9 things you need to know about the “Code in your browser” tool

ScraperWiki has always made it as easy as possible to code scripts to get data from web pages. Our new platform is no exception. The new browser-based coding environment is a tool like any other. Here are 9 things you should know about … Continue reading

Posted in developer | 2 Comments

Book Review: Clean Code by Robert C. Martin

Following my revelations regarding sharing code with other people I thought I’d read more about the craft of writing code in the form of Clean Code: A Handbook of Agile Software Craftmanship by Robert C. Martin. Despite the appearance of … Continue reading

Posted in developer | Tagged | Leave a comment

npm install urchin

Urchin, the shell testing framework for extreme hipster superheroes (I’m not including myself in that group I should add), is now available as an npm package. That means you can install it using npm: sudo npm install -g urchin If … Continue reading

Posted in developer | Tagged , , | Leave a comment

10 technical things you didn’t know about the new ScraperWiki

1. Scrapers are now completely language neutral. Not just Python and Ruby – but anything open source that can make or read an SQLite file, from R to Clojure. 2. Scrapers can have as many files as they like. So … Continue reading

Posted in beta, developer | 1 Comment

Quick start guide: Make your first data tool

The new ScraperWiki is all about tools, like a hammer rather than a washing machine. For data scientists and developers we’ve made a new quick start guide, that takes you through making your first tool. We’re looking forward to seeing … Continue reading

Posted in beta, developer, Tools | 1 Comment