Here at ScraperWiki we’re all about making your lives easier (and our lives harder!). We want to help you help yourselves (and in the process make our lives easier!). Part of that means letting you collaborate and share nifty pieces of code to help build and maintain your scrapers.
So welcome to the users corner, where every week (hopefully!) we’ll feature one of you and what you do in a sort of who-should-know-who on our site. Today we welcome Robert Brook.
From the Register of Hereditary Peers to NASA’s Picture of the Day, Robert has a rather eclectic bunch of scrapers. Like most of you I imagine, he uses ScraperWiki for work and play. So for anyone who thinks this is a handy work tool for, let’s say, getting Small E-Money Issuer Certificates for a financial consultancy firm, why not fool around in your free time with BeeHive Hockey Pool Standings. For those of you who don’t use ScraperWiki for work, you should quit the day job!
“It’s made my life considerably easier – and allowed me to consider some data that previously I wouldn’t have tried scraping.”
– Robert Brook
He works in the UK Parliament and uses ScraperWiki for prototyping work. The great thing about having your scraper on a community site is finding what others are doing. Robert says he also finds all the scrapers with the ‘parliament’ tag interesting as it gives him a clue about what information people actually want out of particular websites.
He’s a dual Ruby-Python programmer and if you’re new to scraping it’s worthwhile checking out his scrapers as he manages to get a lot with very little code. In fact, he’s written around lines 1,300 of code, saving 23,615 rows of data to the datastore with 28 scrapers.